BROWSE
Detail
Journal Article
EdgeQ-GEMM: A Single-Engine Bit-Partitioned GEMM Accelerator With Integrated On-the-Fly Quantization for Edge AI Processors
Cited - time in 
Share 
Share
- Authors
- Chaebin Jung, Kyeongwon Lee, Sechan Park, Jongin Choi, Jae-Jin Lee, Woojoo Lee
- Citation
- IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Early Access
- ISSN
- 1063-8210
- Publisher
- IEEE
- Language
- English
- Type
- Journal Article
- Abstract
- Continuous-sensor edge AI workloads, such as keyword spotting (KWS) and human-activity recognition, process streaming inputs under tight power and area constraints. Many integrated on-the-fly (OTF) edge accelerators couple high-precision activation conversion to a fixed-precision GEMM engine, whereas many precision-flexible accelerators operate on operands already represented in low-precision integer formats. EdgeQ-GEMM combines these two design domains by integrating FP32/INT32-to-A8 activation conversion with a precision-flexible, bit-partitioned GEMM datapath. Within the tile-streaming datapath, the GEMM engine directly consumes either the full 8-bit activation code or its MSB-derived 4-bit slice, while the dequantization path receives the associated scale and compensation metadata without software-side quantization preprocessing or intermediate quantized-activation SRAM materialization. A single fixed-throughput engine thereby supports W8 × A8, MSB-derived W8 × A4-slice, and W4 × A8 execution without duplicated precision-specific GEMM engines or widened external interfaces. The MSB-derived A4-slice mode uses the scale and offset/compensation parameters associated with the A8 code and is not an independently quantized INT4 activation mode. The same architecture supports an adaptive streaming mode that selects between A8 and the MSB-derived A4 slice, as well as a layerwise mixed-precision quantization (MPQ) mode using W4/W8 weights and OTF-quantized A8 activations. Implemented in a Rocket-based edge processor and evaluated through FPGA prototyping and 45-nm synthesis, EdgeQ-GEMM reduces OTF-GEMM accelerator energy by 14.8%–27.0% at the energy-oriented adaptive setting. At the accuracy-oriented setting, the accuracy loss remains at or below 1.0 percentage point across the evaluated KWS and human-activity recognition (HAR) workloads. For ResNet-50/ImageNet layerwise MPQ, it reduces OTF-GEMM accelerator energy by up to 11.4% relative to a Dual-OTF-GEMM baseline while preserving the accuracy of the same MPQ assignment.
- Keyword
- Bit-partitioned datapath, edge AI processor, low-power DNN acceleration, mixed-precision GEMM (MP-GEMM) accelerator, on-the-fly (OTF) quantization
- KSP Keywords
- Accuracy loss, Area constraints, Compensation parameters, FPGA prototyping, High-precision, Low Power, Low-precision, On-the-fly, Streaming datapath, adaptive streaming, human activity recognition