BROWSE
Detail
Journal Article
QuantuneV2: Compiler-based local metric-driven mixed precision quantization for practical embedded AI applications
Cited 0 time in 
Share 
Share
- Authors
- Jeongseok Kim, Jemin Lee, Yongin Kwon, Daeyoung Kim
- Issue Date
- 2025-05
- Citation
- Future Generation Computer Systems, v.166, pp.1-15
- ISSN
- 0167-739X
- Publisher
- Elsevier BV
- Language
- English
- Type
- Journal Article
- Abstract
- Mixed-precision quantization methods have been proposed to reduce model size while minimizing accuracy degradation. However, existing studies require retraining and do not consider the computational overhead and intermediate representations (IR) generated during the compilation process, limiting their application at the compiler level. This computational overhead refers to the runtime latency caused by frequent quantization and de-quantization operations during inference. Performing these operations at the individual operator level causes significant runtime delays. To address these issues, we propose QuantuneV2, a compiler-based mixed-precision quantization method designed for practical embedded AI applications. QuantuneV2 performs inference only twice – once before quantization and once after quantization – and operates with a computational complexity off O(n) that increases linearly with the number of model parameters. We also made the sensitivity analysis more stable by using local metrics like weights, activation values, the Signal-to-Quantization-Noise Ratio (SQNR), and the Mean Squared Error (MSE). We also cut down on computational overhead by choosing the best IR and using operator fusion. Experimental results show that QuantuneV2 achieved up to a 10.28% improvement in accuracy and a 12.52% increase in speed compared to existing methods across five models: ResNet18v1, ResNet50v1, SqueezeNetv1, VGGNet, and MobileNetv2. This demonstrates that QuantuneV2 enhances model performance while maintaining computational efficiency, making it suitable for deployment in embedded AI environments.
- KSP Keywords
- AI Applications, Computational Efficiency, Computational complexity, Embedded AI, Local metrics, Mixed precision, Model parameter, Model performance, Sensitivity analysis, de-quantization, intermediate representation