BROWSE
Detail
Journal Article
Transformer를 활용한 인공신경망의 경량화 알고리즘 및 하드웨어 가속 기술 동향
- Issue Date
- 2023-10
- Citation
- 전자통신동향분석, v.38, no.5, pp.12-22
- ISSN
- 1225-6455
- Publisher
- 한국전자통신연구원
- Language
- Korean
- Type
- Journal Article
- Abstract
- The development of neural networks is evolving towards the adoption of transformer structures with attention modules. Hence, active research focused on extending the concept of lightweight neural network algorithms and hardware acceleration is being conducted for the transition from conventional convolutional neural networks to transformer-based networks. We present a survey of state-of-the-art research on lightweight neural network algorithms and hardware architectures to reduce memory usage and accelerate both inference and training. To describe the corresponding trends, we review recent studies on token pruning, quantization, and architecture tuning for the vision transformer. In addition, we present a hardware architecture that incorporates lightweight algorithms into artificial intelligence processors to accelerate processing.
- KSP Keywords
- Convolution neural network(CNN), Hardware Acceleration, Network Algorithm, artificial intelligence, hardware architecture, memory usage, neural network(NN), state-of-The-Art, transformer-based
This work is distributed under the term of Korea Open Government License (KOGL)
(Type 4: : Type 1 + Commercial Use Prohibition+Change Prohibition)
(Type 4: : Type 1 + Commercial Use Prohibition+Change Prohibition)
