ETRI-Knowledge Sharing Plaform

KOREAN
논문 검색
Type SCI
Year ~ Keyword

Detail

Journal Article Transformer를 활용한 인공신경망의 경량화 알고리즘 및 하드웨어 가속 기술 동향
Cited - time in scopus Download 213 time Share share facebook twitter linkedin kakaostory
Authors
김혜지, 여준기
Issue Date
2023-10
Citation
전자통신동향분석, v.38, no.5, pp.12-22
ISSN
1225-6455
Publisher
한국전자통신연구원
Language
Korean
Type
Journal Article
DOI
https://dx.doi.org/10.22648/ETRI.2023.J.380502
Abstract
The development of neural networks is evolving towards the adoption of transformer structures with attention modules. Hence, active research focused on extending the concept of lightweight neural network algorithms and hardware acceleration is being conducted for the transition from conventional convolutional neural networks to transformer-based networks. We present a survey of state-of-the-art research on lightweight neural network algorithms and hardware architectures to reduce memory usage and accelerate both inference and training. To describe the corresponding trends, we review recent studies on token pruning, quantization, and architecture tuning for the vision transformer. In addition, we present a hardware architecture that incorporates lightweight algorithms into artificial intelligence processors to accelerate processing.
KSP Keywords
Convolution neural network(CNN), Hardware Acceleration, Network Algorithm, artificial intelligence, hardware architecture, memory usage, neural network(NN), state-of-The-Art, transformer-based
This work is distributed under the term of Korea Open Government License (KOGL)
(Type 4: : Type 1 + Commercial Use Prohibition+Change Prohibition)
Type 4: