BROWSE
Detail
Journal Article
PartitionTuner: An operator scheduler for deep‐learning compilers supporting multiple heterogeneous processing units
- Authors
- Misun Yu, Yongin Kwon, Jemin Lee, Jeman Park, Junmo Park, Taeho Kim
- Issue Date
- 2023-04
- Citation
- ETRI Journal, v.45, no.2, pp.318-328
- ISSN
- 1225-6463
- Publisher
- 한국전자통신연구원
- Language
- English
- Type
- Journal Article
- Abstract
- Recently, embedded systems, such as mobile platforms, have multiple processing units that can operate in parallel, such as centralized processing units (CPUs) and neural processing units (NPUs). We can use deep‐learning compilers to generate machine code optimized for these embedded systems from a deep neural network (DNN). However, the deep‐learning compilers proposed so far generate codes that sequentially execute DNN operators on a single processing unit or parallel codes for graphic processing units (GPUs). In this study, we propose PartitionTuner, an operator scheduler for deep‐learning compilers that supports multiple heterogeneous PUs including CPUs and NPUs. PartitionTuner can generate an operator‐scheduling plan that uses all available PUs simultaneously to minimize overall DNN inference time. Operator scheduling is based on the analysis of DNN architecture and the performance profiles of individual and group operators measured on heterogeneous processing units. By the experiments for seven DNNs, PartitionTuner generates scheduling plans that perform 5.03% better than a static type‐based operator‐scheduling technique for SqueezeNet. In addition, PartitionTuner outperforms recent profiling‐based operator‐scheduling techniques for ResNet50, ResNet18, and SqueezeNet by 7.18%, 5.36%, and 2.73%, respectively.
- KSP Keywords
- Deep neural network(DNN), Graphic processing unit(GPU), Heterogeneous processing, Machine code, Mobile platform, Neural processing, Scheduling plans, Scheduling technique, centralized processing, embedded system, neural network(NN)
This work is distributed under the term of Korea Open Government License (KOGL)
(Type 4: : Type 1 + Commercial Use Prohibition+Change Prohibition)
(Type 4: : Type 1 + Commercial Use Prohibition+Change Prohibition)
