BROWSE
Detail
Conference Paper
MPViT: Multi-Path Vision Transformer for Dense Prediction
Cited 185 time in 
Share 
Share
- Authors
- Youngwan Lee, Jonghee Kim, Jeffrey Willette, Sung Ju Hwang
- Issue Date
- 2022-06
- Citation
- Conference on Computer Vision and Pattern Recognition (CVPR) 2022, pp.7287-7296
- Language
- English
- Type
- Conference Paper
- Abstract
- Dense computer vision tasks such as object detection and segmentation require effective multi-scale feature representation for detecting or classifying objects or regions with varying sizes. While Convolutional Neural Networks (CNNs) have been the dominant architectures for such tasks, recently introduced Vision Transformers (ViTs) aim to replace them as a backbone. Similar to CNNs, ViTs build a simple multi-stage structure (i.e., fine-to-coarse) for multi-scale representation with single-scale patches. In this work, with a different perspective from existing Transformers, we explore multi-scale patch embedding and multi-path structure, constructing the Multi-Path Vision Transformer (MPViT). MPViT embeds features of the same size (i.e., sequence length) with patches of different scales simultaneously by using overlapping convolutional patch embedding. Tokens of different scales are then independently fed into the Transformer encoders via multiple paths and the resulting features are aggregated, enabling both fine and coarse feature representations at the same feature level. Thanks to the diverse, multi-scale feature representations, our MPViTs scaling from tiny (5M) to base (73M) consistently achieve superior performance over state-of-the-art Vision Transformers on ImageNet classification, object detection, instance segmentation, and semantic segmentation.
- KSP Keywords
- Computer Vision(CV), Convolution neural network(CNN), Feature level, Feature representation, Multi-scale feature, Multi-scale patch, Multi-stage, Object detection, Semantic segmentation, Sequence length, Stage Structure