BROWSE
Detail
Conference Paper
EgoCoord: Self-calibrated Egocentric 3D Body Pose Estimation using Pixel-wise Coordinate Encoding
Cited 0 time in 
Share 
Share
- Authors
- Jong-Bae Lee, Hyoung Lee, Beom-Ryeol Lee, Byung-Gook Lee, Wook-Ho Son
- Issue Date
- 2024-12
- Citation
- Asian Conference on Computer Vision (ACCV) 2024, pp.1-17
- Language
- English
- Type
- Conference Paper
- Abstract
- The primary challenges for egocentric 3D human pose estimation techniques are the perspective and radial distortions introduced by fisheye lenses. Previous methods utilized camera calibration for undistortion or utilized neural networks to regress 3D human poses from distorted 2D poses. In this paper, we propose a novel approach that integrates a pixel-wise coordinate encoding technique for recognizing image distortion and utilizes the Vision Transformer (ViT) to extract distortion and pose tokens from the input image. The extracted tokens are used in a 3D volumetric heatmap-based egocentric pose estimator, which predicts the 3D human pose using pose tokens and performs pose correction using distortion tokens. The approach integrates CoordConv’s positional encoding strategies, neural network-based camera calibration methods, and the volumetric heatmap-based 3D human pose estimation method. We evaluate the proposed model’s performance using our new evaluation dataset and compare it with state-of-the-art models. Additionally, we perform an ablation study to demonstrate the individual effects of each module in the proposed model.
- KSP Keywords
- 3D human pose estimation, Calibration method, Camera Calibration, Encoding Technique, Estimation technique, Fisheye lenses, Image distortion, Network-based, Novel approach, Pose correction, Pose estimator