상세정보
학술지
Constrained optimization for audio-to-visual conversion
- 저자
- 최경호, Jenq-Neng Hwang
- 발행일
- 200406
- 출처
- IEEE Transactions on Signal Processing, v.52 no.6, pp.1783-1790
- ISSN
- 1053-587X
- 출판사
- IEEE
- 협약과제
- 04MD1500, 멀티센서 공간영상정보 통합처리기술 개발, 김경옥
- 초록
- We have developed a new audio-to-visual conversion algorithm that uses a constrained optimization approach to take advantage of dynamics of mouth movements. Based on facial muscle analysis, the dynamics of mouth movements is modeled, and constraints are obtained from it. The obtained constraints are used to estimate visual parameters from speech in a framework of hidden Markov model (HMM)-based visual parameter estimation. To solve the constrained optimization problem, the Lagrangian approach is used to transform the consolidated problem into an unconstrained problem in our implementation. The proposed method is tested on various noisy environments to show its robustness and correctness. Our proposed algorithm is favorably compared with the mixture-based HMM method, which also uses audio-visual HMMs and finds optimal estimates based on a joint audio-visual probability distribution. Our proposed algorithm can estimate optimal visual parameters while satisfying the constraints and avoiding performance degradation in noisy environments.
- KSP 제안 키워드
- Audio-visual, Constrained optimization approach, Lagrangian approach, Parameter estimation, Probability distribution, Unconstrained problem, combinatorial optimization problems(COPs), facial muscle, hidden Markov Model, noisy environments, performance degradation