상세정보
학술지
Deep Learning-Based Optimization of Visual?Auditory Sensory Substitution
- 발행일
- 202302
- 출처
- IEEE Access, v.11, pp.14169-14180
- ISSN
- 2169-3536
- 출판사
- IEEE
- 협약과제
- 23ZS1200, 인간중심의 자율지능시스템 원천기술연구, 최정단
- 초록
- Visual-auditory sensory substitution systems can aid visually impaired people in traveling to various places and recognizing their own environments without help from others. Although several such systems have been developed, they are either not widely used or are limited to laboratory-scale research. Among various factors that hinder the widespread use of these systems, one of the most important issues to consider is the optimization of the algorithms for sensory substitution. This study is the first attempt at exploring the possibility of using deep learning for the objective quantification of sensory substitution. To this end, we used generative adversarial networks to investigate the possibility of optimizing the vOICe algorithm, a representative visual-auditory sensory substitution method, by controlling the parameters of the method for converting an image to sound. Furthermore, we explored the effect of the parameters on the conversion scheme for the vOICe system and performed frequency-range and frequency-mapping-function experiments. The process of sensory substitution in humans was modeled to use generative models to assess the extent of visual perception from the substituted sensory signals. We verified the human-based experimental results against the modeling results. The results suggested that deep learning could be used for evaluating the efficiency of algorithms for visual-auditory sensory substitutions without labor-intensive human behavioral experiments. The introduction of deep learning for optimizing the visual-auditory conversion method is expected to facilitate studies on various aspects of sensory substitution, such as generalization and estimation of algorithm efficiency.
- KSP 제안 키워드
- Behavioral experiments, Conversion method, Laboratory scale, Sensory Substitution, Substitution method, Substitution systems, Visual Perception, Visually Impaired People, algorithm efficiency, deep learning(DL), generative adversarial network
본 저작물은 크리에이티브 커먼즈 저작자 표시 (CC BY) 조건에 따라 이용할 수 있습니다.
