상세정보
학술지
Analysis and Validation of Cross-Modal Generative Adversarial Network for Sensory Substitution
- 발행일
- 202106
- 출처
- International Journal of Environmental Research and Public Health, v.18 no.12, pp.1-22
- ISSN
- 1661-7827
- 출판사
- MDPI
- 협약과제
- 21ZS1200, 인간중심의 자율지능시스템 원천기술연구, 최정단
- 초록
- Visual-auditory sensory substitution has demonstrated great potential to help visually impaired and blind groups to recognize objects and to perform basic navigational tasks. However, the high latency between visual information acquisition and auditory transduction may contribute to the lack of the successful adoption of such aid technologies in the blind community; thus far, substitution methods have remained only laboratory-scale research or pilot demonstrations. This high latency for data conversion leads to challenges in perceiving fast-moving objects or rapid environmental changes. To reduce this latency, prior analysis of auditory sensitivity is necessary. However, existing auditory sensitivity analyses are subjective because they were conducted using human behavioral analysis. Therefore, in this study, we propose a cross-modal generative adversarial network-based evaluation method to find an optimal auditory sensitivity to reduce transmission latency in visual-auditory sensory substitution, which is related to the perception of visual information. We further conducted a human-based assessment to evaluate the effectiveness of the proposed model-based analysis in human behavioral experiments. We conducted experiments with three participant groups, including sighted users (SU), congenitally blind (CB) and late-blind (LB) individuals. Experimental results from the proposed model showed that the temporal length of the auditory signal for sensory substitution could be reduced by 50%. This result indicates the possibility of improving the performance of the conventional vOICe method by up to two times. We confirmed that our experimental results are consistent with human assessment through behavioral experiments. Analyzing auditory sensitivity with deep learning models has the potential to improve the efficiency of sensory substitution.
- KSP 제안 키워드
- Auditory signal, Behavioral experiments, Environmental change, Evaluation method, High latency, Laboratory scale, Model-based analysis, Moving Object, Proposed model, Sensitivity to, Sensory Substitution
본 저작물은 크리에이티브 커먼즈 저작자 표시 (CC BY) 조건에 따라 이용할 수 있습니다.
