상세정보
학술대회
Automatic Evaluation of English-to-Korean and Korean-to-English Neural Machine Translation Systems by Linguistic Test Points
Cited 0 time in 
Download 4 time
Share 
Download 4 time
Share
- 발행일
- 201812
- 출처
- Pacific Asia Conference on Language, Information and Computation (PACLIC) 2018, pp.1-8
- 협약과제
- 18HS4200, 지식증강형 실시간 동시통역 원천기술 개발, 김영길
- 초록
- BLEU is the most well-known automaticevaluation technology in assessing theperformance of machine translation systems.However, BLEU does not know which parts ofthe NMT translation results are good or bad.This paper describes the automatic evaluationapproach of NMT systems by linguistic testpoints. This approach allows automaticevaluation of each linguistic test point notshown in BLEU and provides intuitive insightinto the strengths and flaws of NMT systems inhandling various important linguistic test points.The linguistic test points used for automaticevaluation were 58 and consisted of 630sentences. We conducted the evaluation of twobidirectional English/Korean NMT systems.BLEUs of English-to-Korean NMT systemswere 0.0898 and 0.2081 respectively, and theirautomatic evaluations by linguistic test pointswere 58.35% and 77.31%, respectively. BLEUsof Korean-to-English NMT systems were0.3939 and 0.4512 respectively, and theirautomatic evaluations by linguistic test pointswere 33.10% and 40.47%, respectively. Thismeans that the automatic evaluation approachby linguistic test points has similar results asBLEU assessment. According to automaticevaluation by linguistic test points, we knowthat both English-to-Korean NMT systems andKorean-to-English NMT systems havestrengths in polysemy translations, but hasflaws in style translations and translations ofsentences with complex syntactic structures.
- KSP 제안 키워드
- Machine Translation(MT), Neural machine translation, automatic evaluation