BROWSE
Detail
Conference Paper
More Insightful Feedback for Tutoring: Enhancing Generation Mechanisms and Automatic Evaluation
Cited 0 time in 
Share 
Share
- Authors
- Wencke Liermann, Jin-Xia Huang, Yohan Lee, Kong Joo Lee
- Issue Date
- 2024-11
- Citation
- Conference on Empirical Methods in Natural Language Processing (EMNLP) 2024, pp.10838-10851
- Language
- English
- Type
- Conference Paper
- Abstract
- Incorrect student answers can become valuable learning opportunities, provided that the student understands where they went wrong and why. To this end, rather than being given the correct answer, students should receive elaborated feedback on how to correct a mistake on their own. Highlighting the complex demands that the generation of such feedback places on a model's input utilization abilities, we propose two extensions to the training pipeline. Firstly, we employ a KL regularization term between a standard and enriched input format to achieve more targeted input representations. Secondly, we add a preference optimization step to encourage student answer-adaptive feedback generation. The effectiveness of those extensions is underlined by a significant increase in model performance of 3.3 METEOR points. We go beyond traditional surface form-based metrics to assess two important dimensions of feedback quality, i.e., faithfulness and informativeness. Hereby, we are the first to propose an automatic metric measuring the degree to which feedback divulges the correct answer, that we call Informativeness Index I2. We verify in how far each metric captures feedback quality.
- KSP Keywords
- Adaptive feedback, Learning opportunities, Model performance, Preference optimization, Regularization term, automatic evaluation, generation mechanism