BROWSE
Detail
Conference Paper
Selecting Test Inputs for DNNs using Differential Testing with Subspecialized Model Instances
Cited 4 time in 
Share 
Share
- Authors
- Yu-Seung Ma, Shin Yoo, Taeho Kim
- Issue Date
- 2021-08
- Citation
- ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) 2021, pp.1467-1470
- Publisher
- ACM
- Language
- English
- Type
- Conference Paper
- Abstract
- Testing of Deep Learning (DL) models is difficult due to the lack of automated test oracle and the high cost of human labelling. Differential testing has been used as a surrogate oracle, but there is no systematic guide on how to choose the reference model to use for differential testing. We propose a novel differential testing approach based on subspecialized models, i.e., models that are trained on sliced training data only (hence specialized for the slice). A preliminary evaluation of our approach with an CNN-based EMNIST image classifier shows that it can achieve higher error detection rate with selected inputs compared to using more advanced ResNet and LeNet as the reference model for differential testing. Our approach also outperforms N-version testing, i.e., the use of the same DL model architecture trained separately but using the same data.
- KSP Keywords
- Automated Test, DL model, Differential testing, Error Detection Rate, Image classifier, Model architecture, N-version, Preliminary evaluation, Reference model, Test oracle, deep learning(DL)