Home

Improving the Quality of Computational Science Software by Using Metamorphic Relations to Test Machine Learning Applications

Xiaoyuan Xie; Joshua Ho; Christian Murphy; Gail E. Kaiser; Baowen Xu; T. Y. Chen

Title:
Improving the Quality of Computational Science Software by Using Metamorphic Relations to Test Machine Learning Applications
Author(s):
Xie, Xiaoyuan
Ho, Joshua
Murphy, Christian
Kaiser, Gail E.
Xu, Baowen
Chen, T. Y.
Date:
Type:
Technical reports
Department:
Computer Science
Permanent URL:
Series:
Columbia University Computer Science Technical Reports
Part Number:
CUCS-004-09
Publisher:
Department of Computer Science, Columbia University
Publisher Location:
New York
Abstract:
Many applications in the field of scientific computing - such as computational biology, computational linguistics, and others - depend on Machine Learning algorithms to provide important core functionality to support solutions in the particular problem domains. However, it is difficult to test such applications because often there is no 'test oracle' to indicate what the correct output should be for arbitrary input. To help address the quality of scientific computing software, in this paper we present a technique for testing the implementations of machine learning classification algorithms on which such scientific computing software depends. Our technique is based on an approach called 'metamorphic testing', which has been shown to be effective in such cases. In addition to presenting our technique, we describe a case study we performed on a real-world machine learning application framework, and discuss how programmers implementing machine learning algorithms can avoid the common pitfalls discovered in our study. We also discuss how our findings can be of use to other areas of computational science and engineering.
Subject(s):
Computer science
Item views:
180
Metadata:
text | xml

In Partnership with the Center for Digital Research and Scholarship at Columbia University Libraries/Information Services | Terms of Use