Metamorphic Runtime Checking of Applications Without Test Oracles

Bell, Jonathan Schaffer; Murphy, Christian; Kaiser, Gail E.

For some applications, it is impossible or impractical to know what the correct output should be for an arbitrary input, making testing difficult. Many machine-­learning applications for “big data”, bioinformatics and cyberphysical systems fall in this scope: they do not have a test oracle. Metamorphic Testing, a simple testing technique that does not require a test oracle, has been shown to be effective for testing such applications. We present Metamorphic Runtime Checking, a novel approach that conducts metamorphic testing of both the entire application and individual functions during a program’s execution. We have applied Metamorphic Runtime Checking to 9 machine-­‐learning applications, finding it to be on average 170% more effective than traditional metamorphic testing at only the full application level.



More About This Work

Academic Units
Computer Science
Department of Computer Science, Columbia University
Columbia University Computer Science Technical Reports, CUCS-024-14
Published Here
January 30, 2015