Murphy Christian author Columbia University. Computer Science Kaiser Gail E. author Columbia University. Computer Science Columbia University. Computer Science originator contributor text Technical reports New York Department of Computer Science, Columbia University 2010 Software testing of applications in fields like scientific computing, simulation, machine learning, etc. is particularly challenging because many applications in these domains have no reliable "test oracle" to indicate whether the program's output is correct when given arbitrary input. A common approach to testing such applications has been to use a "pseudo-oracle", in which multiple independently-developed implementations of an algorithm process an input and the results are compared. Other approaches include the use of program invariants, formal specification languages, trace and log file analysis, and metamorphic testing. In this paper, we present the results of two empirical studies in which we compare the effectiveness of some of these approaches, including metamorphic testing, pseudo-oracles, and runtime assertion checking. We also analyze the results in terms of the software development process, and discuss suggestions for practitioners and researchers who need to test software without a test oracle. Computer science CUCS-039-10 http://hdl.handle.net/10022/AC:P:10525 English NNC NNC 2011-06-09 12:01:18 -0400 2011-08-14 19:25:39 -0400 4447 eng