Practical testing of a C99 compiler using output comparison
Article first published online: 3 APR 2007
Copyright © 2007 John Wiley & Sons, Ltd.
Software: Practice and Experience
Volume 37, Issue 14, pages 1475–1488, 25 November 2007
How to Cite
Sheridan, F. (2007), Practical testing of a C99 compiler using output comparison. Softw: Pract. Exper., 37: 1475–1488. doi: 10.1002/spe.812
- Issue published online: 9 OCT 2007
- Article first published online: 3 APR 2007
- Manuscript Accepted: 23 DEC 2006
- Manuscript Revised: 21 DEC 2006
- Manuscript Received: 21 DEC 2005
- program testing;
A simple technique is presented for testing a C99 compiler, by comparing its output with the output from pre-existing tools. The advantage to this approach is that new test cases can be added in bulk from existing sources, reducing the need for in-depth investigation of correctness issues and for creating new test code by hand. This technique was used in testing the PalmSource Palm OS® Cobalt ARM C/C++ cross-compiler for Palm-Powered® personal digital assistants, primarily for standards compliance and the correct execution of generated code. The technique described here found several hundred bugs, mostly in our in-house code, but also in longstanding high-quality front- and back-end code from Edison Design Group and Apogee Software. It also found 18 bugs in the GNU C compiler, as well as a bug specific to the Apple version of GCC, a bug specific to the Suse version of GCC, and a dozen bugs in versions of GCC for the ARM processor, several of which were critical. Copyright © 2007 John Wiley & Sons, Ltd.