Towards a framework and a benchmark for testing tools for multi-threaded programs
Article first published online: 1 AUG 2006
Copyright © 2006 John Wiley & Sons, Ltd.
Concurrency and Computation: Practice and Experience
Special Issue: Parallel and Distributed Systems: Testing and Debugging (PADTAD)
Volume 19, Issue 3, pages 267–279, 10 March 2007
How to Cite
Eytani, Y., Havelund, K., Stoller, S. D. and Ur, S. (2007), Towards a framework and a benchmark for testing tools for multi-threaded programs. Concurrency Computat.: Pract. Exper., 19: 267–279. doi: 10.1002/cpe.1068
- Issue published online: 30 JAN 2007
- Article first published online: 1 AUG 2006
- Manuscript Accepted: 4 FEB 2006
- Manuscript Revised: 1 MAR 2005
- Manuscript Received: 1 AUG 2004
Multi-threaded code is becoming very common, both on the server side, and very recently for personal computers as well. Consequently, looking for intermittent bugs is a problem that is receiving more and more attention. As there is no silver bullet, research focuses on a variety of partial solutions. We outline a road map for combining the research within the different disciplines of testing multi-threaded programs and for evaluating the quality of this research. We have three main goals. First, to create a benchmark that can be used to evaluate different solutions. Second, to create a framework with open application programming interfaces that enables the combination of techniques in the multi-threading domain. Third, to create a focus for the research in this area around which a community of people who try to solve similar problems with different techniques can congregate. We have started creating such a benchmark and describe the lessons learned in the process. The framework will enable technology developers, for example, developers of race detection algorithms, to concentrate on their components and use other ready made components (e.g. an instrumentor) to create a testing solution. Copyright © 2006 John Wiley & Sons, Ltd.