This paper mechanises conformance verification in the setting of the CSP process algebra. The verification strategy is captured by a theorem stated as a process refinement expression, which can be verified by a model checker such as FDR. The conformance relation, cspio, distinguishes input and output events. The process algebraic framework of CSP is used to address compositional conformance verification by establishing compositionality properties for cspio with respect to the CSP operators. Although cspio has been defined in the standard CSP traces model, one can address quiescence situations using a special output event, in which case it is formally established that cspio is equivalent to Tretmans ioco. All the results have been mechanically proved using the CSP-Prover. The proposed testing theory has been adopted in an industrial context involving collaboration with Motorola, on testing mobile applications. Several examples and a case study are presented to illustrate the overall approach. Copyright © 2013 John Wiley & Sons, Ltd.

This paper mechanises conformance verification in the setting of the CSP process algebra, using the FDR model checker, by establishing compositionality properties for the conformance relation (cspio) with respect to the CSP operators. We explore under which conditions cspio is equivalent to Tretmans ioco. All the results have been mechanically proved using the CSP-Prover. The proposed testing theory has been adopted in an industrial context involving collaboration with Motorola, on testing mobile applications.

A known problem of traditional coverage metrics is that they do not assess oracle quality—that is, whether the computation result is actually checked against expectations. In this paper, we introduce the concept of checked coverage—the dynamic slice of covered statements that actually influence an oracle. Our experiments on seven open-source projects show that checked coverage is a sure indicator for oracle quality and even more sensitive than mutation testing. Copyright © 2013 John Wiley & Sons, Ltd.

A known problem of traditional coverage metrics is that they do not assess oracle quality—that is, whether the computation result is actually checked against expectations. In this paper, we introduce the concept of checked coverage, the dynamic slice of covered statements that actually influence an oracle. Our experiments on seven open-source projects show that checked coverage is a sure indicator for oracle quality and even more sensitive than mutation testing. Copyright ©2013 John Wiley & Sons, Ltd.

As web applications become more complex and are used more pervasively, testing demands are increasing without corresponding automated support. One promising approach to automatic test generation is statistical model-based testing, where logged user behaviour is used to build a usage-based model of web application navigation, from which abstract test cases are generated. Executable test cases are then created by adding parameter values to the abstract test cases. Several researchers have proposed variations of this approach; however, no one has empirically examined the tradeoffs and implications of the different ways to represent user behaviour in a navigation model and the characteristics of the test cases automatically generated from different models. This paper reports on our exploratory study of automatically generated abstract test cases and the underlying usage-based navigation models constructed from over 19,000 user sessions across five publicly deployed web applications. Our results suggest how web testers can easily configure statistical model-based automatic test case generators for web applications toward generating tests closely related to user behaviour or toward new navigations without using large additional test resources. Copyright © 2013 John Wiley & Sons, Ltd.

Statistical model-based testing is an automated approach to generating test cases for web applications, where logged user behaviour is used to build a usage-based model of web application navigation. This paper reports on our exploratory study of the tradeoffs and implications of the ways to represent user behaviour in a navigation model and the characteristics of the test cases automatically generated from various models. Our results suggest how testers can easily configure statistical model-based automatic test case generators for web applications.

The length of test cases is a little investigated topic in search-based test generation for object-oriented software, where test cases are sequences of method calls. Although intuitively longer tests can achieve higher overall code coverage, there is always the threat of bloat – a complex phenomenon in evolutionary computation, where the length abnormally grows over time. In this paper, we show that bloat indeed also occurs in the context of test generation for object-oriented software. We present different techniques to overcome the problem of length bloat, and evaluate all possible combinations of these techniques using different starting lengths for the search. Experiments on a set of difficult search targets, selected from several open source and industrial projects, show that controlling bloat with the appropriate techniques can significantly improve the search performance. Copyright © 2013 John Wiley & Sons, Ltd.

Although intuitively longer tests can achieve higher code coverage, in search-based testing, there is always the threat of bloat, that is, abnormal growth of test length. We present techniques to overcome the problem of length bloat and evaluate all possible combinations of these techniques using different starting lengths for the search. Experiments on a set of difficult search targets, selected from several open source and industrial projects, show that controlling bloat with the appropriate techniques can significantly improve the search performance.

This paper addresses integration testing of data-centric and event-based dynamic service compositions. The compositions under test define abstract services that are replaced by concrete candidate services at runtime. Testing all possible instantiations of a composition leads to combinatorial explosion and is often infeasible. We consider data dependencies between services as potential points of failure and introduce the k-node data flow test coverage metric, which helps to significantly reduce the number of test combinations. We formulate a combinatorial optimization problem for generating minimal sets of test cases. On the basis of this formalization, we present a mapping to the model of FoCuS, a coverage analysis tool. FoCuS efficiently computes near-optimal solutions, which are used to automatically generate test instances. The proposed approach is applicable to various composition paradigms. We illustrate the end-to-end practicability based on an integrated scenario, which uses two diverse composition techniques: on the one hand, the Web Services Business Process Execution Language and on the other hand, WS-Aggregation, a platform for event-based service composition.Copyright © 2013 John Wiley & Sons, Ltd.

This paper addresses the integration testing of data-centric and event-based dynamic service compositions, in which abstract services are replaced by concrete candidate services at runtime. We consider data dependencies between services as potential points of failure and introduce the k-node data flow test coverage metric, which helps to significantly reduce the number of generated test combinations. We illustrate the end-to-end practicability based on an integrated scenario, which uses two diverse service composition techniques (Web Services Business Process Execution Language and WS-Aggregation).

From experience in component-based software engineering, it is known that the integration of high-quality components may not yield high-quality software systems. It is difficult to evaluate all possible interactions between the components in the system to uncover inter-component misfunctions. The problem is even harder when the components are used without source code, specifications or formal models. Such components are called black boxes in literature. This paper presents an iterative approach of combining model learning and testing techniques for the formal analysis of a system of black-box components. In the approach, individual components in the system are learned as finite state machines that (partially) model the behavioural structure of the components. The learned models are then used to derive tests for refining the partial models and/or finding integration faults in the system. The approach has been applied on case studies that have produced encouraging results. Copyright © 2013 John Wiley & Sons, Ltd.

An iterative approach is presented that combines techniques from grammatical inference and model-based testing domains to fulfil two main purposes: (i) reverse engineering of black-box components by inferring (partial) models and (ii) validating integrated systems of such components for compositional problems and generic errors by using their inferred models. The paper investigates how the ‘oracle’ assumption in the classic learning theory can be avoided practically and furthermore how existing model-based approaches can be exploited for testing components whose models are unavailable.

Model-based test generation techniques based on random input generation and guided simulation do not satisfy the demands of high test coverage and completeness guarantees as required by safety-critical applications. Recently, test generation techniques based on model checking have been reported to bridge this gap. To evaluate the effectiveness of these techniques, an in-house tool suite, AutoMOTGen, has been developed for Simulink/Stateflow and applied on real-life case studies at General Motors. This paper outlines the test generation methodology of AutoMOTGen and gives a comparative study with a commercial, primarily random input-based, test generation tool on the same set of examples. The results indicate that in terms of coverage, model checking-based techniques complement the random input-based techniques. In addition, they provide proofs for unreachability that can aid in debugging the models. Therefore, it is recommended that model checking-based tools be utilized to complement and enhance the effectiveness of model-based testing methods in safety-critical systems engineering.Copyright © 2013 John Wiley & Sons, Ltd.

In this paper, model checking-based techniques are shown to complement the existing random inputbased and simulation-based methods in model-based testing of safety-critical embedded systems. They are also shown to enhance the effectiveness of test coverage by providing proofs of unreachability that also aids in debugging. Copyright © 2013 John Wiley & Sons, Ltd.

Software quality is very important in today's competitive business environment. It is a critical constraint on software projects. Software organizations’ major objectives are delivering products on time and achieving quality goals. Quality is directly dependent on software processes, which are inherently variable and uncertain, involving substantial risk. Managing quality risk is an important challenge. The conventional approach to quality risk management for ongoing software processes has two major deficiencies: static analytic models are used, and structured methodologies to enhance processes and improve quality are not systematically applied. This new practical method uses Six Sigma and Monte Carlo Simulation for ongoing quality risk management. DMAIC (Define, Measure, Analyse, Improve, Control) is systematically applied as a tactical framework to enhance the process and improve quality. Simulation predicts quality (reliability) at the expected process end and identifies and quantifies risk. DMAIC is a verified structured methodology for systematic process and quality improvements. Monte Carlo Simulation is superior to conventional risk models. These synergetic enhancements eliminate observed deficiencies. The method has been successfully proven and applied practically on real in-house projects. Substantial savings, quality and customer satisfaction have been achieved. An application on an internal project and obtained results are presented. The method is simplistically elaborated on a published third-party project answering key research questions from practical perspectives. This CMMI® compliant method offers important benefits including savings, quality and customer satisfaction. Copyright © 2013 John Wiley & Sons, Ltd.

Software quality is very important in today's competitive business environment. Managing software quality risk is an important challenge. The conventional approach to quality risk management for ongoing software processes uses analytic models and structured methodologies to enhance processes and improve quality are not systematically applied, which are major deficiencies. This new method uses Monte Carlo Simulation and Six Sigma for ongoing quality risk management. Simulation predicts quality (reliability) and identifies and quantifies the risk. DMAIC (Define, Measure, Analyze, Improve, Control), tactically applied, enhances the process and improves quality. Simulation is superior to conventional risk models. DMAIC is a verified structured methodology for systematic process and quality improvements. These synergetic and significant enhancements eliminate observed deficiencies. The method has been successfully proven and applied practically on real in-house projects. Such an application and obtained results are presented. Substantial savings, quality improvements and customer satisfaction were achieved. This CMMI® compliant method offers very important gains. Copyright © 2013 John Wiley & Sons, Ltd.

Randomized algorithms are widely used to address many types of software engineering problems, especially in the area of software verification and validation with a strong emphasis on test automation. However, randomized algorithms are affected by chance and so require the use of appropriate statistical tests to be properly analysed in a sound manner. This paper features a systematic review regarding recent publications in 2009 and 2010 showing that, overall, empirical analyses involving randomized algorithms in software engineering tend to not properly account for the random nature of these algorithms. Many of the novel techniques presented clearly appear promising, but the lack of soundness in their empirical evaluations casts unfortunate doubts on their actual usefulness. In software engineering, although there are guidelines on how to carry out empirical analyses involving human subjects, those guidelines are not directly and fully applicable to randomized algorithms. Furthermore, many of the textbooks on statistical analysis are written from the viewpoints of social and natural sciences, which present different challenges from randomized algorithms. To address the questionable overall quality of the empirical analyses reported in the systematic review, this paper provides guidelines on how to carry out and properly analyse randomized algorithms applied to solve software engineering tasks, with a particular focus on software testing, which is by far the most frequent application area of randomized algorithms within software engineering. Copyright © 2012 John Wiley & Sons, Ltd.

Randomized algorithms are widely used to address many types of software engineering problems, but they are affected by chance and so require the use of appropriate statistical tests to be properly analysed. To address this issue, this paper provides guidelines on how to carry out and properly analyse randomized algorithms applied to solve software engineering tasks, with a particular focus on software testing.

Code-coverage-based testing is a widely-used testing strategy with the aim of providing a meaningful decision criterion for the adequacy of a test suite. Code-coverage-based testing is also mandated for the development of safety-critical applications; for example, the DO178b document requires the application of the modified condition/decision coverage. One critical issue of code-coverage testing is that structural code coverage criteria are typically applied to source code whereas the generated machine code may result in a different code structure because of code optimizations performed by a compiler. In this work, we present the automatic calculation of coverage profiles describing which structural code-coverage criteria are preserved by which code optimization, independently of the concrete test suite. These coverage profiles allow to easily extend compilers with the feature of preserving any given code-coverage criteria by enabling only those code optimizations that preserve it. Furthermore, we describe the integration of these coverage profile into the compiler GCC. With these coverage profiles, we answer the question of how much code optimization is possible without compromising the error-detection likelihood of a given test suite. Experimental results conclude that the performance cost to achieve preservation of structural code coverage in GCC is rather low. Copyright © 2012 John Wiley & Sons, Ltd.

The SECCO approach ensures that test coverage achieved at source code is preserved during optimizing compilation. A typical use case of coverage preservation is source-based test-data generation. Experiments show that the guaranteed coverage preservation has a negligible impact on the resulting code performance.

Software testing is the most time-intensive and resource-intensive aspect of software development. Can support for testing be improved? This case study describes the motivations and design decisions behind the development of the testing tool, CoTester and its deployment to multiple development teams. CoTester outperforms available testing tools by representing tests using an easy-to-understand scripting language and thus making the tests easily editable. The design decisions of the testing tool were derived after conducting a series of interviews with testers and collecting their experiences with manual as well as automated testing. CoTester was developed to support these users, working in an environment of mixed manual and automatic tests, with a progression from manual to automatic testing when circumstances warrant. A series of deployments to four development teams showed that CoTester worked very well for non-professional testers (i.e. those who do testing only part-time), and it was also found to be useful by some professional testers. Copyright © 2012 John Wiley & Sons, Ltd.

This case study describes the motivations and design decisions behind the development of the testing tool, CoTester, and its deployment to multiple development teams. CoTester outperforms available testing tools by representing tests using an easy-to-understand scripting language, thus making the tests easily editable. A series of deployments to four development teams showed that CoTester worked very well for non-professional testers, and it was also found to be useful by some professional testers.

Model checking is an effective technique used to identify subtle problems in software safety using a comprehensive search algorithm. However, this comprehensiveness requires a large number of resources and is often too expensive to be applied in practice. This work strives to find a practical solution to model-checking automotive operating systems for the purpose of safety analysis, with minimum requirements and a systematic engineering approach for applying the technique in practice. The paper presents methods for converting the Trampoline kernel code into formal models for the model checker SPIN, a series of experiments using an incremental verification approach, and the use of embedded C constructs for performance improvement. The conversion methods include functional modularization and treatment for hardware-dependent code, such as memory access for context switching. The incremental verification approach aims at increasing the level of confidence in the verification even when comprehensiveness cannot be provided because of the limitations of the hardware resource. We also report on potential safety issues found in the Trampoline operating system during the experiments and present experimental evidence of the performance improvement using the embedded C constructs in SPIN. Copyright © 2012 John Wiley & Sons, Ltd.

This paper presents methods for converting the Trampoline kernel code into formal models for the model checker SPIN and a series of experiments using an incremental verification approach. The conversion methods include functional modularization and treatment for hardware-dependent code, such as memory access for context switching. It also reports on potential safety issues found in the Trampoline operating system during the experiments and presents experimental evidence of the performance improvement using the embedded C constructs in SPIN. Copyright © 2012 John Wiley & Sons, Ltd.

With the growing complexity of industrial software applications, industrials are looking for efficient and practical methods to validate the software. This paper develops a model-based statistical testing approach that automatically generates online and offline test cases for embedded software. It discusses an integrated framework that combines solutions for three major software testing research questions: (i) how to select test inputs; (ii) how to predict the expected results of a test; and (iii) when to stop testing software. The automatic selection of test inputs is based on a stochastic test model that accounts for the main particularity of embedded software: time sensitivity. Software test practitioners may design one or more test models when they generate random, user-oriented, or fault-oriented test inputs. A formal framework integrating existing and appropriate specification techniques was developed for the design of automated test oracles (executable software specifications) and the formal measurement of functional coverage. The decision to stop testing software is based on both test coverage objectives and cost constraints. This approach was tested on two representative case studies from the automotive industry. The experiment was performed at unit testing level in a simulated environment on a host personal computer (automatic test execution). The two software functionalities tested had previously been unit tested and validated using the test design approach conventionally used in the industry. Applying the proposed model-based statistical testing approach to these two case studies, we obtained significant improvements in performing functional unit testing in a real and complex industrial context: more bugs were detected earlier and in a shorter time. Copyright © 2012 John Wiley & Sons, Ltd.

This paper presents a model-based statistical testing (MBST) approach for generating functional test cases. The proposed approach consists of eight activities as shown in the figure. These activities answer the three major software testing research questions: (i) how to select test inputs; (ii) how to predict the expected results of a test; and (iii) when to stop testing software. This approach was experimented on two representative case studies from the automotive industry: more bugs were detected earlier and in a shorter time.

Because of the computationally expensive cost of mutation testing, automated system support is indispensable for conducting mutation testing. Mutation systems can be classified into interpretive and noninterpretive, but recent systems are noninterpretive. Weak mutation is a well-known cost reduction method of mutation testing, but it is not directly applicable to noninterpretive mutation systems. To address the problem and take advantage of the efficiency of weak mutation, this paper presents a combined weak and strong mutation for noninterpretive Java mutation systems. The new term ‘serialmutant’ is defined as a specialized program to conduct weak mutation against all mutants in an execution and report only weakly killed mutants as strong mutation candidates. Then strong mutation is conducted only for those reported mutants. The paper also describes an implementation based on a previous Java mutation tool, MuJava. Method-level mutation operators for Java are also redesigned. Experimental results show that the proposed approach efficiently improves the mutation cost in a noninterpretive mutation system. Copyright © 2012 John Wiley & Sons, Ltd.

This paper presents a combined weak and strong mutation for noninterpretive Java mutation system. The new term ‘serialmutant’ is defined as a specialized program to conduct weak mutation against all mutants in an execution and report only weakly killed mutants as strong mutation candidates. Then strong mutation is conducted only for those reported mutants.

This paper describes tool support that has been implemented for the Test Template Framework (TTF). The TTF is a model-based testing (MBT) method that is especially well suited for unit testing from Z specifications. Although the TTF is a sound MBT method and it has been widely referenced since its first publication, attention in recent years has decayed. In fact, some have argued that generating abstract test cases following the TTF is a manual task requiring its users to perform complex predicate manipulations. This paper shows that these observations are dubious by describing Fastest, a tool that implements solutions for all these issues and, according to many experiments, produces abstract test cases for more than 80% of the satisfiable test specifications. Furthermore, it is claimed that Fastest fulfils the needs of the Z user community regarding MBT tools, which is supported with a range of case studies.Copyright © 2012 John Wiley & Sons, Ltd.

Fastest provides tool support for a solid, model-based testing framework for the Z notation known as the Test Template Framework (TTF). The Test Template Framework uses only Z and is aimed mainly to unit testing. The tool automates testing tactic application and semi-automates elimination of unsatisfiable test specifications, generation of abstract test cases and translation of test cases into natural language.

Software change impact analysis (CIA) is a technique for identifying the effects of a change, or estimating what needs to be modified to accomplish a change. Since the 1980s, there have been many investigations on CIA, especially for code-based CIA techniques. However, there have been very few surveys on this topic. This article tries to fill this gap. And 30 papers that provide empirical evaluation on 23 code-based CIA techniques are identified. Then, data was synthesized against four research questions. The study presents a comparative framework including seven properties, which characterize the CIA techniques, and identifies key applications of CIA techniques in software maintenance. In addition, the need for further research is also presented in the following areas: evaluating existing CIA techniques and proposing new CIA techniques under the proposed framework, developing more mature tools to support CIA, comparing current CIA techniques empirically with unified metrics and common benchmarks, and applying the CIA more extensively and effectively in the software maintenance phase. Copyright © 2012 John Wiley & Sons, Ltd.

The study presents a comparative framework, including seven properties, which characterize the change impact analysis (CIA) techniques, identifies key applications of CIA techniques in software maintenance, and discusses the need for further research.

The automatic generation of test suites for systems modelled as finite state machines (FSMs) is an important problem that impacts several critical applications. Known methods that automatically generate tests for FSMs, specially the W-method and some derivations, strongly assume that the number of system states is small. If the overall number of states in the FSM specification is relatively large, such methods become difficult to use. However, often in practice, a system is defined as a combination of several subsystems, with the latter already independently designed, developed and tested. In this paper, we define the concept of combined FSMs and introduce a new method to test modular compositions of FSMs. This method allows for a new incremental testing strategy that turns the testing of new systems into a much more scalable process. As an example, we present an infinite family of naturally occurring FSM models for which our method produces exponentially more compact test suites than the W-method. Copyright © 2012 John Wiley & Sons, Ltd.

This paper proposes a new incremental testing strategy that turns the test of finite state machines (FSMs) a much more scalable process. The C-method has been developed to efficiently test modular compositions of FSMs, thus alleviating the effect of traversal sets on the size of generated test suites for such models. An analysis of an infinite family of FSMs shows that incremental testing can be exponentially more efficient than testing using the traditional W-method.

Mutation testing measures the adequacy of a test suite by seeding artificial defects (mutations) into a program. If a test suite fails to detect a mutation, it may also fail to detect real defects—and hence should be improved. However, there are also mutations that keep the program semantics unchanged and thus cannot be detected by any test suite. Such equivalent mutants must be weeded out manually, which is a tedious task. In this paper, we examine whether changes in coverage can be used to detect non-equivalent mutants: If a mutant changes the coverage of a run, it is more likely to be non-equivalent. In a sample of 140 manually classified mutations of seven Java programs with 5000 to 100 000 lines of code, we found that (i) the problem is serious and widespread—about 45% of all undetected mutants turned out to be equivalent; (ii) manual classification takes time—about 15 min per mutation; (iii) coverage is a simple, efficient and effective means to identify equivalent mutants—with a classification precision of 75% and a recall of 56%; and (iv) coverage as an equivalence detector is superior to the state of the art, in particular violations of dynamic invariants. Our detectors have been released as part of the open-source JAVALANCHE framework; the data set is publicly available for replication and extension of experiments. Copyright © 2012 John Wiley & Sons, Ltd.

In this paper, we examine whether changes in coverage can be used to detect non-equivalent mutants. In a sample of 140 manually classified mutations of seven Java programs with 5000 to 100 000 lines of code, we found that: the problem is serious and widespread (45% of all undetected mutants are equivalent); manual classification takes time (15 min per mutation); coverage is a simple and effective means to identify equivalent mutants (with a classification precision of 75% and 56% recall).

Mutation testing is a well-established method for measuring and improving the quality of test suites. A major cost of mutation testing is the time required to execute the test suite on all the mutants. This cost is even greater when the system under test is multithreaded: not only are test cases from the test suite executed on many mutants but also each test case is executed—or more precisely, explored—for multiple possible thread schedules. This paper introduces a general framework for efficient exploration that can reduce the time for mutation testing of multithreaded code. The paper presents five techniques (four optimizations and one heuristic) that are implemented in a tool called MuTMuT within the general framework. Evaluation of MuTMuT on mutation testing of 12 multithreaded programs shows that it can substantially reduce the time required for mutation testing of multithreaded code.Copyright © 2012 John Wiley & Sons, Ltd.

Mutation testing of multithreaded code is highly time intensive: for each mutant, every test case may be explored for multiple possible thread schedules. This paper introduces a general framework for efficient exploration that can reduce the time for mutation testing of multithreaded code. The paper presents five techniques within the framework that are implemented in a tool calledMuTMuT. Evaluation of MuTMuT on mutation testing of 12 multithreaded programs shows that it indeed substantially reduces the time required.

Scenario-based specifications (SBSs), such as UML interaction models, offer an intuitive and visual way of describing design requirements, and are playing an increasingly important role in the design of software systems. This paper presents an approach to timing analysis of SBSs expressed by UML interaction models. The approach considers more general and expressive timing constraints in UML sequence diagrams (SDs), and gives a solution to the reachability analysis, constraint conformance analysis and bounded delay analysis problems, which reduces these problems into linear programs. With the synchronous interpretation of the SD compositions, the timing analysis algorithms in the approach form a decision procedure for a class of SBSs where any loop in any path is time-independent of the other parts in the path. These algorithms are also a semi-decision procedure for general SBSs with both the synchronous and asynchronous composition semantics. The approach also supports bounded timing analysis of SBSs, which investigates all the paths in the bound limit one by one, and performs the timing analysis for each finite path by linear programming. A tool prototype has been developed to support this approach. Copyright © 2010 John Wiley & Sons, Ltd.

This paper presents a linear programming-based approach to timing analysis of scenario-based specifications (SBSs) expressed by UML interaction models. With more general and expressive timing constraints in UML sequence diagrams, the algorithms in the approach solve the problems of the reachability, constraint conformance and bounded delay analysis of SBSs. These algorithms form a decision procedure for the loop-unlimited SBSs where any loop in any path is time-independent of the other parts in the path, and a semi-decision procedure for general SBSs. Copyright © 2010 John Wiley & Sons, Ltd.

Regression testing is a testing activity that is performed to provide confidence that changes do not harm the existing behaviour of the software. Test suites tend to grow in size as software evolves, often making it too costly to execute entire test suites. A number of different approaches have been studied to maximize the value of the accrued test suite: minimization, selection and prioritization. Test suite minimization seeks to eliminate redundant test cases in order to reduce the number of tests to run. Test case selection seeks to identify the test cases that are relevant to some set of recent changes. Test case prioritization seeks to order test cases in such a way that early fault detection is maximized. This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research. Copyright © 2010 John Wiley & Sons, Ltd.

Test suite minimization, Regression Test Selection (RTS) and test case prioritization are all techniques that aim to reduce the cost of regression testing. This paper presents the first survey of all three of these closely related areas and discusses open problems and the potential directions for future research. Copyright © 2010 John Wiley & Sons, Ltd.

Service-oriented architecture (SOA) is gaining momentum as an emerging distributed system architecture for business-to-business collaborations. This momentum can be observed in both industry and academic research. SOA presents new challenges and opportunities for testing and verification, leading to an upsurge in research. This paper surveys the previous work undertaken on testing and verification of service-centric systems, which in total are 177 papers, showing the strengths and weaknesses of current strategies and testing tools and identifying issues for future work. Copyright © 2012 John Wiley & Sons, Ltd.

Service-oriented architecture (SOA) is gaining momentum as an emerging distributed system architecture for business-to-business collaborations. This momentum can be observed in both industry and academic research. SOA presents new challenges and opportunities for testing and verification, leading to an upsurge in research. This paper surveys the previous work undertaken on testing and verification of service-centric systems, which in total are 177 papers, showing the strengths and weaknesses of current strategies and testing tools and identifying issues for future work.

Despite the existing techniques to reduce the costs of mutation analysis, the computational cost to apply mutation testing with large applications can be very high. One effective technique to improve the efficiency of mutation without losing effectiveness is parallel execution, where mutants and tests are executed in parallel processors, reducing the total time needed to perform mutation analysis. This paper presents a study of this technique adapted to current technologies. Five algorithms to execute mutants in parallel are analysed with three studies that use different network configurations and different number of processors with diverse characteristics. The experiments are performed with Bacterio ^P, a tool that is also presented. Unlike previous studies about parallel mutant execution, which date from the mid-1990s, in the studies in this paper, the communication time in parallel systems no longer acts as a bottleneck. Thus, dynamic strategies, which require more communication, combined with other mutant cost reduction techniques, are the best strategies to run mutants in parallel.Copyright © 2012 John Wiley & Sons, Ltd.

Parallel execution of tests to perform a mutation analysis can improve the efficiency of mutation testing without lose effectiveness. This paper presents a study of different algorithms to parallelize the execution of tests and mutants. Two main findings were found in this study: (i) it is not necessary for a large number of parallel processors to get a big improvement, and (ii) with the current technology, communication time between processors is not a bottleneck.