Modern software systems ranging from e-commerce websites to telecommunication infrastructures must service millions of users. Many field problems of these systems are due to their inability to scale to field workloads, not due to feature bugs. In addition to conventional functional testing, these systems must be tested with large volumes of concurrent requests (called the load) to ensure the quality of these systems. The theme of this year’s workshop is Large-Scale Testing, which includes all different objectives and strategies of testing large-scale software systems using load. Examples of large-scale testing include live upgrade testing, load testing, high availability testing, operational profile testing, performance testing, reliability testing, stability testing and stress testing.

Large-Scale testing is a difficult task requiring a great understanding of the system under test. Practitioners face many challenges such as tooling (choosing/implementing the testing tools), environments (software, hardware setup) and time (limited time to design, test, and analyze). Yet, little research is done in the software engineering research. Large-scale testing is gaining more importance, as an increasing number of services are being offered in the cloud to millions or even billions of users. This one-day workshop brings together software testing researchers, practitioners and tool developers to discuss the challenges and opportunities of conducting research on large-scale software testing. Our ultimate goal is to grow an active community around this important and practical research topic.

We solicit the following two tracks of papers: Technical track (4 pages) and Industrial track (2 pages). Papers should follow the two-column ACM conference publication format and need to be submitted electronically via EasyChair. Accepted papers will be published in the ICPE 2014 Proceedings. Submitted papers can be research papers, position papers, case studies or experience reports addressing issues including but not limited to the following:
- Efficient and cost-effective test executions;
- Rapid and scalable analysis of the test results;
- Case studies and experience reports on large-scale testing;
- Leveraging cloud computing to conduct large-scale testing;
- Large-scale testing on emerging systems (e.g., adaptive/autonomic systems or cloud services);
- Taxonomies of testing large-scale software systems;
- Large-scale testing in the context of agile software development process.