Design of experiments and the virtual PCR simulator: An online game for pharmaceutical scientists and biotechnologists

Design of experiments (DOE) enables scientists to explore complex

rapid iteration between data generation and analysis. This enables the student to perform complex iterative experimental design strategies within a short teaching session. Here, we provide a brief overview of the user interface and describe our experience using this tool in a teaching environment.  7 Students may select settings for 12 experimental factors-see Table 1. DNA amplification through PCR involves cycling through three key steps-denaturation, annealing, and extension. The temperature and duration of each these three steps are controlled during each cycle. In addition, there are four reagent volumes and one categorical choice of polymerase. The objective is to maximize yield and purity of the amplified DNA while minimizing the total cycle time. Users perform experiments. The results are then used to create a leader board where users are ranked over a gliding window of 1 hour. Gaming introduces a competitive element and is an effective motivational tool. 4,6 PCR experiments are simulated by numerically integrating a differential equation model of the reaction, subject to the selected temperature cycle and initial reagent concentrations. The concentrations of the latter are subjected to pipetting "noise" using random values within acceptable operating boundaries of commonly used pipettes. The model captures melting and annealing of all DNA species, binding and unbinding of the polymerase, primer extension, and polymerase degradation. Rates of all reactions are dependent on the temperature. 7 The model qualitatively captures parameter responses of PCR while producing numerical results within seconds.
The user interface is a single web page, or "laboratory notebook," combining an input form for experimental settings, data from all experiments with simulated electrophoresis gel images, and a parameter table from which the user can return to any previously used settings (see Figure 1). The "lab book" also displays the rank of the user's performance within their cohort. Experiments (factors and responses) can be downloaded in CSV format for offline analysis and experimental planning. When the simulator is in qPCR mode, the user may toggle between gel images and DNA concentration time series.

FIGURE 2
Real-time data can be extracted from the PCR simulator allowing the class to monitor their progress. In panel A, we have data for an individual scientist performing experiments, one at a time over a 40-minute practical. The yield of their process is plotted together with their choices of settings for four of the parameters-denaturing time, denaturing temperature, annealing time, and annealing temperature (panel B). Note that denaturing and annealing temperatures are not tested at all, and much of the design space remains unexplored. This is not an isolated example. Even large groups of highly experienced scientists, free to explore the design space however they want, leave much of the design space completely unexplored. In panel C, we have plotted three of the variables-the number of cycles, together with the denaturing temperature and denaturing time, for Phusion and Taq polymerases-for an entire class. Most experiments are performed in narrow regions of the design space, some variables are not investigated at all, and their impact upon robustness is completely untested. See text for details For the last 3 years, we have employed the simulator to teach DoE to small cohorts of around 20 scientists drawn from academic and industrial backgrounds with a range of laboratory and statistical experience. In each class, the simulator sessions are interspersed with lectures, seminars, and workshop activities and last 15 to 45 minutes. Participants apply learning from various sessions in their interactions with the simulator, starting with free exploration of parameter space moving towards screening and optimization experiments. The competitive leader-board component enhances engagement. Data can be exported and interrogated providing information to the instructors and feedback to the scientist.
The PCR simulator provides a virtual experience of a complex laboratory protocol with the simulated reactions qualitatively matching real experiments. We have used it to assess user engagement with the task and to demonstrate how traditional approaches to experimentation often result in suboptimal results, narrow search strategies, and a poor understanding of variability and reproducibility (Figure 2). The teaching interface allows the instructor to draw out personal and group learning. This includes information on the performance of individual users, and how widely or effectively the cohort is exploring potential experimental options.

| CONCLUSION
Our experience has been that the virtual PCR simulator shatters, once and for all, the delusion that the complex spaces we are asked to explore in pharmaceutical R&D and biotechnology are amenable to simplistic OFAT approaches. The problems we work on simply do not lend themselves to such naïve experimentation. Attendees tell us that the simulator demonstrates the real value of a statistical, designed approach. The virtual PCR simulator allows scientists to explore these complex multidimensional spaces in the comparative safety of a simulation environment before trying the tools in the laboratory or the plant.
We invite others to explore the tool at http://virtual-pcr.ico2s.org/.