The content of this paper is solely the responsibility of the authors and does not represent the official views or recommendations of the Agency for Healthcare Research and Quality or the Department of Health and Human Services.
While questionnaires have been developed to capture patient-reported outcomes (PROs) in rheumatology practice, these instruments are not widely used. We developed a touchscreen interface designed to provide reliable and efficient data collection. Using the touchscreen to obtain PROs, we compared 2 different workflow models implemented separately in 2 rheumatology clinics.
The Plan-Do-Study-Act methodology was used in 2 cycles of workflow redesign. Cycle 1 relied on off-the-shelf questionnaire builder software, and cycle 2 relied on a custom programmed software solution.
During cycle 1, clinic 1 (private practice model, resource replete, simple flow) demonstrated a high completion rate at the start, averaging between 74% and 92% for the first 12 weeks. Clinic 2 (academic model, resource deficient, complex flow) did not achieve a consistent completion rate above 60%. The revised cycle 2 implementation protocol incorporated a 15-minute “nurse visit,” an instant messaging system, and a streamlined authentication process, all of which contributed to substantial improvement in touchscreen questionnaire completion rates of ∼80% that were sustained without the need for any additional clinic staff support.
Process redesign techniques and touchscreen technology were used to develop a highly successful, efficient, and effective process for the routine collection of PROs in a busy, complex, and resource-depleted academic practice and in typical private practice. The successful implementation required both a touchscreen questionnaire, human behavioral redesign, and other technical solutions.
Patient-reported outcomes (PROs) questionnaires have been developed for a diversity of conditions and disease states (1), including assessments of health status, quality of life, symptoms, satisfaction with care or treatments, and economic impact (2, 3). While often used for research, these tools are not widely used in clinical practice (4). In rheumatology, the Multidimensional Health Assessment Questionnaire (MDHAQ) was designed and validated to quantitatively assess a patient's physical function, pain, and global health status (5). The MDHAQ offers outcomes measures for patients with rheumatoid arthritis (6, 7), which can serve to guide rheumatologic treatment decisions (8–11). Moreover, the prognostic value of these PRO data has been compared to the importance of systolic or diastolic blood pressure, cholesterol, and smoking as risk factors for premature cardiovascular death (12).
Pincus et al (13) and Harrington (14) have demonstrated the importance of collecting a standardized data set and routinely assessing disease activity. While the feasibility and value of the MDHAQ in clinical practice is known and can be easily scored (13), the paper-based version of the MDHAQ is only used by ∼20% of rheumatologists. This lack of adoption is explained, in part, by motivation. Highly motivated physicians have successfully accomplished a paper-based PRO capture process in their respective practices (13, 14). The other reason for low adoption may have to do with workflow challenges that become increasingly complex in multiprovider practices. A paper-based process requires presentation of the form to the patient, completion of the form, completion assistance if needed, collection of the form, scoring the form in a timely manner and making it available to the provider, storing the results for future reference, and if at all possible, being able to make comparison with previous values.
Alternatively, computer-based utilities (e.g., office-based touchscreen computers, telephone-based interactive voice response systems, handheld computers, and mobile phones) are emerging to address these workflow challenges in obtaining, aggregating, calculating, and displaying data in real time, as well as minimizing response errors (1, 15). This format for the capture of PROs has been well received by patients (10, 16–20).
Given the prior work in the use of touchscreen questionnaires (10, 16–20), we explored how a computerized version of the MDHAQ could be adopted for routine use in rheumatology practice to facilitate patient-centered care. One limitation to prior studies of touchscreen questionnaires is that the focus has been on implementing a research protocol rather than a protocol for routine practice. Herein, we describe the process models, the clinic-based process redesign, and the performance of 2 pragmatic workflows: one intended for typical community practices and the other intended for large academic-based practices.
Significance & Innovations
Process redesign techniques and touchscreen technology were used to implement a successful care process for the routine collection of patient-reported outcomes in a clinical environment.
Adapting to change, teaming, ownership, and setting expectations were key concepts necessary to the successful integration of a new process into an existing system of care.
MATERIALS AND METHODS
The touchscreen questionnaire was implemented as a quality improvement clinical redesign in 2 clinics within the Geisinger Clinic Department of Rheumatology. Throughout the implementation process, the rheumatology department monitored the percentage of patients who were completing the questionnaire to adjust and improve the workflow. We utilized this de-identified statistic to compare 2 cycles of the touchscreen questionnaire implementation; as such, the work detailed herein was determined to be nonhuman subjects research by the Geisinger Health System (GHS) Institutional Review Board.
We used the MDHAQ developed by Pincus et al (7) that included the 10-question MDHAQ physical function scale; visual scales of pain, fatigue, stiffness, and global assessment; fall assessment; a full review of systems; patient-reported tender joint counts; demographic data (such as marital status and work status); and 10 events of interest that may have occurred since the last encounter (such as fracture, hospitalization, infection, medication side effect). The touchscreen questionnaire included a total of ∼110 items, although certain items (such as a review of systems) were grouped to allow a single answer if all items were negative.
The initial touchscreen questionnaire was built using Prezzatech Checkbox, a web-based survey building tool. The web questionnaire was formatted for use on a touchscreen computer monitor. Questions were grouped on a single page to avoid difficulties with scrolling. The revised, custom-coded touchscreen questionnaire was enhanced to include larger fonts and radio buttons (to allow easy item selection for patients with arthritic fingers) to facilitate viewing and to simplify use (Figure 1). In addition, built-in error checking gave patients the opportunity to go back to a section that was not accurately completed.
Patients accessed the touchscreen questionnaire following completion of an onscreen authentication process (i.e., validating a patient's identity), after which the questionnaire was automatically scored and results were stored in a database. In conjunction with our health system's Information Security Office, we determined that a patient's last name, date of birth, and medical record number were sufficient to authenticate a patient's identity and ensure that patient-reported data were appropriately linked to the patient's electronic health record (EHR).
Study setting and workflow.
The GHS is a large integrated delivery system serving the population in Central and Northeastern Pennsylvania. GHS includes the Geisinger Clinic, a network of 37 community-based ambulatory clinic sites staffed by employed physicians. All Geisinger clinics have used the EpicCare (Epic Systems) EHR since 2001. Two rheumatology clinic sites with distinct workflows were chosen for study redesign and implementation. Rheumatology clinic 1, typical of a private group practice, consisted of 2 full-time clinical rheumatologists who saw return patients every 15 to 20 minutes. The workflow was examination-room centric because the physician to nurse ratio was 1:1 and each physician had 2 examination rooms available for patient encounters (Table 1). Patients checked in at the front desk and were then placed in the examination room for the nurse to take vital signs and for the completion of their touchscreen questionnaire. Rheumatology clinic 2 was more typical of an academic rheumatology department. Daily duties included patient care, teaching residents and fellows, and engaging in research. Department members consisted of staff rheumatologists, fellows, and an advanced-practice rheumatology nurse specialist. The physician to nurse ratio in this clinic was 5:1, and each physician had 1 examination room for patient encounters. Given the constraints of nursing staff and available examination rooms, the workflow involved moving patients from the front desk for check-in to the waiting area for touchscreen questionnaire completion, then to the nurse check-in station for vital signs, and then finally to the examination room for the physician encounter. Workstations were installed in the waiting area for clinic 2 and consisted of a 17-inch touchscreen monitor mounted directly to a small computer unit and desks constructed to allow for a patient-friendly and private workspace. One of the workstations included a slide-out shelf for the touchscreen to accommodate patients in wheelchairs. Each touchscreen workstation cost $1,700. Four workstations were sufficient to provide an available touchscreen questionnaire station at all times for 10 providers. The waiting room space needed to accommodate the 4 workstations was 7 feet by 16 feet.
Table 1. Rheumatology site comparison
Staff, residents, fellows, nurse specialists
Front desk to examination room
Front desk to waiting area to nurse check-in to examination room
Resource replete, simple flow
Resource deficient, complex flow
Informational sheets and a clinic-based poster informed patients of the touchscreen questionnaire process and provided instruction. In advance of implementation, the front desk staff developed patient information sheets and standardized responses to patient queries. This involvement created a sense of ownership among staff and ensured that a consistent message was delivered to patients. The touchscreen questionnaire was offered for return visits to all rheumatology patients at clinics 1 and 2. Patients chose whether or not to complete the touchscreen questionnaire.
Process redesign methodology.
A Plan-Do-Study-Act (PDSA) methodology was used to redesign the workflow and implement the touchscreen questionnaire (21). PDSA involves small-scale rapid-cycle tests of change as a form of “learning in action.” The PDSA cycle methodology involves the following 4 steps: 1) plan (state objective, predict what will happen, develop a plan), 2) do (do it, record problems and observations, begin data analysis), 3) study (complete data analysis, compare to predictions, summarize), and 4) act (what are the modifications, what happens next cycle) (22, 23). Two cycles, described in the Results, were tested. The second cycle was designed and implemented only after the results of the first cycle were recorded and interpreted.
While specific members of the rheumatology nursing, front desk, and provider teams were involved in problem solving, all team members were engaged in each step of the redesign plan to develop a sense of ownership. Weekly reports of questionnaire completion rates were provided to the front desk and nursing staff. Problems encountered and potential solutions were discussed at that time. Additionally, the front desk staff and nursing staff were incentivized by including the questionnaire completion rate as one of the key evaluation measures of their job performance.
Cycle 1 of the touchscreen questionnaire was implemented in both clinics and relied on a keyboard for patients to enter the required authentication data (last name, date of birth, medical record number). At clinic 1 (resource replete), a clinic assistant worked with the nurse to help the patient. At clinic 2 (resource limited and more complex flow), a full-time clinic assistant guided the patients through the authentication process, assisted them with any touchscreen questionnaire–related questions, and notified the nurses that the patient's questionnaire had been completed.
For the first 2 weeks, only a subset of patients was instructed to complete the touchscreen questionnaire. This allowed the clinic staff the time needed to adjust to the new workflow. The new workflow was then fully implemented. During the subsequent 12 weeks, staff members in both clinics were provided with weekly debriefings and feedback regarding the touchscreen questionnaire completion rate. The overall average time required of patients completing the touchscreen questionnaire for the first time was 9.5 minutes during the first 12 weeks (daily range 7.0–11.4 minutes). The weekly completion rate in clinic 1 (resource replete, simple flow) was between 74% and 92% for the first 12 weeks (Table 2), whereas the rate in clinic 2 was below 60%.
Table 2. Cycle 1 touchscreen questionnaire completion rate by clinic site
Resource replete, simple flow, %
Resource deficient, complex flow, %
Several problems were identified at the end of cycle 1. First, there was not enough time built into the clinic schedule to complete all tasks prior to the physician encounter. Second, timely rooming (i.e., the patient was ready to be escorted by the nurse to the examination room) for the physician encounter was a priority relative to completion of the questionnaire. Physicians would interrupt the patient while they were completing the questionnaire and escort them to the examination room. Third, completion of the questionnaire was highly dependent on the presence of a clinic assistant who, as noted, helped with authentication and also notified nurses that the patient was done and available for rooming. These problems were magnified in clinic 2 due to resource constraints and the need for the patient to move from the waiting area after completing the questionnaire.
Cycle 1 findings resulted in several important changes. First, a specific 15-minute “nurse visit” at the beginning of the encounter was implemented for routine return visits. The patient's appointment reminder would therefore reflect, for example, a nurse visit at 8:45 AM and a physician visit at 9:00 AM. Second, a more efficient authentication process was implemented using ColdFusion (Adobe) and deployed in a web-based format. This new authentication process eliminated the need for patient input using a physical keyboard. Instead, a virtual touchscreen keyboard was used to complete the entire questionnaire. The touchscreen keyboard was designed to be more user friendly (i.e., large letters and numbers, in alphabetical/ordinal order) so that even patients not familiar with a typical keyboard could easily find appropriate numbers and letters. The most challenging problem was eliminating the need for a full-time assistant to perform the “help” function (i.e., alerting the clinic staff that the patient was in need of help in completing the questionnaire). Two redesign elements were required.
First, the nursing team agreed to perform the “help” function. Because nurses are mobile and can perform the same clinic tasks, they were best suited to provide help in the waiting area on an ad hoc basis. Second, a system was designed to monitor the status of every patient who began a touchscreen questionnaire. Dual monitors were placed in strategic locations where the nurses usually congregated; one screen was used for the EHR, and the second screen displayed the status of patients who were completing the questionnaire and relied on an instant messenger application. When the patient began the touchscreen questionnaire, a bar would appear on the notification monitor indicating the workstation, the patient's medical record number (with ability to view the patient name by hovering over this number), the status, the start time, the patient's physician, and the physician appointment time (Figure 2). When a patient began working on the questionnaire, the status would read “in progress” and the bar would be yellow. If the patient pressed the help button, the bar turned red and the status changed to “assistance needed.” The nurse who saw the red signal would clear the message and assist the patient. When the patient completed the touchscreen questionnaire, the bar would turn green and the status would change to “completed.” The nurse would then bring the patient to the vitals area to complete the next task. The nursing team agreed that whoever saw the message first was the one who owned the task of assisting the patient.
This new process resulted in an immediate increase in the completion rates to ∼80% (Figure 3), which was sustained for the subsequent 20 weeks. No issues were noted by nurses given the additional responsibility to their daily routine. Cycle 2 continues to run in each clinic; more than 15,000 touchscreen questionnaires have been completed. No additional support staff has been hired to run this process, and routine use of the touchscreen questionnaire has been fully integrated into daily clinical practice.
In describing significant gaps in US health care, the 2001 Institute of Medicine (IOM) Chasm Report focused on means for improving delivery so that it is timely, effective, efficient, patient centric, equitable, and safe (24). We recommend that a cornerstone for fulfilling the vision of the IOM is collecting and using PROs in a manner that saves time, allows for a greater focus on issues important to the patient, and effectively communicates to the provider. A care process that routinely elicits and displays PROs as a standard of care helps to improve efficiency and make patient-centric care possible. Routine PRO collection may also improve the safety and efficacy of health care by providing the clinician with real-time decision-relevant information (e.g., symptom progression, recent events) that is not always systematically collected and available during a typical encounter.
The value of collecting PRO data is difficult to dispute. Over the past 3 decades, questionnaires have been developed and validated for this purpose. Our experience suggests that the primary barrier to adoption of PROs is in devising a workflow that ensures the integrity of scheduling. Pincus et al (7) pioneered the development of PROs for rheumatology and showed that the MDHAQ-related measures predict outcomes and are important to medical decision making. In some sense, the MDHAQ measures are a fifth vital sign (after temperature, pulse, blood pressure, and weight). Additionally, Yazici et al focused on the ease of use of these instruments in a paper-based collection and scoring process (6).
Use of a paper-based questionnaire in clinical practice requires a number of steps, each human dependent. First, a reliable method needs to be used to ensure that the patient is handed the paper form upon arrival. Second, the patient needs to document their identity on the form. Third, the form needs to be completed with minimal questions missed. Fourth, a manual help method needs to be provided. Fifth, the paper form needs to be handed back to a staff person. Sixth, the form needs to be scored in real time. Finally, relevant information must then be made available for review by the physician before the encounter, and current scores need to be interpreted with previous scores. This last task requires that all scores be recorded to a common template to observe trends in relation to treatment and other factors.
This process can indeed work reliably (13, 14), but it is more likely to be problematic in a more complex system of care, i.e., multiple providers, multiple specialties, and shared support-staff resources with competing responsibilities. We have used a paper-based new questionnaire designed with many of the MDHAQ elements for more than 20 years. The value it brings to improving the quality of the first encounter is palpable. However, we have never successfully obtained these data routinely on patients who return for followup visits using a paper-based form, in part due to a complicated clinic workflow and human resources that simply are not available. A computer-based touchscreen questionnaire process offers advantages over a paper-based process in simplifying the collection of PROs, ensuring reliability and validity (e.g., error checking) of the data capture and in simplifying the effective use of the data in real time (Figure 4). Data are not subject to being lost (e.g., misplaced paper questionnaire) or misidentified (e.g., wrong identifiers assigned to completed questionnaire). Error checking and scoring are automatic. Information can be presented in real time electronically with summary scores from previous encounters. The results can be accessed to answer quality or research questions. Therefore, a touchscreen-based system for collection of PROs is an important tool for developing a highly reliable, effective, and efficient process.
However, integrating PROs into daily clinical practice requires redesign, a finding that is reinforced by the IOM conclusion that improvement can only occur by fundamentally redesigning our care delivery system (24). Continuing to practice using the same method will not achieve appreciably different results (25). Yet few rheumatologists have the training and skills in the methodology needed to redesign their local care process. In addition to broadening our personal problem-solving and redesign skills, there needs to be a broader commitment on a national level. We recommend the mandate of the Accreditation Council for Graduate Medical Education to include systems-based learning as part of the 6 core training competencies for residents and fellows (26).
The PDSA process improvement method offers an effective technique for problem solving in the rapid cycle redesign of a clinical practice. PDSA allows for effective problem solving in a continuous fashion. Using this methodology, our results show that the routine use of a touchscreen questionnaire can indeed be tightly integrated with routine clinical care in a busy, complex, and resource-constrained academic rheumatology department. Additionally, we identified 4 key concepts (adapting to change, teaming, ownership, and setting expectations) that act as drivers of either worse or better performance for any clinic wishing to integrate a new process into their existing system of care.
The first concept is adapting to change versus resisting change. Resistance to change seems almost an innate characteristic of medical care, as exemplified by the 17-year average time to adopt new clinical processes that are well established in science (24). Change invariably involves staff members and patients who choose whether to adopt the new process, ignore the new process, or work against the new process. As an example, understanding the physicians' imperative to maintain schedule integrity allowed us to focus on developing a solution (e.g., the instant messaging application) that allowed them to be better supportive of adopting the new touchscreen questionnaire process. Additionally, focusing on patient needs (e.g., blocking specific time for questionnaire completion, developing a confidential touchscreen completion environment, providing a patient-friendly help functionality utilizing the nursing staff) has allowed us to achieve and sustain a >80% touchscreen completion rate. While not quantified, the reasons for noncompletion in the remaining 20% included cognitive impairment, insufficient time, perceived nonvalue, and unfamiliarity with electronic devices. The last 3 items are opportunities for improvement in flow and patient activation. However, given that 16% of the population are nonadopters (27) and that the process of touchscreen questionnaire completion requires 3 separate groups (physicians, nurses, and patients) to complete the process, a >80% sustained completion rate bespeaks a highly effective and reliable process.
The second concept is the importance of teaming. By involving the nurses in the help function, we empowered them to be an even more important member of the health care delivery team. Teaming is highly associated with the third concept, which is ownership. Change can occur by fiat, by buy-in, and through ownership. Change by fiat (“you will change because I tell you to”) and by buy-in (someone else comes in and changes things for you) may lead to temporary improvement, but are not sustainable activities. Change that occurs by ownership, i.e., where the changes in process are led by the very people who are affected, leads to the greatest and most sustainable improvements in care. Teaming and ownership are necessary ingredients for sustained improvement and our results support that contention. Our providers were willing to put teamwork and standardization of clinical processes ahead of individualism and honoring those who resist change. The shared ownership of excellence was fundamental to success.
The fourth key concept is setting expectations. Expectations of participation were set by the project leaders for all staff members (nurses, front desk staff, physicians). These expectations included not just participation, but also regular updates by the leaders to the staff, and regular solicitation of new ideas from the staff (reinforcing the teaming and ownership concepts). Using process redesign techniques and touchscreen questionnaire technology, we were able to develop a highly successful, efficient, and effective process for the routine collection of PROs in a busy, complex, and resource-depleted academic practice as well as in a more typical private practice environment. The next step to meeting the IOM recommendation for care improvement includes the integration of the PROs into a visual display tool to be used by providers at the point of care for real-time medical decision making, patient empowerment, and population-based quality improvement. Therefore, in addition to the touchscreen questionnaire described herein, we have developed a sophisticated, web-based application (Patient Centric Electronic Redesign; PACER) that obtains, aggregates, exchanges, and displays data from the EHR, the patient, the rooming nurse, and the physician. This tool can be accessed seamlessly from within the EHR to provide rheumatologists with a visual dashboard of actionable information needed during patient encounters.
Using both the touchscreen questionnaire and the web-based dashboard, we have developed an efficient and effective process for routinely collecting, scoring, storing, and presenting physicians with PROs at the point of care.
All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be submitted for publication. Dr. Newman had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study conception and design. Newman, Stewart.
Acquisition of data. Newman, Lerch, Jones, Stewart.
Analysis and interpretation of data. Newman, Lerch, Stewart.