Measuring Pain as the 5th Vital Sign Does Not Improve Quality of Pain Management
The authors have no conflicts of interest to report.An abstract of this work was presented as a poster at the VA HSR&D National Meeting, 2005 as Mularski RA, White-Chu F, Overbay D, Miller L, Asch SM, Ganzini L. Pain as the 5th vital sign: no detectable impact on pain management in a VHA outpatient clinic. Available at http://www1.va.gov/hsrd/about/national_meeting/2005/display_abstract.cfm?RecordID=196.
Address correspondence and requests for reprints to Dr. Mularski: Veterans Affairs Greater Los Angeles Healthcare System, Department of Medicine; UCLA School of Public Health, Health Services; RAND Health, 11301 Wilshire Blvd. mailcode 111G, Los Angeles, CA 90073 (e-mail: Richard.Mularski@med.va.gov).
BACKGROUND: To improve pain management, the Veterans Health Administration launched the “Pain as the 5th Vital Sign” initiative in 1999, requiring a pain intensity rating (0 to 10) at all clinical encounters.
OBJECTIVE: To measure the initiative's impact on the quality of pain management.
DESIGN: We retrospectively reviewed medical records at a single medical center to compare providers' pain management before and after implementing the initiative and performed a subgroup analysis of patients reporting substantial pain (≥4) during a postimplementation visit.
PARTICIPANTS: Unique patient visits selected from all 15 primary care providers of a general medicine outpatient clinic.
MEASUREMENTS: We used 7 process indicators of quality pain management, based on appropriately evaluating and treating pain, to assess 300 randomly selected visits before and 300 visits after implementing the pain initiative.
RESULTS: The quality of pain care was unchanged between visits before and after the pain initiative (P>.05 for all comparisons): subjective provider assessment (49.3% before, 48.7% after), pain exam (26.3%, 26.0%), orders to assess pain (11.7%, 8.3%), new analgesic (8.7%, 11.0%), change in existing analgesics (6.7%, 4.3%), other pain treatment (11.7%, 13.7%), or follow-up plans (10.0%, 8.7%). Patients (n=79) who reported substantial pain often did not receive recommended care: 22% had no attention to pain documented in the medical record, 27% had no further assessment documented, and 52% received no new therapy for pain at that visit.
CONCLUSIONS: Routinely measuring pain by the 5th vital sign did not increase the quality of pain management. Patients with substantial pain documented by the 5th vital sign often had inadequate pain management.
Pain is one of the most common reasons people seek medical care. Acute pain has been reported as a presenting symptom in over 80% of physician visits and chronic pain has been estimated to be under treated in up to 80% of patients in some settings.1 In a report issued by the Department of Veteran Affairs, chronic nonmalignant pain was cited as afflicting over 34 million people with substantial cost in lost work days.2 Uncontrolled pain not only results in unnecessary suffering, but compromises the care of underlying diseases and can lead to depression, decreased enjoyment of life, and less productivity.3 In 1995, the American Pain Society (APS) set out guidelines delineating that a first step in improving the treatment of pain is assessment and recording of patients' reports of pain.4 The cited impetus for adopting this strategy is captured in the presidential address to the APS in November 1996 by James Campbell, MD, who stated that “… vital signs are taken seriously … if pain were assessed with the same zeal as other vital signs are, it would have a much better chance of being treated properly.”5
In November 1998, the Veterans Health Administration (VHA) began a national strategy to improve pain management for its patients.6,7 In a 1999 memorandum to its over 1,200 sites of medical care, the VHA initiated the measurement and documentation in the electronic medical record of patients' self-report of pain.8 The initiative called “Pain as the 5th Vital Sign” required use of a Numeric Rating Scale (NRS) for all clinical encounters.9 The NRS is a standardized 10-integer verbal and/or visual analog scale (0=no pain, 10=worst possible pain) for patient self-report of pain intensity. The initiative required that intake nurses assess pain and intensity for outpatient visits using the NRS and record the result in the medical record with the vital signs. The implementation phase of the pain-as-the-5th-vital-sign (P5VS) initiative included mandatory classroom-based provider education and supplemental materials to support practice change. Beyond the documentation of pain intensity, the initiative expected “… that a pain score of 4 or higher would trigger a comprehensive pain assessment and prompt intervention …” by the healthcare provider.9
We report an evaluation on the effect of P5VS on pain management at an outpatient general internal medical clinic within the VA health care system. Our project had 2 major aims. First, we compared pain management before and after implementation of the P5VS initiative using explicit quality indicators. Second, we evaluated the quality of pain management provided to patients in the postimplementation group who reported a substantial level of pain.
To achieve our first aim, we conducted a retrospective cross-sectional medical record review at a single Veteran Affairs medical center. We extracted data from the electronic medical record for random unique patient visits for 1 group before implementation of the initiative (preimplementation group) and for 1 group after implementation of the initiative (postimplementation group).
To achieve the second aim of a quality of care evaluation for veterans with a substantial level of pain, we analyzed the visits in the postimplementation group for patients who reported pain intensity on the NRS greater than or equal to 4. The pain level criterion of 4 was selected based on the guidelines for care established in the P5VS initiative.
We performed this study with a waiver for consent and with approval from the Institutional Review Board.
Sampling Strategy and Data Extraction
The VHA pain initiative was adopted in late 1999 but it was not fully implemented until January 2001 because of modification to the computerized patient record system (CPRS) and provider education. To cleanly separate the groups, we constructed the preimplementation group to consist of randomly selected clinic visits dating from before October 1999 and the postimplementation group to consist of visits dating after January 2001.
We extracted data from CPRS for unique outpatient visits for each of the attending general internal medicine physicians, including only 1 visit per patient. We used a balanced sampling strategy, randomly selecting 20 visits from 100 sequential eligible visits for each and every attending internal medicine physician who practiced in the nontrainee clinic throughout the study period. Data were extracted from provider progress notes, pharmacy fields, order fields, and the vital sign fields of the electronic medical record to include demographic data, pain levels, and clinical information. We measured the quality of care provided for pain management by whether or not documentation of pain assessment, evaluation, or treatment satisfied the criteria for 7 unique explicit process indicators.
Quality Indicators of Pain Management
We developed 7 explicit process indicators on the basis of VHA P5VS initiative toolkit and published standards of care for pain management (Table 1).3,9–12 The indicators were designed to measure providers' attention, evaluation, and treatment of pain. Development of the indicators involved iterative discussions of potential indicators with experts at the Center for Research on Symptom Management in Life-Threatening Illness at the Oregon Health & Science University, School of Nursing. As the first aim of our study was to compare pain management before and after implementation of the P5VS initiative, all indicators were applied to all visits, assuming that equal numbers of random visits would have indications for pain management. We scored indicators as being satisfied if any documentation existed in the medical record to demonstrate the process of care was provided, including notation of the absence of a finding or that a test or therapy was unneeded or unwanted. Three composite scores were created by combining the 7 indicators by evaluation, treatment, or any attention to pain (Table 1).
Table 1. Explicit Process Indicators for Quality Pain Management
|1. Subjective assessment||Did MD document subjective assessment in note (y/n)|
|2. Exam for pain complaint||Did MD document pain examination evaluating pain (y/n)|
|3. Any orders to assess pain||Did MD order any evaluations to assess pain (y/n)|
|4. New pain prescription||Did MD order any new medications for pain (y/n)|
|5. Change in pain prescription||Did MD change any medications for pain (y/n)|
|6. Any other pain treatment||Did MD order other therapies for pain treatment (y/n)|
|7. Follow-up plan made||Did MD document a pain follow-up plan (y/n)|
| Composite Scores|
| Composite evaluation||Any criteria met from indicators 1, 2, or 3|
| Composite treatment||Any criteria met from indicators 4, 5, or 6|
| Any attention to pain||Any of the 7 indicators were met|
Interrater reliability was assessed on the first 10% of the extracted records for the 2 independent data extractors and reviews performed by the principal investigator. We compared performance between the pre- and postgroups with χ2 analysis and with logistic models using random effects to account for clustering on providers. As no differences were found between pre- and postgroups, we aggregated data for exploratory models and used fixed effects to control for any group differences. We used multivariate logistic regression to explore for associations to the evaluation or treatment of pain, controlling for clustering within the 15 physicians using random effects. Relative risk computations were done for statistically significant covariates using simulation with 5,000 iterations for 95% confidence interval (CI).
We initially powered the study for the pre-post χ2 comparison to detect a difference of 10% between groups with an α set at 0.05 and a β at 0.20. A post hoc calculation was performed to determine the minimal difference detectable between the groups.13
As our second aim was to measure the quality of care provided to patients who reported a substantial level of pain (NRS level ≥4 as defined by the VHA initiative),9 we computed rates for how often criteria were met for the 7 process indicators and 3 summary scores.
We reviewed a total of 600 records, 300 in the preimplementation group and 300 in the postimplementation groups. Interrater reliability on the first 10% of records revealed agreement across all variables that exceeded 98%, κ levels >0.97. Missing data were less than 0.2%; we performed no imputation for missing values.
We found no differences in patient or visit characteristics between the pre- and postgroups (Table 2). The mean age of patients (aggregated between both groups) was 63 years and the majority of patients were male (94%). Overall, 56% of patients had been prescribed pain medications preceding the study visit (as documented by pharmacy records) and prior pain diagnoses were documented in 64% of the sample. For patients who had pain level recorded in any place in the medical record in either group (nurse or provider notes before P5VS in the pre group, n=94 or the 5th vital sign field in the postgroup, n=247), mean pain level was not different between groups (P=.14).
Table 2. Patient Characteristics*
|Age (y±standard deviation)||62±14||63±14|
|Caucasian race (%)||74||72|
|Male gender (%)||94||93|
|Pain diagnosis before study visit (%)||61||67|
|Visit with primary care provider (%)||99||100|
|Pain complaint as reason for visit (%)||8||5|
|Primary diagnosis (%)|
|On pain medication before study visit (%)||59||54|
|Opiate containing medications||36||32|
|Pain level recorded in P5VS field (%)||(N/A)||82 (n=247)|
|Pain level recorded anywhere in record (%)¶||31 (n=94)||82 (n=247)|
|Mean pain level (score on 0 to 10 scale±standard deviation)||1.8±2.8 (n=94)||2.4±3.1 (n=247)|
For those records in the postimplementation group, 82% of patients had their pain level recorded in the 5th vital sign field. Forty-five percent of patients reported any pain and 32% reported considerable pain greater than or equal to 4.
Pre- Versus Postcomparison
The main result of our study is that we were unable to detect any improvements in pain management after P5VS (Table 3). The proportion of medical records meeting the quality criteria for pain management were equivalent across all 7 explicit process quality indictors and composite scores before and after implementation of the P5VS initiative, P>.05 for all comparisons. Post-hoc analysis indicated that we had sufficient power to detect differences greater than 10% between the groups. New or changed opiate prescriptions were documented in less than 1% of patients in each group.
Table 3. Comparison of Care Between Preintervention and Postintervention Groups
|Exam for pain complaint||26.0||26.0||.924|
|Other orders to assess pain||11.7||8.3||.171|
|New pain prescription||8.7||11.0||.331|
|Change existing pain meds||6.7||4.3||.208|
|Any other form of treatment||11.7||13.7||.461|
|Follow-up plan made||10.0||8.7||.567|
|Any attention to pain||52.0||53.0||.802|
Quality of Care for Substantial Pain
Seventy-nine patients (32%) in the postimplementation group reported a substantial level of pain greater than or equal to 4 on the NRS. The average pain level in this subgroup was 6.5 (SD 1.9). For these patients with substantial pain, 22% had no attention to pain documented in the medical record by the provider, 27% had no evaluation documented, and 52% received no new or adjusted therapy for pain at that visit (Table 4). We identified only 1 new prescription of opiate-containing medications and 3 dose adjustments to opiate-containing medications among those in this substantial pain subgroup.
Table 4. Quality of Pain Management for Patients with Substantial Pain (NRS ≥4)
|Exam for pain complaint||38.0|
|Any orders to assess pain||15.2|
|New pain prescription||21.5|
|Change in pain prescription||11.4|
|Any other pain treatment||22.8|
|Follow-up plan made||12.7|
|Any attention to pain||78.5|
Associations to Assessment and Treatment of Pain
Across the overall sample, patients who had a provider evaluation were 10 times more likely to have received pain therapy at the visit (P<.0001 by bivariate analysis). There were no differences in evaluation or therapy by individual providers; α levels for bivariate correlation between providers and indicators ranged from P=.27 to P=.57. Models exploring associations to provider evaluation and treatment revealed few relationships (Table 5). Logistic regression suggested increased provider evaluation was associated with pain level and the patient having a prior pain disorder; provider evaluation was negatively related to the patient's age. Provider treatment of pain was associated with pain level, having a visit reason expressly for pain, and the presence of a prior pain disorder.
Table 5. Regression Results Exploring for Associations to Pain Evaluation and Treatment
|Age of patient at time of visit||−0.005||.012||0.98 (0.97, 0.997)†|
|Pain diagnosis before study visit||0.230||<.001||1.09 (1.05, 1.14)|
|Visit with primary care provider||0.463||.239||—|
|Pain complaint as reason for visit||0.354||<.001||1.13 (1.06, 1.19)|
|Pain level (any recorded level)||0.037||<.001||1.06 (1.06, 1.08)‡|
|Age of patient at time of visit||−0.003||.085||—|
|Pain diagnosis before study visit||0.100||.049||1.04 (1.00, 1.09)|
|Visit with primary care provider||0.300||.477||—|
|Pain complaint as reason for visit||0.260||.006||1.11 (1.03, 1.19)|
|Pain level (any recorded level)||0.037||<.001||1.06 (1.04, 1.09)‡|
We found that the P5VS initiative by itself did not improve the quality of pain management in an outpatient internal medicine clinic as assessed by our 7 basic process indicators. The measurement and documentation of patients' pain level is a necessary aspect of pain management. However, we were unable to detect any improvements in evaluation and treatment of pain in the year after the P5VS initiative was implemented at a single VA institution. Moreover, for patients who had pain levels greater than or equal to 4 at clinic intake, we demonstrated substantial deficits in the evaluation and therapy of pain. Over one-fifth of patients who reported substantial pain had no attention to pain in the medical record by the providers in our study and fewer than half of these patients had therapeutic interventions at the time of their clinic visit. Our investigation of the P5VS initiative at a single VA site has shown the routine documentation of pain levels, even with system-wide support and broad-based provider education, was ineffective in improving the quality-of-care.
The VHA adopted the logical first step of requiring measurement in the P5VS initiative to identify patients with pain, as did the Joint Commission of Accreditation of Healthcare Organizations and others.11 The underlying assumption is that measurement, identification, and documentation of patients in pain should lead to improved management.5,14 However, efficacy trials of this strategy have not been performed to date and our results suggest that other strategies are needed to change provider behavior.
What opportunities for improvement do these data suggest? Of medical records reviewed after the initiation of the P5VS initiative, 82% had a pain level recorded in the vital sign field. Our study documented a lower compliance with recording of the pain 5th vital sign than prior testing with the initiative (Tollett JH. R.E. Research on pain 5th vital sign, July 31, 2002, personal email communication). However, the ineffectiveness of the initiative on quality of care is not likely to be explained simply on the basis of missing 18% of pain measurement opportunities.
Qualitative studies that interview physicians may be able to identify barriers or targets for quality of pain management improvement. Informal discussions with practitioners at the study site and at other VA clinics across the country have suggested to our research team that some practitioners are not aware of the 5th vital sign field or do not have the pain level available at the time of patient interview. A number of practitioners reported that they receive a paper sheet after clinic intake with basic patient information and only the other 4 vital signs for use during the visit. Some providers noted that they did not use the electronic medical record (and hence do not have the possibility of reviewing the pain 5th vital sign) until after the patient had left and they were charting the visit. Such comments suggest that the utility of the pain 5th vital sign may have been undermined by failures in communication that persist even with documentation. These possibilities should be explored formally in future research and quality improvement activities. Our link between physician's subjective documentation and increased treatment suggest that once a provider performs a pain evaluation, there is a greater chance of management.
The P5VS initiative used both an adaptation of the electronic record and provider education to attempt to change provider behavior. Prior studies have shown that such strategies often fail to change provider behavior in the absence of systematic organizational change.15,16 Moreover, pain treatment may have unique challenges because of various barriers to effective pain management from both patients and providers.3,17 Improvements in the quality of pain management will likely require multimodality quality improvement and provider change strategies.18 To improve the quality of pain management, the VA and other health care systems should investigate overlaying other methods like audit/feedback and multidisciplinary Plan-Do-Study-Act improvement cycles as well as improved use of information technology such as clinical alerts and decision support.15,16,19 Since 1999, the VHA has broadened the approach to improving pain management consistent with the efforts it has made in improving other areas of care.7,20–25
Although a small effect, our finding that evaluation of pain decreased with increasing age replicates findings of under-identification and under-treatment of pain in older adults from previous studies.17,26–28 Because of the tendency of older adults to under-report pain, a proactive stance by primary practitioners is needed to identify and effectively manage pain.17
Our finding of deficient care for those with substantial pain in the postimplementation group might be tempered by the finding that 56% of patients were already taking pain medications at baseline. It is conceivable that our examination of a single visit did not adequately reflect the provider's attention and management of pain. However, for those patients who reported substantial pain levels, prompt intervention would be indicated. The initiation of treatment in only 48% of patients at the time of a visit with pain ≥4 is comparable to the recent national study that documented that patients receive only 55% of recommended care on average.29
There are several limitations to our study. First, there is much more to pain management than the 7 simple criteria evaluated in this study. A more complete study of pain management would evaluate a larger set of process measures and ideally use a measure of palliation to determine if assessment, evaluation, or treatment strategies resulted in changes in outcomes, such as the improvement in patient suffering.21,30
Second, our study employed a retrospective medical record review, which carries the risk of missing important performance factors or undocumented clinician behavior. However, several investigations have validated the medical record review method.31,32 Further, the P5VS initiative clearly required documentation of pain management as part of the process of care.
Third, the pre-post design opens the possibility that temporal and secular differences may have affected the results independent of the P5VS initiative. However, as no differences were detected, it is unlikely that such influences importantly affected the results or conclusions of the study. Last, our study was performed in a single clinic at 1 institution and may not generalize to other settings. Our results should be verified in other care settings and geographic locations and explore other aspects of provider behavior change include physician characteristics.
Our evaluation of the P5VS strategy indicates that routine documentation of pain intensity, while necessary for quality care, may not be sufficient by itself to improve the quality of pain management. Pain evaluation and treatment remains an important focus for medical quality improvement. Additional interventions are needed to improve providers awareness of patients' pain and to increase the rates at which they provide appropriate therapy.
We are grateful to the following individuals who contributed to the completion and analysis of this project: Molly Delorit, Theresa Demadura, Anne Rosenfeld, Theresa A. Harvath, Lillian Nail, Jonathan Fields, Nancy Perrin, Jane Tollett, Tuyen Hoang, Lisa Rubenstein, Elizabeth McGlynn, and Yelena Kholodenko. This study was supported as a subproject under a grant from the National Institutes of Nursing Research Exploratory Nursing Research Center Grant 1P20NR0780. The work originated at the Portland Veterans Affairs Medical Center and Oregon Health & Science University, Department of Medicine and School of Nursing; the work was performed at the Portland Veterans Affairs Medical Center and the Veterans Affairs Greater Los Angeles Healthcare System.