SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. The purpose of performance indicators
  5. Approaches to measuring quality and performance
  6. Moving from mortality to non-mortality outcomes
  7. Moving from short-term to long-term outcome measurement
  8. Moving from hospital performance to system performance
  9. Discussion
  10. Acknowledgements
  11. References

Background:

Valid and reliable measures of trauma system performance are needed to guide improvement activities, benchmarking and public reporting, future investment and research. Traditional measures of in-hospital mortality fail to take into account prehospital and posthospital care, recovery after discharge, and the nature and costs of long-term disability.

Methods:

Drawing on recent systematic reviews, an overview was conducted of existing and emerging trauma care performance indicators. Changes in the nature and purpose of indicators were assessed.

Results:

Among a large number of existing, mostly locally developed performance indicators, only peer review of deaths has evidence of validity or reliability. The usefulness of the traditional performance measure of in-hospital mortality has been challenged. There is an emerging shift in focus from mortality to non-mortality outcomes, from hospital-based to long-term community-based outcome assessment, and from single measures of trauma centre performance to measures better suited to monitoring the performance of systems of care spanning the entire patient journey. As a result, a new generation of indicators is emerging that are both feasible and potentially more useful for commissioners and payers of population-based services.

Conclusion:

A global endeavour is now under way to agree on a set of standardized performance indicators that are meaningful to patients, carers, clinicians, managers and service funders, are likely to contribute to desired outcomes, and are valid, reliable and have a strong evidence base. Copyright © 2011 British Journal of Surgery Society Ltd. Published by John Wiley & Sons, Ltd.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. The purpose of performance indicators
  5. Approaches to measuring quality and performance
  6. Moving from mortality to non-mortality outcomes
  7. Moving from short-term to long-term outcome measurement
  8. Moving from hospital performance to system performance
  9. Discussion
  10. Acknowledgements
  11. References

Systems of trauma care are necessary to resuscitate, repair and rehabilitate injured people in order to reduce mortality, improve the quality of survival and lessen the overall burden of injury. Measurement and feedback of performance are integral to the concept of a system of care.

Realization that death due to suboptimal trauma care was preventable has led to the development of trauma systems and, in particular, the designation of major trauma centres in the USA, since the 1970s1. Preventable death rates and risk-adjusted mortality rates have since become standard methods of measuring and comparing trauma centre and trauma system performance. With improvements in trauma care, the mortality rate among severely injured people treated in mature trauma centres has declined from one in three to around one in ten, and preventable death rates have declined from one in three to less than one in ten, and in some centres as low as one in 502.

Inpatient mortality is an easy outcome to define and feasible to collect. However, as preventable mortality has diminished, so too has the usefulness of mortality as an indicator of performance; it is unlikely that future improvements in care will result in substantial further reductions in mortality.

Furthermore, inpatient mortality says little about the burden of injury that is felt both in and outside of hospitals, the cost of which is borne by patients, governments, insurers and the community. Although historically useful, mortality alone is no longer an adequate performance measure for a system of trauma care, nor is it acceptable to limit the measurement of the impact of an entire trauma system to that of a major trauma centre.

This overview highlights the main developments and challenges in measuring improvements in care in established trauma systems. It describes the limitations of inpatient mortality as a trauma care performance measure, and the paucity of validity and reliability evidence for existing structure, process and outcome indicators. Furthermore, the indicator requirements of various stakeholders are considered, ranging from patients and their families, to clinicians and managers and payers. In addition, an outline is presented of why it is important to pursue performance indicators that include quality of survival as well as survival, that go beyond the acute hospital episode to reflect the long-term results of acute and rehabilitation care and the stable long-term disability needs of the patient, and that reflect performance of important aspects of a trauma system before, during and after hospital care.

The purpose of performance indicators

  1. Top of page
  2. Abstract
  3. Introduction
  4. The purpose of performance indicators
  5. Approaches to measuring quality and performance
  6. Moving from mortality to non-mortality outcomes
  7. Moving from short-term to long-term outcome measurement
  8. Moving from hospital performance to system performance
  9. Discussion
  10. Acknowledgements
  11. References

Performance monitoring can serve several purposes, and appropriate indicators are best chosen according to the intended purpose (Fig.1).

thumbnail image

Figure 1. Pictorial representation of the injured patient's journey from injury to death, recovery, or long-term disability. The top row depicts examples of structure, process and outcome indicators. The middle row depicts composite indicators that encompass several components of the journey, such as time-critical care, and recovery and long-term disability needs. The bottom row emphasizes the need to evaluate trauma system performance using a range of indicators. GOS-E, extended version of the Glasgow Outcome Scale; QoL, quality of life

Download figure to PowerPoint

Internal use for quality improvement

Driven by clinicians or by managers, performance indicators are useful within a system for monitoring outcomes or compliance with guidelines and particular aspects of care, and benchmarking performance over time. Prehospital time, rates of pulmonary emboli and compliance with venous thromboembolism prophylaxis are examples. Within a stable system these indicators do not always need risk adjustment.

External presentation for benchmarking purposes

Risk adjustment becomes particularly important when comparing centres or systems of care. Differences in patient case mix and delivery of trauma care, over time or across trauma systems, have an impact on the capacity to achieve valid benchmarking. Valid benchmarking requires detailed knowledge of factors that affect the outcome of interest (such as patient age, injury severity, co-morbid status), and adjustment for these factors in any analysis or presentation of data, to provide an accurate estimate of the association between trauma centre (or trauma system) and outcome.

Commissioners' and/or payers' use to determine future investment

In systems in which investment is tied to performance—whether through performance-based payments or funders aiming to get value for money more generally—indicators need to be chosen carefully to reflect important care goals. Although performance-based payments can be useful to elicit change, undesired and unexpected effects can result and should also be monitored.

Research

Particularly when standardized in format and data collection, indicators can enable higher-quality comparative studies of interventions such as compensation systems that are unsuitable for randomization.

Approaches to measuring quality and performance

  1. Top of page
  2. Abstract
  3. Introduction
  4. The purpose of performance indicators
  5. Approaches to measuring quality and performance
  6. Moving from mortality to non-mortality outcomes
  7. Moving from short-term to long-term outcome measurement
  8. Moving from hospital performance to system performance
  9. Discussion
  10. Acknowledgements
  11. References

To examine trauma system performance in providing quality care to injured patients, a conceptual model such as Donabedian's framework of quality of care is helpful. In Donabedian's framework, the three components of healthcare quality are structure, process and outcome3. Their relationship is depicted in Fig.2. The structure is the environment in which healthcare is provided, and includes material and health resources, operational factors, and organizational characteristics of the healthcare facility. For example, designation of trauma centres, trauma centre volume, prehospital notification and presence of a quality improvement programme have been identified as independent determinants of Canadian trauma centre in-hospital mortality4. The process is the method by which healthcare is provided, and includes the giving and receiving of care by the providers and healthcare system. The outcome is the consequence of healthcare, and includes the health status of patients and communities. Prehospital, hospital, posthospital and prevention performance indicators can be chosen to capture the spectrum of care from the moment of injury to recovery; examples are shown in Table1.

thumbnail image

Figure 2. Relationship between the structure, process and outcome of healthcare.

Download figure to PowerPoint

Table 1. Conceptual model of performance measurement in trauma care
Phase of careStructureProcessOutcome
  1. Table shows examples of performance measures. Some measures, such as pain, may cut across all domains. For example, a protocol for pain scoring and management is a prehospital, hospital and posthospital structural indicator; compliance with protocol is a process measure; and pain score at 6 months after discharge is an outcome measure. GOS-E, Glasgow Outcome Scale–extended.

PrehospitalDispatch criteriaTotal prehospital time with component partsPrehospital deaths
 Training Long-term outcomes
 Prehospital triage  
HospitalMassive transfusion protocolAppropriate activation of massive transfusion protocolDeaths due to haemorrhagic shock
 Head injury protocol Risk-adjusted mortality for head injury
   GOS-E at 6 and 12 months after injury
  Proportion of eligible patients receiving protocol-based management 
PosthospitalRehabilitation facility in communityTime to rehabilitation consultationsReturn to work, adjusted for severity
 Standardized rehabilitation protocols  
PreventionInjury prevention activitiesProportion of eligible patients receiving alcohol screening and brief interventionProportion of patients returning with new alcohol- or drug-related injuries

A very large number of such indicators have been defined and used in trauma care around the world. A scoping review identified 1572 indicators that could be classified into eight categories: non-American College of Surgeons Committee on Trauma (ACS-COT) audit filters (42·0 per cent), ACS-COT audit filters (19·1 per cent), patient safety indicators (13·2 per cent), trauma centre/system criteria (10·2 per cent), indicators measuring or benchmarking outcomes of care (7·4 per cent), peer review (5·5 per cent), general audit measures (1·8 per cent), and guideline availability or adherence (0·8 per cent). Measures of prehospital and hospital processes (60·4 per cent) and outcomes (22·8 per cent) were the most common, and posthospital and secondary injury prevention indicators accounted for less than 5 per cent5.

Based on appraisal of the validity and reliability of identified indicators6, 7, strengths and weaknesses of existing performance measurement are apparent. First, a common set of clearly defined, evidence-based and broadly accepted performance measures for evaluating the quality of trauma care does not exist. Instead, many different indicators are used. It is not possible to have a systematic data-driven performance measurement and feedback system if there are no generally agreed measures. Second, originating from surgical audit, quality improvement efforts in trauma care have historically been institution-specific. Performance measures should also serve state or national goals. Third, current performance measures have been developed largely to address the measurement needs of providers, but have failed to capture patient care along the journey from injury to recovery. For example, most published performance indicators examine prehospital and hospital processes and outcomes of care. There has been limited development of measures to examine posthospital care or injury prevention.

Further, there is little evidence to support the use of most current performance indicators in trauma care. For example, only a single measure—peer review for preventable death—has both reliability and validity evidence7. In paediatrics, even less evidence is available, with most measures originating from the adult literature and subsequently applied to paediatric care6. Clinicians caring for patients with traumatic injuries do not have the same evidence base to draw upon in developing quality indicators.

Finally, developing performance indicators for diseases or disorders that are homogeneous is easier than for those that are heterogeneous. Trauma is not a disease, but a collection of injuries; performance indicators may therefore need to focus on common pathways and outcomes that are important to patients.

Choosing important, usable, scientifically sound and feasible indicators

Through consultation with developers and users, for the Agency for Health Research and Quality, Hussey and colleagues8 identified four broad attributes for evaluating performance indicators: importance, usability, scientific soundness and feasibility. Important indicators are those that are relevant to stakeholders, and show potential for improvement. The structures and processes measured should be those that contribute, or are likely to contribute, to desired outcomes. Things that make indicators usable, for example, include standardized templates and clear guidance for public reporting or pay-for-performance programmes. Scientifically sound indicators have a strong evidence base, are reliable and valid, allow for patient/consumer perspectives, and together form a comprehensive evaluation of the quality of trauma care. Feasible indicators are those for which clearly specified numerator and denominator data are readily available from administrative data, trauma registries, linked databases or feasible purpose-designed data sources.

Mortality measures

In-hospital mortality is a feasible and reliable outcome, but there are significant problems with interpreting this outcome for hospital and system benchmarking purposes. Trauma deaths may occur before reaching hospital, during interhospital transfer or during convalescence from the acute hospital stay; therefore, in-hospital deaths represent only part of the overall picture. Measurements of severity of injury are not coded in a standard way, the whole spectrum of Abbreviated Injury Scale versions are used, coding may be translated from International Classification of Diseases data sets, and training and auditing are inconsistent. Attempts at risk adjustment using the Trauma and Injury Severity Score and other scores have been shown to be unreliable9, 10 because of the coarse categorization of risk (for example age over 55 years) and inconsistent attempts at managing missing data11. If in-hospital mortality is to be used as a performance measure for benchmarking, international agreement is needed on standardization of reporting12.

Moving from mortality to non-mortality outcomes

  1. Top of page
  2. Abstract
  3. Introduction
  4. The purpose of performance indicators
  5. Approaches to measuring quality and performance
  6. Moving from mortality to non-mortality outcomes
  7. Moving from short-term to long-term outcome measurement
  8. Moving from hospital performance to system performance
  9. Discussion
  10. Acknowledgements
  11. References

The burden of traumatic injury is multidimensional; Lyons and co-workers13 have described 20 groups of injury consequences: 12 related to the individual, three relevant to family and close friends, and five to the wider society (Table2). Although consensus statements exist, the recommendations for selection of instruments for measuring important outcomes and time points differ, and the relevance to system performance and quality improvement has not been addressed14–16.

Table 2. List Of All Deficits (LOAD) framework of injury consequences13
Individual
 Death, including fetal
 Pain and discomfort
 Reduced short-term physical activity
 Long-term physical disability
 Psychological disability
 Concomitant diseases
 Development of secondary conditions
 Behavioural change and secondary health loss
 Fear of repeated injury
 Tangible costs
 Intangible costs
 Decreased quality of life
Family
 Observer consequences
 Carer consequences
 Dependant consequences
Societal
 Societal fear of injury
 Psychological consequences in observers
 Copycat events
 Direct medical costs
 Indirect costs

Function and quality of life

Although arguments have been made that functional and quality-of-life (QoL) outcomes should have a higher priority than mortality17, the uptake of routine collection of these outcomes by trauma registries and trauma systems has been poor. The Functional Capacity Index remains the only disease-specific measure of function developed for a general trauma context18, 19, but its validity for predicting long-term outcome has been questioned20 and none of the consensus statements has recommended its inclusion in injury outcome studies14–16. The Functional Independence Measure (FIM), commonly used in rehabilitation settings, may be a useful measure of function at hospital discharge21, but lacks the responsiveness to change of more comprehensive measures of function such as the Glasgow Outcome Scale (GOS) and its extended version (GOS-E)22. The GOS scales consider higher levels of functioning and participation, such as community participation, transport, return to work, relationships, and participation in social and leisure activities, factors important to patients, caregivers and society that are not included in the FIM.

Generic measures of health status or health-related QoL are recommended for use in trauma populations16, 23. These measures enable comparison of functioning of injured patients with that of other disease groups and the general population. There are numerous instruments available, including the 12- and 36-item Short Form Health Surveys (SF-12® and SF-36®; QualityMetric, Lincoln, Rhode Island, USA), the EQ-5DTM (EuroQol Group, Rotterdam, The Netherlands), the Quality of Wellbeing Scale and the Health Utilities Index. For each instrument, societal and cultural differences, and the availability of local population norms may limit comparisons. For example, the consensus statement by van Beeck and colleagues24 recommended the use of the EQ-5DTM for injury outcome studies, but its utility is unknown in populations that lack appropriate population norms. In contrast, population norms for most countries are available for the SF-36® and SF-12® instruments, although they may not cover particular population subgroups, including those in which injury is more prevalent25. The purpose of benchmarking should be considered before selecting generic health status measures and, in particular, which groups will be compared and how comparisons will be made.

Benchmarking of trauma centre or system performance on non-mortality outcomes has been limited to short-term morbidity (such as complications) and function at discharge26–28. Function at discharge will vary widely depending on the instrument chosen. Instruments performing well at discharge provide limited capacity to measure change and recovery21, 22. Studies using function at discharge to evaluate quality of care should be interpreted cautiously given the wide variation in discharge practices, availability of rehabilitation beds and patient case mix21. Benchmarking of functional or health-related QoL outcomes should be performed at standard time points after injury, rather than at poorly standardized time points such as hospital or rehabilitation discharge, as length of stay varies considerably.

Moving from short-term to long-term outcome measurement

  1. Top of page
  2. Abstract
  3. Introduction
  4. The purpose of performance indicators
  5. Approaches to measuring quality and performance
  6. Moving from mortality to non-mortality outcomes
  7. Moving from short-term to long-term outcome measurement
  8. Moving from hospital performance to system performance
  9. Discussion
  10. Acknowledgements
  11. References

The use of inpatient outcome measures is inadequate for several reasons. First, severely injured patients rarely achieve a steady state by the time they are discharged from hospital, and status at discharge fails to take into account the care provided and improvements made after discharge from an acute hospital. The limited available data to date suggest that improvement has slowed and most patients have reached a steady state by 12 months after injury29–31.

Second, inpatient mortality and function at discharge are affected by discharge practices, availability of rehabilitation beds and patient case mix21. Mortality over 1 year after injury is likely to be more meaningful than inpatient mortality. For example, among 124 421 patients treated in a mature trauma system and coded in the Washington State Trauma Registry between 1995 and 2008, 5·8 per cent died during the initial hospital admission; however, by 1, 2 and 3 years, 9·8, 13·2 and 16·0 per cent respectively had died32. This represented excess postinjury mortality at 3 years of 10 per cent above age- and sex-adjusted norms. Furthermore, although the in-hospital mortality rate decreased from 8 to 4·5 per cent between 1995 and 2008, mortality after discharge increased commensurately, such that the overall 1-year mortality rate after injury was unchanged.

A third reason for extending the period of outcome measurement is that inpatient mortality correlates poorly with costs to payers. The least costly patients are those who recover well and regain independence, and those who die early. The most expensive are those who survive but have permanent severe disabilities, do not return to work and require prolonged attendant care. Systems that compensate for the healthcare costs, and lifetime healthcare and disability costs of severely injured people have found that the majority of long-term liabilities relate to the disability care of permanently incapacitated clients33.

Numerous studies have described the long-term functional and health-related QoL outcomes following trauma for individual cohorts34. Mackenzie and co-workers35 compared trauma centre care with non-trauma centre care using 12-month functional outcomes. Yet few jurisdictions routinely use such measures for monitoring performance and improvement. Questions of feasibility of collecting this information have been raised, such as the selection of instruments for follow-up and the potential for bias related to loss to follow-up. However, these challenges should not be considered insurmountable36.

Recently, Gabbe and colleagues37 described the development and implementation of population monitoring of functional and QoL outcomes at 6, 12 and 24 months after major trauma through the Victorian State Trauma Registry (VSTR). The key elements of the VSTR methodology are listed in Table3.

Table 3. Key elements of a successful registry that includes long-term functional and quality-of-life outcomes of injured patients
  1. Characteristics derived from the Victorian State Trauma Registry37. QoL, quality of life.

An opt-out consent process whereby all eligible patients are included, and sent a letter and brochure describing the registry's purpose, that patients will be followed up by telephone, and details of how to opt-out of the registry
Collection of both patient and next-of-kin contact details
Telephone interview of patients or their next of kin, rather than mailing self-administered questionnaires, to limit loss to follow-up
Brief (approximately 10–15 min) follow-up interview including functional and health-related QoL measures, and a proxy version in which health-related QoL measures are excluded
Centralization of telephone interviewing to ensure supervision and quality control of interviews
Telephone interviewers with patient contact and/or clinical experience

The VSTR approach has been implemented at relatively low cost (approximately 30 000 Australian dollars per time point for 2000 patients who suffered major trauma), low opt-out rates (less than 1 per cent) and high follow-up rates (about 85 per cent)37. Importantly, across the system, there is early evidence of improvement in patient function over time, as well as continued reductions in mortality, reinforcing the importance of benchmarking on outcomes other than mortality38. The experience of the VSTR has shown that the collection of long-term functional and QoL outcomes across a trauma system is feasible, relatively inexpensive and useful, if investment is made in the integration of such monitoring into the trauma system.

Moving from hospital performance to system performance

  1. Top of page
  2. Abstract
  3. Introduction
  4. The purpose of performance indicators
  5. Approaches to measuring quality and performance
  6. Moving from mortality to non-mortality outcomes
  7. Moving from short-term to long-term outcome measurement
  8. Moving from hospital performance to system performance
  9. Discussion
  10. Acknowledgements
  11. References

Measuring performance in one acute hospital involving multiple services (for example emergency, intensive care, surgery, anaesthetics) is difficult. Measuring and monitoring processes and outcomes across a system of care that involves prehospital care, acute care, rehabilitation and community services is even more complicated. However, to assess truly whether outcomes for injured patients are improving over time and where the opportunities for improvement lie, systemwide monitoring is essential. This can happen effectively only with linkage of databases containing crash data, ambulance service data, administrative and discharge data from hospitals within the region, data on rehabilitation services, coroner findings and insurance data. This information should be linked with registry data that include validated long-term disability outcome measures.

Once a system monitoring process is in place, it is possible to measure the effects of key processes such as trauma triage, transfer protocols, new resuscitation procedures, compensation payments and rehabilitation. Given the cost of providing this care and the potential improvements in outcome, the cost of systemwide monitoring is relatively low. It is important that there is proper analysis and governance of this data collection process to ensure that the data drive system change. Carefully selected indicators show promise in the goal of system-level monitoring. Examples include composite indicators, such as time to haemorrhage control, that bring together several elements of the patient journey and allow for various context-specific ways of achieving the same goal.

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. The purpose of performance indicators
  5. Approaches to measuring quality and performance
  6. Moving from mortality to non-mortality outcomes
  7. Moving from short-term to long-term outcome measurement
  8. Moving from hospital performance to system performance
  9. Discussion
  10. Acknowledgements
  11. References

In established trauma systems the need to pursue developments in quality indicators is pressing; without doing so, the effects of quality improvement initiatives on long-term patient outcomes, the requirements of health and other services, and costs to the system will remain unknown. Settings with reasonable quality data sources and sufficient resources for follow-up should aim to develop the capacity to collect long-term outcome data on survivors, link hospital data to other data sets, and work with their jurisdictions to determine key indicators and how they will be used.

An urgent international effort is required to agree on standard indicators and risk adjustment techniques. A suite of structure, process and outcome indicators is needed that can facilitate system performance monitoring and benchmarking everywhere. Efforts are under way to foster such agreement.

As highlighted, there is also a need to improve the science behind the development, validation and use of indicators. It is no longer acceptable for institutions to rely solely on locally developed indicators that have not been subjected to rigorous validity and reliability testing. Although structure and process indicators are very useful in understanding key aspects of care and in their more rapid turnaround compared with long-term outcome measures, their relationship to important outcomes must be defined and validated. For the measurement of system performance there may be a place for development and testing of composite process indicators, such as time to haemorrhage control for patients with haemorrhagic shock, or compliance with condition-specific best practice care bundles.

In jurisdictions without developed trauma systems or established hospital-based data systems, performance improvement will rely on morbidity and mortality review, limited audit of specific evidence-based processes, and peer review of preventable deaths and other adverse events. Such methods are globally applicable trauma quality improvement techniques that can be instituted in even the least developed settings39. Excellent examples exist of the use of these techniques to begin effective quality improvement programmes in low- and middle-income countries40, 41, although they give little or no opportunity for benchmarking of performance within or between centres and systems of care. The use of local preventability audits should, therefore, not preclude efforts to establish a trauma registry that links with population-based injury data, and can capture prehospital information and postdischarge outcomes. This type of measurement requires concerted efforts at city, state and national levels, and cannot be left solely to individual acute care facilities, although they may be initiators and drivers of the process.

With a standardized international approach to the development of well constructed registries, a system of meaningful and useful performance indicators that facilitate comparison, improvement and resource allocation decisions is possible. Doing so will also help to build an evidence base about effective ways of improving trauma care given that many potential systemwide interventions are unsuitable for randomized trials, and instead rely on valid comparisons from high-quality registry data.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. The purpose of performance indicators
  5. Approaches to measuring quality and performance
  6. Moving from mortality to non-mortality outcomes
  7. Moving from short-term to long-term outcome measurement
  8. Moving from hospital performance to system performance
  9. Discussion
  10. Acknowledgements
  11. References

R.L.G. is supported by a Career Development Award from the National Health and Medical Research Council. B.J.G. is supported by a Postgraduate Award from the National Health and Medical Research Council. H.T.S. is supported by a New Investigator Award from the Canadian Institutes of Health Research. P.A.C. is supported by a Practitioner Fellowship Award from the National Health and Medical Research Council.

Disclosure: The authors declare no conflict of interest.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. The purpose of performance indicators
  5. Approaches to measuring quality and performance
  6. Moving from mortality to non-mortality outcomes
  7. Moving from short-term to long-term outcome measurement
  8. Moving from hospital performance to system performance
  9. Discussion
  10. Acknowledgements
  11. References
  • 1
    West JG, Trunkey DD, Lim RC. Systems of trauma care. A study of two counties. Arch Surg 1979; 114: 455460.
  • 2
    Gruen RL, Jurkovich GJ, McIntyre LK, Foy HM, Maier RV. Patterns of errors contributing to trauma mortality: lessons learned from 2594 deaths. Ann Surg 2006; 244: 371380.
  • 3
    Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966; 44(Suppl): 166206.
  • 4
    Liberman M, Mulder DS, Jurkovich GJ, Sampalis JS. The association between trauma system and trauma center components and outcome in a mature regionalized trauma system. Surgery 2005; 137: 647658.
  • 5
    Stelfox HT, Bobranska-Artiuch B, Nathens A, Straus SE. Quality indicators for evaluating trauma care: a scoping review. Arch Surg 2010; 145: 286295.
  • 6
    Stelfox HT, Bobranska-Artiuch B, Nathens A, Straus SE. A systematic review of quality indicators for evaluating pediatric trauma care. Crit Care Med 2010; 38: 11871196.
  • 7
    Stelfox HT, Straus SE, Nathens A, Bobranska-Artiuch B. Evidence for quality indicators to evaluate adult trauma care: a systematic review. Crit Care Med 2011; 39: 846859.
  • 8
    Hussey PS, Mattke S, Morse L, Ridgeley MS. Evaluation of the Use of AHRQ and Other Quality Indicators. AHRQ Publication No. 08-M012-EF. Agency for Health Research and Quality: Rockville, 2007; http://www.ahrq.gov/about/evaluations/qualityindicators/qualityindicators.pdf [accessed 1 May 2011].
  • 9
    Gabbe BJ, Cameron PA, Wolfe R. TRISS: does it get better than this? Acad Emerg Med 2004; 11: 181186.
  • 10
    Demetriades D, Chan LS, Velmahos G, Berne TV, Cornwell EE III, Belzberg H et al. TRISS methodology in trauma: the need for alternatives. Br J Surg 1998; 85: 379384.
  • 11
    O'Reilly GM, Jolley DJ, Cameron PA, Gabbe B. Missing in action: a case study of the application of methods for dealing with missing data to trauma system benchmarking. Acad Emerg Med 2010; 17: 11221129.
  • 12
    Cameron PA, Palmer C. Developing consensus on injury coding. Injury 2011; 42: 1011.
  • 13
    Lyons RA, Finch CF, McClure R, van Beeck E, Macey S. The injury List of All Deficits (LOAD) Framework—conceptualizing the full range of deficits and adverse outcomes following injury and violence. Int J Inj Contr Saf Promot 2010; 17: 145159.
  • 14
    Bouillon B, Kreder HJ, Eypasch E, Holbrook TL, Mayou R, Nast-Kolb D et al. Quality of life in patients with multiple injuries—basic issues, assessment, and recommendations. Restor Neurol Neurosci 2002; 20: 125134.
  • 15
    Neugebauer E, Bouillon B, Bullinger M, Wood-Dauphinée S. Quality of life after multiple trauma—summary and recommendations of the consensus conference. Restor Neurol Neurosci 2002; 20: 161167.
  • 16
    Van Beeck EF, Larsen CF, Lyons RA, Meerding WJ, Mulder S, Essink-Bot ML. Guidelines for the conduction of follow-up studies measuring injury-related disability. J Trauma 2007; 62: 534550.
  • 17
    Mann NC, Mullins RJ. Research recommendations and proposed action items to facilitate trauma system implementation and evaluation. J Trauma 1999; 47(Suppl): S75S88.
  • 18
    MacKenzie EJ, Damiano A, Miller T, Luchter S. The development of the Functional Capacity Index. J Trauma 1996; 41: 799807.
  • 19
    MacKenzie EJ, Sacco WJ, Luchter S, Ditunno JF, Staz CF, Gruen GS et al.; Pennsylvania Study Group on Functional Outcomes Following Trauma. Validating the Functional Capacity Index as a measure of outcome following blunt multiple trauma. Qual Life Res 2002; 11: 797808.
  • 20
    Schluter PJ, Neale R, Scott D, Luchter S, McClure RJ. Validating the functional capacity index: a comparison of predicted versus observed total body scores. J Trauma 2005; 58: 259263.
  • 21
    Gabbe BJ, Simpson PM, Sutherland AM, Williamson OD, Judson R, Kossmann T et al. Functional measures at discharge: are they useful predictors of longer term outcomes for trauma registries? Ann Surg 2008; 247: 854859.
  • 22
    Williamson OD, Gabbe BJ, Sutherland AM, Wolfe R, Forbes AB, Cameron PA. Comparing the responsiveness of functional outcome assessment measures for trauma registries. J Trauma 2011; 71: 6368.
  • 23
    Gabbe BJ, Williamson OD, Cameron PA, Dowrick AS. Choosing outcome assessment instruments for trauma registries. Acad Emerg Med 2005; 12: 751758.
  • 24
    van Beeck EF, Larsen CF, Lyons RA, Meerding WJ, Mulder S, Essink-Bot ML. Guidelines for the conduction of follow-up studies measuring injury-related disability. J Trauma 2007; 62: 534550.
  • 25
    Scott KM, Sarfati D, Tobias MI, Haslett SJ. A challenge to the cross-cultural validity of the SF-36 health survey: factor structure in Maori, Pacific and New Zealand European ethnic groups. Soc Sci Med 2000; 51: 16551664.
  • 26
    Glance LG, Dick A, Osler TM, Mukamel D. Judging trauma center quality: does it depend on the choice of outcomes? J Trauma 2004; 56: 165172.
  • 27
    Helling TS, Davit F, Edwards K. First echelon hospital care before trauma center transfer in a rural trauma system: does it affect outcome? J Trauma 2010; 69: 13621366.
  • 28
    Ingraham AM, Cohen ME, Raval MV, Ko CY, Nathens AB. Effect of trauma center status on 30-day outcomes after emergency general surgery. J Am Coll Surg 2011; 212: 277286.
  • 29
    Holbrook TL, Anderson JP, Sieber WJ, Browner D, Hoyt DB. Outcome after major trauma: discharge and 6-month follow-up results from the Trauma Recovery Project. J Trauma 1998; 45: 315323.
  • 30
    Holbrook TL, Anderson JP, Sieber WJ, Browner D, Hoyt DB. Outcome after major trauma: 12-month and 18-month follow-up results from the Trauma Recovery Project. J Trauma 1999; 46: 765771.
  • 31
    Baldry Currens JA, Coats TJ. The timing of disability measurements following injury. Injury 2000; 31: 9398.
  • 32
    Davidson GH, Hamlat CA, Rivara FP, Koepsell TD, Jurkovich GJ, Arbabi S. Long-term survival of adult trauma patients. JAMA 2011; 305: 10011007.
  • 33
    Access Economics. The Economic Cost of Spinal Cord Injury and Traumatic Brain Injury in Australia. Access Economics: Canberra, 2009.
  • 34
    Polinder S, Haagsma JA, Belt E, Lyons RA, Erasmus V, Lund J et al. A systematic review of studies measuring health-related quality of life of general injury populations. BMC Public Health 2010; 10: 783.
  • 35
    Mackenzie EJ, Rivara FP, Jurkovich GJ, Nathens AB, Egleston BL, Salkever DS et al. The impact of trauma-center care on functional outcomes following major lower-limb trauma. J Bone Joint Surg Am 2008; 90: 101109.
  • 36
    Flint L, Meredith J, Schwab WC (eds). Trauma: Contemporary Principles and Theory. Wolters Kluwer Health: Philadelphia, 2008.
  • 37
    Gabbe BJ, Sutherland AM, Hart MJ, Cameron PA. Population-based capture of long-term functional and quality of life outcomes after major trauma: the experiences of the Victorian State Trauma Registry. J Trauma 2010; 69: 532536.
  • 38
    Victorian Department of Health. Victorian State Trauma Registry 2008–09 Summary Report. http://www.health.vic.gov.au/trauma/publications/0809vstorm-sumrep.pdf [accessed 26 May 2011].
  • 39
    World Health Organization. Guidelines for Trauma Quality Improvement Programmes. http://www.who.int/violence_injury_prevention/services/traumacare/traumaguidelines/en/index.html [accessed 26 May 2011].
  • 40
    Chadbunchachai W, Saranrittichai S, Sriwiwat S, Chumsri J, Kulleab S, Jaikwang P. Study on performance following Key Performance Indicators for trauma care: Khon Kaen Hospital 2000. J Med Assoc Thai 2003; 86: 17.
  • 41
    Chadbunchachai W, Sriwiwat S, Kulleab S, Saranrittichai S, Chumsri J, Jaikwang P. The comparative study for quality of trauma treatment before and after the revision of trauma audit filter, Khon Kaen hospital 1998. J Med Assoc Thai 2001; 84: 782790.