• Open Access

The reliability of information on work-related injuries available from hospitalisation data in Australia

Authors


Correspondence to:
Kirsten McKenzie, National Centre for Classification in Health, School of Public Health, Queensland University of Technology, Victoria Park Rd, Kelvin Grove, QLD 4059. Fax: (07) 3138 5515; e-mail: k.mckenzie@qut.edu.au

Abstract

Objective: To examine the reliability of work-related activity coding for injury-related hospitalisations in Australia.

Method: A random sample of 4,373 injury-related hospital separations from 1 July 2002 to 30 June 2004 were obtained from a stratified random sample of 50 hospitals across four states in Australia. From this sample, cases were identified as work-related if they contained an ICD-10-AM work-related activity code (U73) allocated by either: (i) the original coder; (ii) an independent auditor, blinded to the original code; or (iii) a research assistant, blinded to both the original and auditor codes, who reviewed narrative text extracted from the medical record. The concordance of activity coding and number of cases identified as work-related using each method were compared.

Results: Of the 4,373 cases sampled, 318 cases were identified as being work-related using any of the three methods for identification. The original coder identified 217 and the auditor identified 266 work-related cases (68.2% and 83.6% of the total cases identified, respectively). Around 10% of cases were only identified through the text description review. The original coder and auditor agreed on the assignment of work-relatedness for 68.9% of cases.

Conclusions and implications: The best estimates of the frequency of hospital admissions for occupational injury underestimate the burden by around 32%. This is a substantial underestimate that has major implications for public policy, and highlights the need for further work on improving the quality and completeness of routine, administrative data sources for a more complete identification of work-related injuries.

Work-related injury is a global problem of substantial public health importance. Estimates suggest that each year approximately 250 million people worldwide suffer a work-related injury, and of these more than 300,000 people die.1 Australian data confirm the substantial burden attributable to work-related injuries in this country where these injuries are estimated to cost around $15 billion annually.2 However, there remains some concern that the quality of data available for ascertaining the nature and extent of the problem is not sufficient to support an adequate public health response.3 In particular, workers compensation databases in Australia underestimate the number of work-related cases by up to 65%.1,3–5 Hospitalisation data is potentially one of the key alternative sources of information for the enumeration of serious work-related injury, and may improve injury estimates of non-fatal work place injuries.

Injury causation information is collected routinely in hospital records in Australia, and cause of injury information from the medical record is translated by clinical coders into coded form using the International Statistical Classification of Diseases and Related Health Problems, 10th Revision, Australian Modification (ICD-10-AM).6 Causal information is coded regarding the intent, mechanism and object (compiled into one external cause code), the place of occurrence, and the activity at the time of the injury.

Several methods are used by researchers to identify work-related injury cases from hospitalisation data which can result in different case ascertainment.5 The usual method to identify work-related injury cases in hospitalisation data in Australia, is to select cases with an ICD-10-AM activity code of U73 (i.e. while working for income) and, where available, a payment status of ‘compensable’ under workers compensation, combined with a principal diagnosis of an injury (S00-T98) and/or an external cause code (V01-Y98). In addition, the use of supplementary codes of ‘Y96 Work-related condition’, ‘Z04.2 Examination and observation following work accident’ and ‘Z57 Occupational exposure to risk factors’ have been used to identify work-related cases in hospitalisation data, though these additional codes have only been found to identify very small numbers of cases.2,4

Activity codes are the only data field for identifying work-related injury hospitalisations that are available at a national level and they have been found to identify more work-related cases than identified using payment status, both nationally1 and internationally.7 For hospital admissions in New South Wales (NSW), the activity code alone identified 77% of work-related admissions (14,776 unique cases), a payment status of compensable alone identified two-thirds of work-related admissions in NSW (10,937 unique cases), while 46% had both an activity and payment status to indicate work-relatedness.1 Despite identifying more cases than payment status, activity codes have been found to under-enumerate work-related cases by up to 23%.1,8

Despite the fact that activity codes have been the most widely used method in Australia for the identification of work-related injuries in hospital data, there has been limited work which has examined inter-rater agreement in the coding of activity in hospital records, with only one Australian and one New Zealand study conducted using ICD-10-AM coded data.9,10 Previously, a sample of 4,373 randomly selected injury-related hospitalisations across 50 hospitals in Australia reported inter-rater agreement for activity coding (using ICD-10-AM 3rd edition) of 68% overall and 65% for the specific working for an income code.10 Similarly, in New Zealand, a sample of 1,750 randomly selected injury-related hospitalisations across 48 hospitals reported inter-rater agreement for activity coding (using ICD-10-AM 2nd edition) of 70.9% overall.9

Hospitalisation data for work-related injuries has been largely examined through secondary data analysis of state or national databases with limited review of medical records to ascertain the amount of detail available to describe the circumstances surrounding the injury event, injury risk factors, use of preventive/safety devices, or occupation details. Several authors have argued for the need for injury narratives to be included as part of standard national datasets to improve the amount of detail about the circumstances surrounding injuries, with New Zealand one of the only countries worldwide to include a free-text injury description field as part of the national dataset.11–15 Jones and Lyons demonstrated that use of routine narrative fields from emergency department surveillance systems improved the specificity of coded cause of injury information and reduced the number of unspecified codes from 67.5% to 49.5% across external cause categories.11 Lincoln et al. discussed the use of narrative descriptions to facilitate the creation of hazard scenarios for work-related injuries and found that while coded data supplied information on the activity, tasks, injury nature and outcomes, narrative descriptions provided considerably more detail regarding the contributory factors and underlying mechanisms to the injury event.12

Currently, little is known about the reliability of hospitalisation data for coding work-related injuries, particularly in the Australian context. The aim of the study is to examine the reliability of work-related activity coding using injury-related hospitalisation data in Australia.

Methodology

A retrospective on-site medical record review and recoding methodology was used to examine the quality and comprehensiveness of cause of injury information for injury-related hospital admissions. This was a sub-study to a larger study that examined the quality of external cause coding and comprehensiveness of injury information for hospitalisation data in Australia. A subset of work-related cases was extracted from the larger study sample for further analysis. Ethical approval to undertake this study was obtained from all State health department central ethics committees as well as all hospital/district ethics committees.

Study sample

Sample selection followed a two stage procedure designed to ensure a dispersion of cases across locality and hospital case loads, while approximating a probability based result. The sampling process has been described in detail elsewhere.10 This study included a stratified random sample of 50 hospitals across four States in Australia. All public hospitals within the four States were stratified by locality (urban, regional, remote) and injury caseload (large >2,500 injury cases/year, medium 1,000-2,499 injury cases/year, small 200-1,000 injury cases/year, or very small <200 injury cases/year). A random sample of cases were obtained from the selected hospitals within each state from all acute care admissions with a principal diagnosis of an injury-related diagnosis (S00-T79) and at least one assigned external cause code (V01-Y98) between 1 July 2002 and 30 June 2004. Cases were included in the work-related sub-study sample if they contained either a) an ICD-10-AM work-related activity code (U73) by the original coder, b) an ICD-10-AM work-related activity code (U73) by the auditor, or c) were identified as work-related through the text description review.

Data collection process

Full medical records for sampled cases were extracted by the local health record department representative. The auditor (i.e. a qualified health information manager) attended each hospital site and firstly recorded text descriptions of any information available in each record regarding the intent, causal mechanism, objects involved, place of occurrence, activity at the time of the injury, alcohol or drug involvement, perpetrators of assaults, preventive or injury risk factors, and the occupation of the injured person. Information was extracted from all medical record documentation sources including ambulance reports, emergency department notes, clinical notes, progress notes, discharge summaries, specialist reports, and allied health reports. The detailed text descriptions of the injury event and circumstances were recorded for each documentation source identifying the source to which the text description belonged. After completing the text descriptions, the auditor coded the record using ICD-10-AM 3rd Edition (the edition that was in use for the sample years of the study) blinded to the original codes and recorded the new codes on the front sheet of a data collection form.

Narrative coding

The text descriptions were reviewed by an independent research assistant (blinded to the coded activity status) and were coded as being either work-related or not work-related. Text descriptions were coded as indicative of a work-related injury if the description stated that:

  • a) The injury was sustained while ‘at work’;
  • b) The injury was sustained while ‘travelling to or from work’ (Note: The coverage of journeys to and from work under workers' compensation schemes is variable across states, with two of the included States covering journeys and two of the included States not covering journeys.16 The authors chose to use a broad inclusive approach in this study to include these cases under the definition of work-related);
  • c) The case was stated to be a workers' compensation case;
  • d) The patient was performing a task related to a relevant stated occupation; or
  • e) The patient was ‘working’ and the location of a worksite was indicated (i.e. industrial site, construction site).

Presence of work-related information was flagged separately for each documentation source in the medical records.

All selected cases were independently reviewed by the first author (KM) to ensure appropriate identification as work-related. Of the 347 cases selected by the research assistant, 286 were accepted as work-related, with 61 cases rejected. The rejected cases were cases where the description:

  • a) Stated an occupation but the task being performed was not clearly related to the patient's occupation and there was no evidence that the person was at work at the time of the injury (e.g. school teacher injured using circular saw at home) (n=30);
  • b) Stated that the patient was ‘working’ on a task but also stated that they were ‘retired’, ‘pensioner’, or ‘unemployed’ (n=12);
  • c) Indicated working on a task but there was no mention of the work being paid work and/or it was not related to the person's occupation (e.g. working in shed) (n=11);
  • d) Mentioned work (or lack of work) as precipitating stressor (n=3);
  • e) Included a statement that the case was not a workers' compensation case (n=4);
  • f) Indicated that the injury was sustained before or after work with no mention of the person travelling to or from work at the time (e.g. suicide attempt and did not attend work) (n=1).

In addition to the manual review of all cases, all text descriptions that contained the word ‘work’ were flagged for review to ensure the capture of all cases. All flagged cases were reviewed using the same criteria as applied during the complete manual review. This search identified 383 cases which contained the word ‘work’, of which 127 were rejected as not work-related. The reasons for rejection of these cases were as follows:

  • a) A phrase/word containing ‘work’ was documented but not in the context of a person working (e.g. workbench, social work, appliance not working) (n=61);
  • b) Work as a task was mentioned but there was no mention of the work being paid work and/or it was not related to the person's occupation (e.g. working in the garden) (n=26);
  • c) Work (or lack of work) was mentioned as precipitating stressor (n=17);
  • d) An occupation was stated but the task being performed was not clearly related to the patient's occupation (n=11);
  • e) The injury was documented as being sustained before or after work with no mention of the person travelling to or from work at the time (e.g. went to bar after work) (n=8);
  • f) A statement was made that the case was not a workers' compensation case (n=4).

Analysis

Descriptive analyses were conducted to identify the number of cases identified by the original coder, auditor and text description review, the level of agreement of activity coding between the original coder and the auditor, the characteristics of cases (e.g. patient demographics, external cause, place of occurrence) identified as work-related for each identification method, and the main documentation sources for work-related information.

Results

Case identification

Of the 4,373 cases sampled in the larger study, 318 cases were identified as being work-related using any of the three methods for selection: original coded data, auditor coded data, text description review. The original coder identified 217 work-related cases (68.2% of the total cases identified), the auditor identified 266 cases (83.6% of the total cases identified) and around 10% of cases (n=32) were identified only through the text description review. There were an additional 32 cases where the text description was coded as not work-related but the activity was assigned by the original coder or auditor as being work-related (See Figure 1).

Figure 1.

Identification of cases of work-related injuries (n=318).

Activity code agreement

The original coder and auditor agreed on the assignment of a work-related activity for 197 of the 286 cases identified through coded activity data (68.9% agreement). Table 1 shows the comparison of activity categories of the original coder and auditor for work-related cases. The majority of cases that were recoded as work-related by the auditor were those assigned unspecified, other specified or other types of work activity codes by the original coder.

Table 1.  Comparison of activity categories of original coder and auditor for work-related cases.
Auditor assigned activity codes
 Sports and LeisureWorking for incomeOther Types of WorkVital activitiesOther specifiedUnspecifiedTotal
Original activity codesn%N%n%n%n%n%N
Sports and Leisure240.000.000.000.000.000.02
Working for an Income240.019775.2114.31100330.01348.1217
Other Types of Work00.0103.8457.100.0110.013.716
Vital activities00.010.4114.300.000.000.02
Other specified120.0176.500.000.0440.0622.228
Unspecified00.03714.1114.300.0220.0725.947
Total (missing n=6)5100262100710011001010027100312

Table 2 shows the agreement of coders regarding the industry the patient was working in for the cases identified by the auditor as work-related. The main industry categories where the auditor was more likely to identify the industry and/or identify the case as work-related were: agriculture/forestry/fishing (additional 11 cases identified by auditor), manufacturing (additional 11 cases identified by auditor), and transport/storage (additional seven cases identified by auditor).

Table 2.  Comparison of specific work codes for cases assigned as work-related by auditor.
Auditor codes
Original codesAgriculture, forestry, fishMiningM'facturingConstruct'nWholesale/retail tradeTransport and storageGovern admin/DefenceHealth servicesOther spec'dUnspec'd workTotal
Agriculture, forestry, and fishing1300000001014
Mining01000000001
Manufacturing0018010001222
Construction0042000000327
Wholesale and retail trade00006000309
Transport and storage000009002112
Government admin/Defence00000120104
Health services00000004004
Other specified work0052011125944
Unspecified work3044120034360
Other Types of Work101101000610
Vital activities00000000000
Other specified activity600101001817
Unspecified activity20110400111837
Total (missing n=5)2513329819354890261

Characteristics of cases identified using different methods

The number and proportion of cases identified by the original coder, the auditor, and the text description review only were examined for different patient and hospital characteristics (see Table 3). Those aged 15 to 44 years and males had the highest proportion of cases identified as work-related by the original coder. The auditor alone identified a higher proportion of older and female workers. Older age groups (60+), females, and cases in regional hospitals had higher proportions of work-related cases identified using the text description compared to the other groups.

Table 3.  Patient and hospital characteristics for each selection method.
 Identified by original coderIdentified by auditor onlyIdentified by text description onlyTotal
 n%n%n%n
Age
15-298770.22621.0118.9124
30-447770.02320.9109.1110
45-594064.51625.869.762
60-741266.7211.1422.218
75+125.0250.0125.04
Sex
Male19569.95820.8269.3279
Female2256.41128.2615.439
Hospital locality
Urban16169.15423.2187.7233
Regional5665.91517.61416.585
Hospital size
Large17767.66022.9259.5262
Medium3769.8917.0713.253
Small3100.00.00.03
Total21768.26921.73210.1318

The number and proportion of cases identified by the original coder, the auditor, and the text description review only were also examined across different external cause characteristics (see Table 4). Different patterns of identification of work-relatedness were evident across external cause categories with machinery-, cutting/piercing- and electricity-related causes the most likely to be identified as work-related by the original coder. Animal-related, poisoning and transport-related causes were the least likely to be identified by the original coder as work-related. Those cases occurring in industrial and construction sites, or trade and service areas were the most likely to be identified by both the original coder and auditor as work-related, while injuries occurring at home were the most likely to be identified by the text description review as work-related compared to other places of occurrence.

Table 4.  Cause of injury characteristics for each selection method.
 Identified by original coderIdentified by auditor onlyIdentified by text description onlyTotal
 n%n%n%n
Intent
  Unintentional21069.86320.9289.3301
  Intentional self harm0.01100.00.01
  Assault654.5218.2327.311
  Undetermined intent1100.00.00.01
External cause
  Transport events2046.52046.537.043
  Falls4868.61217.11014.370
  Smoke, fire, flames0.0150.0150.02
  Hot object or substance466.7233.30.06
  Poisoning444.4333.3222.29
  Cutting, piercing object2880.0514.325.735
  Animal related433.3433.3433.312
  Machinery in operation5484.4812.523.164
  Electricity375.0125.00.04
  Collision with person660.0220.0220.010
  Collision with object2066.7516.7516.730
  Other external cause2678.8618.213.033
Place
  Home525.0840.0735.020
  School866.7216.7216.712
  Sports area150.0150.00.02
  Street and Highway1337.11851.4411.435
  Trade and Service area3093.813.113.132
  Industrial and Construction7095.911.422.773
  Farm1071.4428.60.014
  Other Specified2180.8311.527.726
  Unspecified5956.73129.81413.5104
Total8928.019761.33210.1318

The text description review identified whether there was any documentation to indicate if the injury was work-related, as well as which of the five main documents (i.e. ambulance report form, emergency department record, clinical progress notes, discharge summary, other documents) in the medical records contained this information. Table 5 shows the number of records containing narrative of any sort in each of the documentation sources, and the number and proportion of these cases that contained information to identify the case as work-related. Discharge summaries and ambulance report forms were the least likely sources to identify whether the injury was work-related, while other documents were the most likely source to identify whether the injury was work-related.

Table 5.  Documentation sources for work-related information.
Documentation sourceNumber of cases containing any narrativeNumber of cases containing work-related narrativeCases with work-related narrative/any narrative
  1. Note: a) Includes: allied health notes, specialist reports and administration notes.

 nn%
Any source31828690.0
Ambulance report form1214638.0
Emergency department record28915051.9
Clinical progress notes1919851.3
Discharge summary2074421.3
Other documentsa18211663.7

Discussion

Hospital admission records have been used previously to estimate the number of occupational injuries represented in the healthcare system,1,5 yet an examination of the accuracy of the identification of these cases has been lacking. This study is the first to examine the reliability of the identification of work-related injury hospital admissions in Australia by comparing original activity codes, against both an auditor's assessment, and a review of narrative text in medical records. This investigation has revealed that hospital records alone (i.e. the original coder) are under-enumerating work-related admissions by 32%. Independent auditing was able to identify an additional 21.6% of cases, and the text description review identified the remaining 10.1% of cases. Inter-rater agreement between the original coder and auditor for the assignment of activity codes was almost 69%.

The current underestimate of work-related hospital admissions is similar to New Zealand, where Davie et al.9 reported a 29% underestimation in work-related admissions, and is slightly more than the 23% underestimation identified by Mitchell et al. using NSW data.1 This difference is likely to be due to the different methodologies used between the two studies, with the previous study using a comparison of activity codes and payment status to identify the under-enumeration of activity codes using all NSW data, while this study used a recoding methodology and text description analysis based on a sample of medical records from four states in Australia.

Different patterns were evident for patient/hospital/cause of injury characteristics for cases identified by the original coder, the auditor, or by the text description alone. These distinctions could be due to the ease of identifying certain types of admissions as work-related. For example, injuries that resulted from actions, objects and machinery that occur in locations that are known to be associated with work appear to be more likely to be identified as such. For example, cases most likely to be assigned a ‘working for income’ code by both the original coder and auditor were males between the ages of 15-29, with external cause categories indicating machinery and electricity-related causes, and those occurring in industrial and construction sites, or trade and service areas. On the other hand, cases where there was less certainty around the assignment of a working activity code (i.e. where only one coder or neither coder identified the case as work-related), were those where the relationship to work might not be as easily recognisable, such as hospitalisations where the external cause was animal-related, poisoning or transport-related causes, and those occurring at home.

Identification of work-related injuries varied by documentation sources in the medical record with discharge summaries and ambulance report forms the least likely sources to identify whether the injury was work-related, while other allied health, specialist reports and administrate notes were the most likely sources to identify whether the injury was work-related. Time available to record working status may be an issue. For example, ambulance paramedics are likely to be intent on immediate treatment and stabilisation of a trauma patient for transport, while allied health and specialists tend to be more involved in later rehabilitation and injury management, where a more detailed description of the injurious incident is likely to be able to be obtained.

There are some caveats to this study that need to be considered. It is possible that the auditor may have assigned a ‘working for income’ code in error. However, the potential for error appears minimal given that there were only 20 cases (7%) out of the 266 auditor-assigned work-related cases that were not also coded as work-related injuries using a ‘blinded’ text review process. In addition, the broad approach to including cases as being work-related if they were travelling to or from work at the time of the injury (given the variability across States in the definition of this situation16 may have produced inconsistencies in the coding of activity between the original coder and the auditor if the original coder did not consider such cases to be work-related. There may have also been cases where the auditor missed narrative text pertaining to a work-related injury and correspondingly did not assign a work-related activity code, hence leading to underestimation of work-related cases. While the extent of errors of omission are difficult to ascertain, there were few additional cases identified by the original coder that were not also identified by the auditor, with only 20 cases uniquely identified as work-related by the original coder (with eight of the text descriptions including text regarding the work-related nature of the injury and 12 cases with no description indicating that the case was work-related). Furthermore, while this study has examined the concordance of estimates of work-related injuries using documentation from medical records, these estimations rely in the first instance on accurate clinical documentation of the work-related nature of a patient's injury. We were unable to ascertain the extent to which the medical record accurately documents the nature of activity at the time of the injury using the current methodology.

On a positive note, the auditor agreed with the original coders assignment of cases as work-related for 91% of cases (197 cases out of 217 assigned by the original coder). Thus, the use of the ‘working for income’ code to identify true cases of work-related injuries appears to be relatively accurate. However, with 32% of cases missed by using the activity code of ‘working for income’, additional data fields need to be used in conjunction with this variable for a more complete capture of work-related hospitalised injuries. As the auditor was particularly focusing on identifying detailed information regarding the activity of the injured person, they may have been more alert to detecting documentation regarding work-relatedness than was the original coder. This is especially pertinent for particular patients (e.g. older individuals and females) and types of injuries (e.g. transport-related). Furthermore, differences in the assignment of activity for transport-related injury events (particularly where the journey was to or from work) is subject to variation given the inconsistencies in inclusion of these events under workers' compensation schemes across States.16 The ‘other specified activity’ and ‘unspecified activity’ codes only appear to include a small number of work-related cases with only 2% of cases assigned to these codes being identified as work-related (75 cases assigned by the original coder as other or unspecified activity were identified in the final dataset as work-related out of 3,098 other or unspecified cases not identified as work-related).

Implications

The extent of underestimation of work-related injuries using activity coding identified in the sample of cases examined in the study can provide an indication as to the possible extent of work-related injuries in hospital data. In addition, this study offers some insights into possible improvements in the coding of activity and the capture of work-related information in routine hospital data. The higher case capture of work-related injuries by the auditor and narrative text review suggests that more work-related cases could be identified through routine clinical coding. Currently, there are no Australian coding standards to inform clinical coders on how to assign activity codes or provide guidance on the criteria to use to assign a case as work-related. Coders could be provided with instructions on the identification of work-related activities, such as those used in this study for the text review process, which should improve the completeness of work-related injury case capture. Further work is needed to explore the potential of introducing an injury description text field to routine hospital data (such as that used in New Zealand admitted patient data and Australian emergency department data), given the fact that 10% of unique work-related cases were identified using the text extract.

In summary, this study found that the best estimates of the frequency of hospital admissions from occupational injury underestimate the burden by around 32%. This is a substantial underestimate that has major implications for public policy, and highlights the need for further work on improving the quality and completeness of routine data sources, such as hospitalisation records, for a more complete identification of work-related injuries.

Acknowledgements

This research is funded by an Australian Research Council Linkage Project grant with industry partners being Injury Prevention and Control Australia, the Victorian Department of Human Services, Queensland Health Information Centre, and the South Australian Department of Human Services. R. Mitchell is supported by the NSW Injury Risk Management Research Centre, with core funding provided by the NSW Health Department, the NSW Roads and Traffic Authority and the Motor Accidents Authority. We wish to thank the three auditors who collected data for this study, Mr Garry Waller, Ms Jude Michel, and Ms Robyn Herrewyn.

Ancillary