Intervention Review

You have free access to this content

Occupational safety and health enforcement tools for preventing occupational diseases and injuries

  1. Christina Mischke1,*,
  2. Jos H Verbeek1,
  3. Jenny Job2,
  4. Thais C Morata3,
  5. Anne Alvesalo-Kuusi4,
  6. Kaisa Neuvonen5,
  7. Simon Clarke6,
  8. Robert I Pedlow7

Editorial Group: Cochrane Work Group

Published Online: 30 AUG 2013

Assessed as up-to-date: 20 MAR 2013

DOI: 10.1002/14651858.CD010183.pub2


How to Cite

Mischke C, Verbeek JH, Job J, Morata TC, Alvesalo-Kuusi A, Neuvonen K, Clarke S, Pedlow RI. Occupational safety and health enforcement tools for preventing occupational diseases and injuries. Cochrane Database of Systematic Reviews 2013, Issue 8. Art. No.: CD010183. DOI: 10.1002/14651858.CD010183.pub2.

Author Information

  1. 1

    Finnish Institute of Occupational Health, Cochrane Occupational Safety and Health Review Group, Kuopio, Finland

  2. 2

    Safe Work Australia, Strategic Policy Branch, Canberra, ACT, Australia

  3. 3

    National Institute for Occupational Safety and Health (NIOSH), Cincinnati, OH, USA

  4. 4

    University of Turku, Faculty of Law, Turku, Finland

  5. 5

    Finnish Institute of Occupational Health, Cochrane Occupational Safety and Health Review Group, Helsinki, Finland

  6. 6

    UK Health and Safety Executive, Merseyside, UK

  7. 7

    Safe Work Australia, Research and Evaluation Team, Canberra, ACT, Australia

*Christina Mischke, Cochrane Occupational Safety and Health Review Group, Finnish Institute of Occupational Health, Neulaniementie 4, PO Box 310, Kuopio, 70101, Finland. christina.mischke@ttl.fi. tinamischke@gmx.de.

Publication History

  1. Publication Status: New
  2. Published Online: 30 AUG 2013

SEARCH

 
Characteristics of included studies [ordered by study ID]
Burstyn 2010

MethodsPanel study


ParticipantsCanada, Alberta

Firms, manufacturing industry

N = ??


InterventionsIntervention group: inspection with or without penalty (proactive inspector)

  • Inspection: OSHA
  • Proactive inspector


Control group: inspection with or without penalty (reactive inspector)

  • Inspection: OSHA
  • Reactive inspector


OutcomesPrimary outcome, exposure:

As # of compliance orders needed to resolve non-compliance


NotesTime of the intervention: 2002-2006

Firm size: not reported

Type of work: not reported

Previous inspections: not reported

Baseline injury rates: not reported

Funding: not reported

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskOutcome compliance, participants not blinded

Blinding (outcome assessors)High riskOutcome assessors not blinded

Retrospective unplanned subgroup analysesHigh riskNo prespecified model

Follow-upHigh riskNo adjustment

Statistical testsHigh riskPoisson regression model, survival analysis missing

ComplianceUnclear riskNot reported

Outcome measuresUnclear riskOutcome was time to compliance, unclear

Selection bias (population)Unclear riskType of industry not reported

Selection bias (time)Low riskRecruited over the same time period

RandomisationHigh riskNot randomised

Allocation concealmentHigh riskNot concealed

Adjustment for confoundingUnclear riskType of work not reported, pre-intervention inspection not assessed, unclear adjustment for baseline injury rates

Incomplete outcome dataUnclear riskNot reported

Chen 2008

MethodsCBA; we used presented data to perform an ITS analysis


ParticipantsUSA

Firms, trucking industry

N = 113,441 between 1999-2001

Firm size: number of employees

Type of work: motor carrier driver

Previous inspections: unknown

Baseline injury rates: median over years 2.99 crashes per 100 trucks


InterventionsIntervention group: inspection, warnings and orders (N = 3705)

  • Inspection (compliance review (CR)): Safety ratings to determine whether a motor carrier meets the Section 385.5 Safety Fitness standards (FMCSA 2006), (1) satisfactory, (2) conditional satisfactory or (3) unsatisfactory, a follow-up review may be conducted to ensure that all necessary corrective actions have been taken
  • Warnings or orders: carriers receiving a conditional satisfactory or unsatisfactory rating are required to undertake corrective actions within 30 days or the carrier’s operating authority will be revoked and an operating out of service will be imposed prohibiting the carrier from operating any motor vehicle in the USA


Control group: no compliance review (N = 109,736)

(Co-interventions not reported)


OutcomesPrimary outcome, exposure:

Number of crashes, involving a truck or a bus of motor carriers operating in the United States with at least 1 fatality, injury or vehicle towed away from the scene as a result of disabling crash damage


NotesTime of the intervention: 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003; 1999-2001 used as years with interruption for ITS analysis

Funding: not reported

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)Low riskNo blinding of participants, but knowing about receiving the inspection does not change the number of reported accidents

Blinding (outcome assessors)Low riskMeasurements used from crash file "contains data from State Police crash reports"

Retrospective unplanned subgroup analysesLow riskNo data dredging, objectives of this study: "whether the reduction occurred in every sub-group of reviewed trucking companies"

Follow-upLow riskSimilar follow-up for cases and controls

Statistical testsLow riskAppropriate

ComplianceUnclear riskNot reported

Outcome measuresUnclear risk"some states did not report all eligible crashes"

Selection bias (population)Unclear riskOutcome on company level, but one carrier could be employed in more than one company

Selection bias (time)Low riskSame time period

RandomisationHigh riskNo randomisation

Allocation concealmentHigh riskNot randomised

Adjustment for confoundingUnclear riskFirm size, previous inspection for control and intervention group unknown

Incomplete outcome dataLow riskNo loss

Foley 2012

MethodsPanel study


ParticipantsUSA, Washington

Firms, mixed types of industry

N = 8752


InterventionsIntervention group: inspections: (N = 440)

  • Programmed or complaint inspection


Control group: no intervention (N = 8312)


OutcomesPrimary outcome, injuries:

Number of registered lost-workday claims per 100 FTEs, the change in the claim rate 1 year after intervention


NotesTime of the intervention: 1999

Firm size: number of employees not reported

Type of work: mixed

Previous inspections: not in 1 year prior intervention

Baseline injury rates: only the account's SFY 1997 compensable claims rate were controlled for in the analysis (intervention and control group 2 and 3 had consistently higher average SFY 1997 claims rates than those accounts with no activity)

Funding: Washington State Department of Labor and Industries

Conflict of interest: authors are employed by the Funder (Washington State Department of Labor and Industries)


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskBased on self reported outcome

Blinding (outcome assessors)Low riskRegistered compensable cases, based on self reporting

Retrospective unplanned subgroup analysesUnclear riskPredefined model not presented

Follow-upLow riskSame time period

Statistical testsLow riskUnivariate and multivariate analyses were undertaken

ComplianceUnclear riskNot reported

Outcome measuresHigh riskBased on self reporting

Selection bias (population)Unclear riskNot reported

Selection bias (time)Low riskSame time period

RandomisationHigh riskNo randomisation

Allocation concealmentHigh riskNot concealed

Adjustment for confoundingLow risk3 out of 4, adjusted for average size and baseline claim-injury rates, no intervention 2 years prior evaluation, type of work unclear

Incomplete outcome dataUnclear riskNot reported

Gray 2005a

MethodsPanel study


ParticipantsUSA

Firms, manufacturing industry

N = not reported


InterventionsIntervention group: inspection with or without penalty

  • Inspection: OSHA (programmed inspections and inspections after complaint)


Control group: no inspection


OutcomesPrimary outcome, injuries:

Total number of lost work days during the year/per firm


NotesTime of the intervention: 1979-1985 (7 years)

Firm size: not reported

Type of work: mostly physical

Previous inspections: not reported

Baseline injury rates: not reported

Funding: not reported

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskNot blinded, lost time injuries self reported

Blinding (outcome assessors)High riskData from data base but relying on self reports

Retrospective unplanned subgroup analysesLow riskPrespecified model

Follow-upLow riskSimilar follow-up

Statistical testsLow riskt-test and maximum likelihood estimates

ComplianceUnclear riskNot reported

Outcome measuresHigh riskRelying on self reports

Selection bias (population)Unclear riskNot reported if firms from different states (USA)

Selection bias (time)Low riskSame time period

RandomisationHigh riskNot randomised

Allocation concealmentHigh riskNot concealed

Adjustment for confoundingHigh riskNot adjusted for different size, injury rates, previous interventions, type of work similar

Incomplete outcome dataUnclear riskNot reported

Gray 2005b

MethodsPanel study


ParticipantsUSA

Firms, manufacturing industry

N = not reported


InterventionsIntervention group: inspection with or without penalty

  • Inspection: OSHA (programmed inspections and inspections after complaint)


Control group: no inspection


OutcomesPrimary outcome, injuries:

Total number of lost work days during the year/per firm


NotesTime of the intervention: 1987-1991 (4 years)

Firm size: not reported

Type of work: mostly physical

Previous inspections: not reported

Baseline injury rates: not reported

Funding: not reported

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskNot blinded, lost time injuries self reported

Blinding (outcome assessors)High riskData from database but relying on self reports

Retrospective unplanned subgroup analysesLow riskPrespecified model

Follow-upLow riskSimilar follow-up

Statistical testsLow riskt-test and maximum likelihood estimates

ComplianceUnclear riskNot reported

Outcome measuresHigh riskRelying on self reports

Selection bias (population)Unclear riskNot reported if firms from different states (USA)

Selection bias (time)Low riskSame time period

RandomisationHigh riskNot randomised

Allocation concealmentHigh riskNot concealed

Adjustment for confoundingHigh riskNot adjusted for different size, injury rates, previous interventions, type of work similar

Incomplete outcome dataUnclear riskNot reported

Gray 2005c

MethodsPanel study


ParticipantsUSA

Firms, manufacturing industry

N = not reported


InterventionsIntervention group: inspection with or without penalty

  • Inspection: OSHA (programmed inspections and inspections after complaint)


Control group: no inspection


OutcomesPrimary outcome, injuries:

Total number of lost work days during the year/per firm


NotesTime of the intervention: 1992-1998 (7 years)

Firm size: not reported

Type of work: mostly physical

Previous inspections: not reported

Baseline injury rates: not reported

Funding: not reported

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskNot blinded, lost time injuries self reported

Blinding (outcome assessors)High riskData from database but relying on self reports

Retrospective unplanned subgroup analysesLow riskPrespecified model

Follow-upLow riskSimilar follow-up

Statistical testsLow riskt-test and maximum likelihood estimates

ComplianceUnclear riskNot reported

Outcome measuresHigh riskRelying on self reports

Selection bias (population)Unclear riskNot reported if firms from different states (USA)

Selection bias (time)Low riskSame time period

RandomisationHigh riskNot randomised

Allocation concealmentHigh riskNot concealed

Adjustment for confoundingHigh riskNot adjusted for different size, injury rates, previous interventions, type of work similar

Incomplete outcome dataUnclear riskNot reported

Haviland 2012

MethodsPanel study, regression analysis


ParticipantsUSA, Pennsylvania

Firms, manufacturing industry

N = 8645


InterventionsIntervention group: programmed inspection with penalty (no detail)

Control group: no intervention or inspection without penalty


OutcomesInjuries: days away from work (DAW), including disease and injury, DAW per 100 person/year


NotesTime of the intervention: 1998-2005 (8 years)

Firm size: only small firms (20 to 250 employees)

Type of work: mostly physical, manufacturing industry

Previous inspections: not reported

Baseline injury rates: not reported

Funding: Commonwealth Pennsylvania

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskNot blinded, outcome self reported (lost time injury data)

Blinding (outcome assessors)High riskWorkers are assessors, data from registry (Worker Compensation data) but based on self report

Retrospective unplanned subgroup analysesHigh riskNo predefined model, firms with 10 to 20 employees excluded

Follow-upLow riskSimilar follow-up

Statistical testsLow riskRegression analysis

ComplianceUnclear riskNot reported

Outcome measuresHigh riskClaims of lost time injuries, self reported

Selection bias (population)Low riskSame population (manufacturing industry and adjustment for SIC, all in Pennsylvania)

Selection bias (time)Low riskSame time

RandomisationHigh riskNot randomised

Allocation concealmentHigh riskNot concealed

Adjustment for confoundingUnclear riskSame type of work and firm size, pre-inspections and baseline injury rate difference not reported/adjusted

Incomplete outcome dataUnclear riskNot reported

Hogg-Johnson 2011

MethodsRCT


ParticipantsCanada

Firms, manufacturing industry

N = 1219

Firm size: not reported, described as similar in intervention and control groups

Type of work: manufacturing industry, mostly physical

Previous inspections: not in 2 years prior study intervention


InterventionsIntervention group: inspection and orders (N = 619)

  • Inspection: once or twice during intervention year, without prior warning
  • Orders: upon inspection, inspectors wrote orders based on non-compliance with legislative and regulatory OSH requirements comprehensive or focused on particular hazard


Control group: no intervention (N = 600)


OutcomesPrimary outcome, injuries:

Annual mean and median claim rates per year as: 1) overall injury claim rate (yearly rates of claims registered per 100 FTE), 2) lost time injury claim rate (yearly rates of claims registered per 100 FTE) and 3) disability day rate (measured as number of calendar days of 100% wage replacement within 2 years of date of accident for all claims filed within the year per 100 FTE)

Adverse outcome, firm closure: mean and median firm closure rates (whether a firm went out of business in a given year)


NotesTime of the intervention: 1 April 2006 to 31 March 2007

Baseline injury rates: exact data not presented, significantly different with higher rates in the intervention than control group 2

Funding: Ontario Workplace Safety & Insurance Board Research Advisory Council, Canadian Institute of Health

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskNot blinded, plausible to bias claim rates (self reported)

Blinding (outcome assessors)Low riskExtracted from official administrative records (Ontario Workplace Safety and Insurance Board)

Retrospective unplanned subgroup analysesLow riskNo data dredging

Follow-upLow riskSimilar follow-up

Statistical testsLow riskGeneralised estimating equation models and Pearson square test, Wilcoxon rank sum test

ComplianceUnclear riskNot reported

Outcome measuresHigh riskSelf reported (claims)

Selection bias (population)Low riskDifferent intervention groups

Selection bias (time)Low riskDifferent intervention groups

RandomisationLow riskConducted using SAS 9.1, afterwards further exclusion but same criteria and similar percentage excluded across study groups

Allocation concealmentHigh riskNot concealed, participants received in addition non-assigned interventions during intervention period (enforcing activities ca. 21%, consulting activities ca. 16%) 

Adjustment for confoundingLow riskFirm size and previous inspections comparable; all firms in manufacturing industry

Incomplete outcome dataLow riskLess than 20%

Kemmlert 1994

MethodsRCT


ParticipantsSweden

Individual workplaces, type of industry not reported

N = 195


InterventionsIntervention group: inspection with or without penalty (N = 98)

  • Inspection: by labour inspectorate, announced inspection, assessment following ergonomic workplace checklist to identify musculoskeletal stress factors, inspector received special training
  • Warning or orders: inspector notices in case of insufficiencies and to express demands


Control group: no intervention (N = 97)


OutcomesPrimary outcome, exposure:

Reduced workload, achieved if the harmful situation reported in the injury report on musculoskeletal injuries did not exist anymore

Adverse outcome, active employment:

Employment status (whether a worker went out of employment in a given workplace after 3 years)


NotesTime of the intervention: 1985

Firm size: not reported

Type of work: not reported

Previous inspections: not reported

Baseline injury rates: occupational musculoskeletal injury report was inclusion criteria for control and intervention group, outcome workplace specific

Funding: no funding

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskSubjects not blinded

Blinding (outcome assessors)Low riskAssessor blinded

Retrospective unplanned subgroup analysesUnclear riskNo protocol available

Follow-upLow riskSimilar follow-up

Statistical testsUnclear riskNot reported

ComplianceUnclear riskNot assessed

Outcome measuresHigh riskOutcome improvement relies on self reported data from baseline

Selection bias (population)Unclear riskNot reported, outcome for individual workplaces

Selection bias (time)Low riskSame time period

RandomisationUnclear riskMethod not described

Allocation concealmentHigh riskNot concealed

Adjustment for confoundingUnclear riskFirm size and previous inspections unknown

Incomplete outcome dataHigh risk> 50% loss of follow-up

Kniesner 2004

MethodsPanel study


ParticipantsUSA

Firms, mining industry

N = 292


InterventionsIntervention: inspection with penalty or inspection with closure order

  • Inspection: MSHA
  • Penalty: fine
  • Closure order: mine closure


Control group: fewer inspections with penalty or fewer inspections with closure order

  • Inspection: MSHA
  • Penalty: fine
  • Closure order: mine closure


OutcomesPrimary outcome, injuries:

Number of lost workday injuries including fatalities per firm per quarter


NotesTime of the intervention: 1983-1997 (15 years)

Firm size: not reported

Type of work: mostly physical

Previous inspections: not reported

Baseline injury rates: not reported

Funding: not reported

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskSelf report

Blinding (outcome assessors)High riskWorker assessed, self reported

Retrospective unplanned subgroup analysesHigh risk+200 models

Follow-upLow riskSame time

Statistical testsLow riskRegression analysis

ComplianceUnclear riskNot reported

Outcome measuresHigh riskSelf reported

Selection bias (population)Unclear riskAdjustment for different location unclear

Selection bias (time)Low riskOver same time

RandomisationHigh riskNo randomisation

Allocation concealmentHigh riskNot concealed

Adjustment for confoundingLow riskAdjustments for firm size and injury rates, same type of work, previous inspections unknown

Incomplete outcome dataUnclear riskNot reported

Levine 2012

MethodsCBA

Comment: intervention subjects randomly chosen, controls matched (when fulfilling inclusion criteria matched according to same industry, same region; the firm with the most similar numbers of employees got chosen)


ParticipantsUSA, California

Firms, mixed industries

N = 818


InterventionsIntervention group: inspection and if indicated further penalties

  • Random inspection by Cal/OSHA inspectors for industries with high injury rates
  • No details about further penalties


Control group: no random inspection


OutcomesPrimary outcome, injuries:

Injury rates

Adverse outcome:

Firm closure (survival), sales, employment, payroll


NotesTime of the intervention:1996-2006

Firm size: mostly small firms, mean 34.28 (36.3) (# 0-570)

Type of work: mostly physical work

Previous inspections: no OSHA inspection in 2 years prior intervention

Baseline injury rates: exact data not presented, pre-trends showed no statistical difference (14% decline in intervention and 12% decline in control group)

Funding: Commission on Health and Safety and Workers’ Compensation; Harvard Business School’s Division of Research and Faculty Development; Kauffman Foundation; University of California at Berkeley’s Institute for Research on Labor and Employment; University of California’s Labor and Employment Relations Fund

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskOutcome self reported to the workers' compensation system

Blinding (outcome assessors)High riskInjury data from the workers' compensation system but self reported

Retrospective unplanned subgroup analysesUnclear riskNo protocol available

Follow-upLow riskSimilar

Statistical testsLow riskRegression analysis

ComplianceUnclear riskNot reported

Outcome measuresHigh riskBased compensation claims

Selection bias (population)Low riskSame population

Selection bias (time)Low riskSame time

RandomisationUnclear riskRandomisation procedure not reported

Allocation concealmentHigh riskInspectors could make a choice

Adjustment for confoundingLow riskControl and intervention group similar at baseline in 3 of 4 confounders (included mostly small firms, mostly physical work, included only if no inspections 2 years prior intervention)

Incomplete outcome dataLow risk6% loss to follow-up

Nelson 1997

MethodsCBA


ParticipantsUSA, Washington

Firms, construction industry

N = 9085 (I = 784, C = 8,301)

Firm size (number of employees): any size, I mean = 22 (1 to 542), C mean = 7 (1 to 404) (number of employees were identified as number of hours worked by employees assuming that each full-time employee works 2000 hours per year (40 hours per week for 50 weeks per year))

Type of work: mostly physical, same type of industry

Previous inspections: not reported

Baseline injury rates: as number of fall claims I = 0.399, C = 0.071; fall injury rate per 200,000 hrs worked I = 1.78 C = 1.04


InterventionsIntervention group: inspection, citations and monetary penalties (N = 784):

  • Visited by state plan safety inspectors, reasons for the inspections included programmed (scheduled) evaluations (83.2%), employee complaints (8.2%), referral (1.8%), inspection of a fatality or accident (0.8%), and other unscheduled evaluations (5.9%)
  • Cited for violating the falls in construction standard
  • Monetary penalties no details


Control group: no intervention (N = 8301)


OutcomesPrimary outcome, injuries:

Fall injury claims with min. 4 days of lost work time because of fall (coded as injury event or exposure codes including fall from elevation, platform or ladder; fall from piled matter; fall on stairs; fall into openings; fall from roof and fall to lower level)


NotesTime of the intervention: 1991-1992, median date of inspection was October 1991

Funding: not reported

Conflict of interest: unknown


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskNot blinded, outcome (compensable fall claims with work time loss) likely to be influenced

Blinding (outcome assessors)Low riskClaims and employment data were obtained from the Washington state department of labour and industries files

Retrospective unplanned subgroup analysesLow riskNo data dredging

Follow-upLow riskSimilar follow-up

Statistical testsLow riskAppropriate

ComplianceUnclear riskNo evidence is reported on whether follow-up inspections were done

Outcome measuresLow riskSelf reported claims from register but unlikely biased (compensable fall claims with work time loss)

Selection bias (population)Low riskSame population

Selection bias (time)Low riskSame time period

RandomisationHigh riskNo randomisation

Allocation concealmentHigh riskNot concealed, allocation to intervention by inspector

Adjustment for confoundingHigh riskInspections prior intervention not assessed, fall injury rate per 200,000 hours worked higher in inspection group than in control group (I = 1.78; C = 1.04 ), not adjusted for different firm size

Incomplete outcome dataLow riskNo loss

Robertson 1983

MethodsPanel study


ParticipantsUSA

Firms, manufacturing industry

N = 3 plants, total 2700 workers


InterventionsIntervention: inspection with penalty by OSHA

  • Any type of inspection
  • Penalties include citation


Control group:

  • Inspection without citation


OutcomesPrimary outcome, injuries:

  • Lost time injuries per firm per year


NotesTime of the intervention: 1973-1980

Firm size: big firms

Type of work: mostly physical work

Previous inspections: not reported

Baseline injury rates: included in analyses as observed/expected injuries, rates not reported

Funding: Yale University by Atlantic Richfield Corporation

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskSelf reported injuries

Blinding (outcome assessors)High riskNot blinded

Retrospective unplanned subgroup analysesLow riskOne analysis

Follow-upLow riskSame follow-up time

Statistical testsLow riskRegression analysis

ComplianceUnclear riskNot reported

Outcome measuresHigh riskSelf reported

Selection bias (population)Low riskFirms from same industry

Selection bias (time)Low riskSame time

RandomisationHigh riskNo randomisation

Allocation concealmentHigh riskNo allocation concealment

Adjustment for confoundingLow riskAdjusted except for pre-intervention inspections

Incomplete outcome dataUnclear riskNot reported

Smith 1979a

MethodsPanel study


ParticipantsUSA

Firms, manufacturing industry

N = unknown


InterventionsIntervention: inspection with or without penalty

Control group: inspection later in that year


OutcomesPrimary outcome, injuries:

  • Lost workday injuries rate (# injuries per 100 full-time workers)


NotesTime of the intervention:between 1972-1973

Firm size: any size, average not reported, separate analysis for firms < 100, 100 to 249 and > 250 workers

Type of work: mostly physical work

Previous inspections: not reported

Baseline injury rates: not reported, included in regression model as injury rate in year of inspection

Funding: Department of Labor

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskNot blinded, outcome self reported

Blinding (outcome assessors)High riskNot blinded

Retrospective unplanned subgroup analysesLow riskOne model

Follow-upLow riskSame time of follow-up

Statistical testsLow riskRegression analysis

ComplianceUnclear riskNot reported

Outcome measuresHigh riskSelf reported

Selection bias (population)Unclear riskAll from manufacturing industry, geographical region unclear

Selection bias (time)Low riskSame time period

RandomisationHigh riskNo randomisation

Allocation concealmentHigh riskNo allocation concealment

Adjustment for confoundingLow risk3 out of 4 (same type of industry, adjusted for injury rates, separate analysis for different firm size, previous inspection unknown)

Incomplete outcome dataUnclear riskNot reported

Smith 1979b

MethodsPanel study


ParticipantsUSA

Firms, manufacturing industry

N = unknown


InterventionsIntervention: inspection with or without penalty

Control group: inspection later in that year


OutcomesPrimary outcome, injuries:

  • Lost workday injuries rate (# injuries per 100 full-time workers)


NotesTime of the intervention:between 1973-1974

Firm size: any size, average not reported, separate analysis for firms < 100, 100 to 249 and > 250 workers

Type of work: mostly physical work

Previous inspections: not reported

Baseline injury rates: not reported, included in regression model as injury rate in year of inspection

Funding: Department of Labor

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskNot blinded, outcome self reported

Blinding (outcome assessors)High riskNot blinded

Retrospective unplanned subgroup analysesLow riskOne model

Follow-upLow riskSame time of follow-up

Statistical testsLow riskRegression analysis

ComplianceUnclear riskNot reported

Outcome measuresHigh riskSelf reported

Selection bias (population)Unclear riskAll from manufacturing industry, geographical region unclear

Selection bias (time)Low riskSame time period

RandomisationHigh riskNo randomisation

Allocation concealmentHigh riskNo allocation concealment

Adjustment for confoundingLow risk3 out of 4 (same type of industry, adjusted for injury rates, separate analysis for different firm size, previous inspection unknown)

Incomplete outcome dataUnclear riskNot reported

Weil 1996

MethodsPanel study


ParticipantsUSA

Firms, custom woodworking industry

N = 250


InterventionsInspection with and without penalties by OSHA

  • Any type of inspection including complaint and follow-up inspections
  • Penalties include fines and citation


Comparisons:

  • More inspections versus fewer inspections
  • Complaint versus any inspection
  • Follow-up versus any inspection
  • Higher inspection intensity versus lower intensity, as in: length of inspection and amount of fine


OutcomesPrimary outcome, exposure:

Violation of safety standard (machine guarding)


NotesTime of the intervention: 1972-1991

Firm size: small, average 52 employees

Type of work: mostly physical work

Previous inspections: not reported

Baseline injury rates: not reported

Funding: National Science Foundation, Boston University

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskWorker not blinded, influence on outcome possible

Blinding (outcome assessors)High riskNot blinded, inspectors measuring outcome themselves (violation of safety standard)

Retrospective unplanned subgroup analysesLow riskOnly one regression model

Follow-upLow riskSame length

Statistical testsLow riskLogistic regression

ComplianceUnclear riskNot reported

Outcome measuresHigh riskSelf reported

Selection bias (population)Low riskSame population

Selection bias (time)Low riskSame time period

RandomisationHigh riskNo randomisation

Allocation concealmentHigh riskNot concealed, not randomised

Adjustment for confoundingHigh riskNot adjusted, differences not measured for pre-intervention inspections and injury rates

Incomplete outcome dataUnclear riskNot reported

Weil 2001

MethodsPanel study


ParticipantsUSA

Firms, construction industry

N = 2060


InterventionsInspection with and without penalties

  • Any type of inspection by OSHA including complaint, accident or fatality investigation


Comparisons:

  • Complaint inspection versus any inspection
  • Accident/fatality investigation versus any inspection
  • More inspections versus fewer inspections
  • Higher inspection intensity versus lower intensity, as in: length of inspection and amount of fine


OutcomesPrimary outcome, exposure:

Violation of safety standard (machine guarding)


NotesTime of the intervention: 1987-1993

Firm size: big firms

Type of work: mostly physical work

Previous inspections: not reported

Baseline injury rates: not reported

Funding: NIOSH

Conflict of interest: no


Risk of bias

BiasAuthors' judgementSupport for judgement

Blinding (subjects)High riskNot blinded

Blinding (outcome assessors)High riskNot blinded, inspectorate measure compliance

Retrospective unplanned subgroup analysesHigh riskMany models

Follow-upLow riskSame follow-up for all

Statistical testsLow riskRegression analysis

ComplianceUnclear riskNot measured

Outcome measuresUnclear riskMeasured by different inspectorates

Selection bias (population)Low riskSame type of industry

Selection bias (time)Low riskSame time

RandomisationHigh riskNot randomised

Allocation concealmentHigh riskNot concealed

Adjustment for confoundingUnclear riskNot adjusted, not measured for pre-intervention inspections and injury rates

Incomplete outcome dataUnclear riskNot reported

 
Characteristics of excluded studies [ordered by study ID]

StudyReason for exclusion

Adams 2007Not about enforcement

Attfield 1992Not about enforcement, but legislation only

Auld 2001Analysis on sub-industry not firm or workplace level

Baron-Epel 2012Qualitative study, not about enforcement of occupational health and safety but of a smoking ban legislation

Boden 1985Panel data, no time variable included in regression analysis, excluded as cross-sectional study

Brown 2003Opinion paper, single person

Joy 2007Evaluation of stricter regulation, not a study of variation of enforcement tools

Ko 2010Panel study, no time lag variable, excluded as cross-sectional study

Lissner 2011Qualitative study, not about enforcement but legislation only

Mancini 2005Enforcement of only part of the assessed intervention

Morantz 2009Panel study that used the same data as Weil 2001. The analysis included state versus federal inspectors. This is not an intervention that could be easily applied and neither is this a factor that is easily explained. We decided therefore to exclude this study to prevent counting studies twice.

Niskanen 2013Survey, only inspectors' opinions not employers' or employees' opinions

Raymond 2003Not about occupational health and safety

Smitha 2001Missing time lag variable

Viscusi 1979Panel data, outcome measured at aggregate industry level not at firm or individual workplace level

 
Characteristics of studies awaiting assessment [ordered by study ID]
Bordas 2001

MethodsTechnique of analyses: triangulation

Data collection: observations (behaviour, at the workplace) and interviews (formal, informal and structured)

ParticipantsUSA, east central Alabama, 1998

Worker and employer, logging industry

N = unknown (5 crews with 2 to 15 workers)

Firm size: small

Type of work: mostly physical

Previous inspections: not reported

InterventionsAny by OSHA

OutcomesMain outcome: hazard and safety perception

Including workers' and employers' perception of OSHA enforcement

NotesFunding: United States forest servings

Conflict of interest: no

Geminiani 2008

MethodsData collection: survey, questionnaire

Theory driven

Technique of analysis: descriptive analysis

ParticipantsRepublic of South Africa, time of study unclear

Civil and building constructors, construction industry

N = 626 included, 107 respondents

Firm size: not reported

Type of work: mostly office work

Previous inspections: not reported

InterventionsInspectorate in general

OutcomesMain outcome: effectiveness of labour inspections

Including: opinions and beliefs regarding inspectorates

NotesFunding: not reported

Conflict of interest: no

Gillen 2004

MethodsTechnique of analyses: content analysis, thematic

Data collection: focus group with semi-structured interviews and behavioural observations

ParticipantsUSA, California, 2000

Safety managers, construction industry

N = 22

Firm size: any, average not reported

Type of work: mostly physical

Previous inspections: not reported

InterventionsAny enforcement activity by Cal/OSHA

OutcomesMain outcome: perception of construction safety practices

Including: managers' opinion and beliefs about OSHA enforcement

NotesFunding: California Department of Health Services

Conflict of interest: no

Gray 2006

MethodsTechnique of analyses: narrative, grounded ethnographic themes

Data collection: in depth participative observation (5 months)

ParticipantsCanada, time of study not reported

Workers and employer, manufacturing industry

N = 1 firm

Firm size: not reported

Type of work: mostly physical

Previous inspections: not reported

InterventionsAny enforcement activity of occupational health and safety standards eligible, analysed for inspections

OutcomesMain outcome: the role of worker agencies in regulatory enforcement

Including: reaction by workforce towards planned inspections

NotesFunding: not reported

Conflict of interest: no

Guidotti 1996

MethodsTechnique of analyses: descriptive (cross-tabulation)

Data collection: telephone survey

ParticipantsCanada, Fort Mc Murray region, 1992

Workers, sand oil industry (predominantly 30 to 44 years old (55%), 96% male)

N = 150

Firm size: not reported

Type of work: mostly physical

Previous inspections: not reported

InterventionsAny enforcement activity of occupational health and safety standards

OutcomesMain outcome: health- and safety-related behaviour among oil sands workers

Including: opinion towards enforcement of occupational health and safety standard even if it increases cost or time to complete a job

NotesFunding: Occupational Health and Safety Heritage Grant Program of Labour

Conflict of interest: no

Mayhew 1999

MethodsTechnique of analyses: content analyses, quantitative

Data collection: semi-structured face-to-face interview plus questionnaire with closed and open-ended questions

ParticipantsAustralia, Queensland, 1997

Demolishers, construction industry

N = 31 (18.4% of 168 included workers)

Firm size: not reported

Type of work: mostly physical

Previous inspections: some yes some no, type and time not reported

InterventionsInspection/audit with or without further enforcement activity by jurisdiction

OutcomesMain outcome: impact on OSH performance

Including opinion and beliefs about inspection with or without further enforcement

NotesFunding: DETIR and National Occupational Health and Safety Commission

Conflict of interest: No

 
Comparison 1. Inspection versus no intervention

Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size

 1 Non-fatal injuries, short-term, RCT1Rate Ratio (Random, 95% CI)Totals not selected

    1.1 Unannounced inspection with orders, mostly physical work, unknown firm size, no inspections 2 years prior
1Rate Ratio (Random, 95% CI)0.0 [0.0, 0.0]

 2 Fatal and non-fatal injuries, short-term, CBA1Risk Ratio (Random, 95% CI)Totals not selected

    2.1 Random inspection with or without penalty (type of penalty unknown), mostly physical work, small firms, no inspections 2 years prior intervention
1Risk Ratio (Random, 95% CI)0.0 [0.0, 0.0]

 3 Non-fatal injuries, short-term, panel study3Risk Ratio (Random, 95% CI)0.92 [0.89, 0.95]

    3.1 Inspection (type unknown) with or without penalty (type unknown), mostly physical work, any firm size, prior inspections unknown
2Risk Ratio (Random, 95% CI)0.92 [0.87, 0.97]

    3.2 Inspection (type unknown) with or without citations, mixed type of work (non-fixed site), unknown firm size, no inspections 1 year prior intervention
1Risk Ratio (Random, 95% CI)0.92 [0.86, 0.99]

    3.3 Inspection (type unknown) with or without citations, mixed type of work (fixed site), unknown firm size, no inspections 1 year prior intervention
1Risk Ratio (Random, 95% CI)0.92 [0.86, 0.99]

 4 Fatal and non-fatal injuries, medium-term, CBA1Risk Ratio (Random, 95% CI)Totals not selected

    4.1 Random inspection with or without penalty (type unknown), mostly physical work, small firms, no inspections 2 years prior intervention
1Risk Ratio (Random, 95% CI)0.0 [0.0, 0.0]

 5 Non-fatal injuries, medium-term, panel study3Risk Ratio (Random, 95% CI)0.97 [0.94, 1.01]

    5.1 Programmed and complaint inspection with or without penalty (type unknown), mixed type of work, unknown firm size, prior inspections unknown
3Risk Ratio (Random, 95% CI)0.97 [0.94, 1.01]

 6 Fatal and non-fatal injuries, long-term, CBA1Risk Ratio (Random, 95% CI)Totals not selected

    6.1 Random inspection with or without penalty (type unknown), mostly physical work, small firms, no inspections 2 years prior intervention
1Risk Ratio (Random, 95% CI)0.0 [0.0, 0.0]

 7 Fatal and non-fatal injuries, long-term, ITS-level1Std. Mean Difference (Random, 95% CI)Totals not selected

    7.1 Inspections with or without warning and orders, mostly physical work, unknown firm size, prior inspections unknown
1Std. Mean Difference (Random, 95% CI)0.0 [0.0, 0.0]

 8 Fatal and non-fatal injuries, long-term, ITS-slope1Std. Mean Difference (Random, 95% CI)Totals not selected

    8.1 Inspection with or without warning and orders, mostly physical work, unknown firm size, prior inspections unknown
1Std. Mean Difference (Random, 95% CI)0.0 [0.0, 0.0]

 9 Reduced Exposure, medium-term, RCT1Risk Ratio (Random, 95% CI)Totals not selected

    9.1 Announced inspections with or without warning and orders, type of work not reported
1Risk Ratio (Random, 95% CI)0.0 [0.0, 0.0]

 
Comparison 2. Inspection type versus any other type of Inspection with or without penalties
 
Comparison 3. Inspection with citation versus inspection without citation

Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size

 1 Short-term, non-fatal injuries, panel study1Mean Difference (Fixed, 95% CI)Totals not selected

    1.1 Any type of inspection with citation, mostly physical work, big firms, inspections prior unknown
1Mean Difference (Fixed, 95% CI)0.0 [0.0, 0.0]

 2 Medium-term, non-fatal injuries, panel studies1Mean Difference (Fixed, 95% CI)Totals not selected

    2.1 Any type of inspection with citation, mostly physical work, big firms, inspections prior unknown
1Mean Difference (Fixed, 95% CI)0.0 [0.0, 0.0]

 
Comparison 4. Inspection with penalty versus no intervention or inspection only

Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size

 1 Short-term, non-fatal injury, panel study1Odds Ratio (Random, 95% CI)Totals not selected

    1.1 Programmed or complaint inspection with penalty (type unknown), mostly physical work, small firms, prior inspections unknown
1Odds Ratio (Random, 95% CI)0.0 [0.0, 0.0]

 2 Medium-term, non-fatal injury, panel study1Odds Ratio (Random, 95% CI)Totals not selected

    2.1 Programmed or complaint inspection with penalty (type unknown), mostly physical work, small firms, prior inspections unknown
1Odds Ratio (Random, 95% CI)0.0 [0.0, 0.0]

 
Comparison 5. More penalties versus fewer penalties

Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size

 1 Short-term, exposure compliance (< 1 violation), panel study2Rate Ratio (Fixed, 95% CI)Totals not selected

    1.1 Inspection (type unknown) with penalty (type unknown), mostly physical work, small firms, inspections prior unknown
1Rate Ratio (Fixed, 95% CI)0.0 [0.0, 0.0]

    1.2 Inspection (type unknown) with penalty (type unknown), mostly physical work, big firms, inspections prior unknown
1Rate Ratio (Fixed, 95% CI)0.0 [0.0, 0.0]

 
Comparison 6. First inspection versus more than one inspection

Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size

 1 Short-term, exposure compliance (< 1 violation), panel study2Rate Ratio (Random, 95% CI)Totals not selected

    1.1 Any inspection with or without penalty (type unknown), mostly physical work, small firms, inspections prior unknown
1Rate Ratio (Random, 95% CI)0.0 [0.0, 0.0]

    1.2 Any inspection with or without penalty (type unknown), mostly physical work, big firms, inspections prior unknown
1Rate Ratio (Random, 95% CI)0.0 [0.0, 0.0]

 
Comparison 7. Six inspections versus more than six inspections

Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size

 1 Short-term, exposure compliance (< 1 violation), panel study1Rate Ratio (Random, 95% CI)Totals not selected

    1.1 Any inspection with or without penalties (type unknown), mostly physical work, big firms, inspections prior unknown
1Rate Ratio (Random, 95% CI)0.0 [0.0, 0.0]

 
Comparison 8. More inspection hours versus fewer hours

Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size

 1 Short-term, exposure compliance (< 1 violation), panel study2Rate Ratio (Random, 95% CI)0.96 [0.94, 0.99]

    1.1 Any type of inspection with or without penalty (type unknown), mostly physical work, small firms, inspections prior unknown
1Rate Ratio (Random, 95% CI)1.06 [0.87, 1.28]

    1.2 Any type of inspection with or without penalty (type unknown), mostly physical work, big firms, inspections prior unknown
1Rate Ratio (Random, 95% CI)0.96 [0.93, 0.99]

 
Comparison 9. Autonomy oriented versus coercive oriented inspectors

Outcome or subgroup titleNo. of studiesNo. of participantsStatistical methodEffect size

 1 Long-term, exposure (number of visits needed to resolve non-compliance), panel study1Rate Ratio (Random, 95% CI)Totals not selected

    1.1 Type of inspection unknown with or without compliance orders, type of work unknown, firm size unknown, inspections prior unknown
1Rate Ratio (Random, 95% CI)0.0 [0.0, 0.0]

 
Summary of findings for the main comparison. Inspection compared to no intervention for preventing occupational diseases and injuries

Inspection compared to no intervention for preventing occupational diseases and injuries

Patient or population: firms potentially subject to inspection

Setting: verification of compliance with occupational health and safety legislation
Intervention: inspection
Comparison: no intervention

OutcomesIllustrative comparative risks* (95% CI)Relative effect
(95% CI)
No of participating firms
(studies)
Quality of the evidence
(GRADE)
Comments

Assumed riskCorresponding risk

No interventionInspection

Fatal and non-fatal injuries in RCT, short-term follow-up
WC claims
Follow-up: mean 21 months
ModerateRR 1.04
(0.9 to 1.21)
1402
(1 study)
⊕⊕⊝⊝
low1,2

41 per 100043 per 1000
(37 to 50)

Fatal and non-fatal injuries in CBA, medium-term follow-up
WC claims
Follow-up: mean 24 months
ModerateRR 0.87
(0.75 to 1.02)
818
(1 study)
⊕⊕⊝⊝
low1

31 per 100027 per 1000
(23 to 32)

Fatal and non-fatal injuries in CBA, long-term follow-up
WC claims
Follow-up: mean 48 months
ModerateRR 0.77
(0.64 to 0.92)
818
(1 study)
⊕⊕⊝⊝
low

31 per 100024 per 1000
(20 to 29)

Fatal and non-fatal crashes, ITS-level
Crash data
Follow-up: mean 36 months
The median level of fatal and non-fatal crashes was
2.99 crashes per 100 trucks
The mean level of fatal and non-fatal crashes in the year after the intervention was
2.42 standard deviations lower
(2.88 to 1.96 lower)
6200
(1 study)
⊕⊕⊝⊝
low

Fatal and non-fatal crashes, ITS-slope
Crash data
Follow-up: mean 36 months
The median fatal and non-fatal crashes was
2.99 crashes per 100 trucks
The trend of fatal and non-fatal crashes after the intervention was
0.89 standard deviations lower
(0.98 to 0.8 lower)
6200
(1 study)
⊕⊕⊝⊝
low

The basis for the assumed risk (e.g. the median control group risk across studies) is provided in footnotes. The corresponding risk (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI).
CBA: controlled before-after study; CI: confidence interval;ITS: interrupted time series; RCT: randomised controlled trial; RR: risk ratio; WC: Workers' Compensation

GRADE Working Group grades of evidence
High quality: Further research is very unlikely to change our confidence in the estimate of effect.
Moderate quality: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.
Low quality: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.
Very low quality: We are very uncertain about the estimate.

 1Compliance with inspections unclear; no blinding.
2Wide confidence interval.
 
Table 1. Characteristics of panel studies

Article IDIndustry type*Country*Outcome*Time SpanData sourceN of

variables
Intervention type*Study ID

IMISBLS

Weil 2001ConstructionUSAExposure87 - 93 x20Inspection with or without penaltyWeil 2001

Morantz 2007ConstructionUSAExposure87 - 93 x 

Weil 1996Custom woodworkUSAExposure72 - 91 x 13Inspection with or without penaltyWeil 1996

Robertson 1983ManufacturingUSAInjury73 - 80  x5Inspection and penaltyRobertson 1983

Scholz 1990ManufacturingUSAInjury79 - 85 x xInspection with or without penaltyGray 2005a

Scholz 1997ManufacturingUSAInjury79 - 85 x xInspection with or without penalty

Gray 2005aManufacturingUSAInjury79 - 85 x10Inspection with or without penalty

Gray 2005bManufacturingUSAInjury87 - 91 x10Inspection with or without penaltyGray 2005b

Mendelhoff 2005ManufacturingUSAInjury92 - 98 x xInspection with or without penaltyGray 2005c

Gray 2005cManufacturingUSAInjury92 - 98 x x10Inspection with or without penalty

Haviland 2012ManufacturingUSAInjury98 - 05 x 7Inspection with penaltyHaviland 2012

Haviland 2010ManufacturingUSAInjury98 - 05 x Inspection with penalty

Haviland 2008ManufacturingUSAInjury98 - 05 x Inspection with penalty

Kniesner 2004MiningUSAInjury83 - 97  Inspection with penaltyKniesner 2004

Burstyn 2010MixedCanadaExposure03 - 06  Inspection with or without penaltyBurstyn 2010

Foley 2012MixedUSAInjury99 - 00xInspection with or without penaltyFoley 2012

Smith 1979aMixedUSAInjury72 - 73  xInspection with or without penaltySmith 1979a

Smith 1979bMixedUSAInjury73 - 74  xInspection with or without penaltySmith 1979b

 *Articles with same characteristics are considered same study.
  • IMIS: Integrated Management Information System, (Occupational Safety and Health Administration, USA)
  • BLS: Bureau of Labor Statistics, (USA)

 
Table 2. Characteristics of included qualitative studies

Bordas 2001

MethodsTechnique of analyses: triangulation

Data collection: observations (behaviour, at the workplace) and interviews (formal, informal and structured)

ParticipantsUSA, east central Alabama, 1998

Worker and employer, logging industry

N = unknown (5 crews with 2 to 15 workers)

Firm size: small

Type of work: mostly physical

Previous inspections: not reported

InterventionAny by OSHA

OutcomeMain outcome: hazard and safety perception

Including workers' and employers' perception of OSHA enforcement

NotesStudy year: 1998

Funding: United States forest servings

Conflict of interest: no

Geminiani 2008

MethodsData collection: survey questionnaire

Theory driven

Technique of analysis: descriptive analysis

ParticipantsRepublic of South Africa, time of study unclear

Civil and building constructors, construction industry

N = 626 included, 107 responses

Firm size: not reported

Type if work: mostly office work

Previous inspections: not reported

InterventionInspections in general

OutcomeMain outcome: effectiveness of labour inspections

Including: opinions and beliefs regarding Inspections

NotesFunding: not reported

Conflict of interest: none

Gillen 2004

MethodsTechnique of analysis: content analysis, thematic

Data collection: focus group with semi-structured interviews and behavioural observations

ParticipantsUSA, California, 2000

Safety managers, construction industry

N = 22

Firm size: any, average not reported

Type of work: mostly physical

Previous inspections: not reported

InterventionAny enforcement by Cal/OSHA

OutcomeMain outcome: perception of construction safety practices

Including: managers' opinions and beliefs about OSHA enforcement

Gray 2006

MethodsTechnique of analyses: narrative, grounded ethnographic themes

Data collection: in depth participative observation (5 months)

ParticipantsCanada, time of study not reported

Workers and employee, manufacturing industry

N = 1 firm

Firm size: not reported

Type of work: mostly physical

Previous inspections: not reported

InterventionAny enforcement activity of occupational health and safety standards eligible, analysed for inspections

OutcomeMain outcome: the role of worker agencies in regulatory enforcement

Including: reaction by workforce towards planned inspections

NotesStudy year: not reported

Funding: not reported

Conflict of interest: no

Guidotti 1996

MethodsTechnique of analyses: descriptive (cross-tabulation)

Data collection: telephone survey

ParticipantsCanada, Fort McMurray region, 1992

Workers, sand oil industry (predominantly 30 to 44 years old (55%), 96% male)

N = 150

Firm size: not reported

Type of work: mostly physical

Previous inspections: not reported

InterventionAny enforcement activity of occupational health and safety standards

OutcomeMain outcome: health- and safety-related behaviour among oil sands workers

Including: opinion towards enforcement of occupational health and safety standard even if it increases cost or time to complete a job

NotesStudy year: 1992

Funding: Occupational Health and Safety Heritage Grant Program of Labour

Conflict of interest: no

Mayhew 1999

MethodsTechnique of analyses: content analyses, quantitative

Data collection: semi-structured face-to-face interview plus questionnaire with closed and open-ended questions

ParticipantsAustralia, Queensland, 1997

Demolishers, construction industry

N = 31 (18.4% of 168 included workers)

Firm size: not reported

Type of work: mostly physical

Previous inspections: some yes some no, type and time not reported

Interventioninspection/audit with or without further enforcement activity by jurisdiction

OutcomeMain outcome: impact on OSH performance

Including opinion and beliefs about inspection with or without further enforcement

NotesStudy year: 1997

Funding: DETIR and National Occupational Health and Safety Commission

Conflict of interest: no

 
  • OSHA: Occupational Safety and Health Administration

 
Table 3. Critical appraisal of included qualitative studies

Qualitative Study ID:Bordas 2001Geminiani 2008Gillen 2004Gray 2006Guidotti 1996Mayhew 1999

Method, Reporting – consistent and neutral

1. Is there a clear connection between philosophical perspective, methodology,

objectives, methods used to collect data, representation and analyses of data?
Unclear: applied method not described, no themes derivedNo: review of literature, does not address the differences in contractor- inspector-attitudesYesYesUnclear: not reportedYes

2. Were the researchers open about potential bias (context presented and

analysed in conclusion)?
Unclear: context not describedNo: context rarely described, not included in conclusionUnclear: context not describedYesUnclear: context not describedUnclear: context not described

3. Is the reporting clear and coherent (sampling method, recruitment conditions,

inclusion and exclusion criteria, method of data collection, description of the

derivation of themes and inclusion of supporting quotations)?
No: description of derivation of themes and inclusion of citations missingNo: descriptions of the derivation of the themes and inclusion of citations missingYesNo: method of data collection not described, description of derivation of themes missingNo: not reportedNo: description of derivation of themes and inclusion of citations missing

Method, Subjects – credibility

4. Is the recruitment of the study participants free of selection bias (e.g. workers are not selected by employer)?Unclear: selection not fully described, voluntary participationUnclear: survey with 18% response rateYes (various methods of outreach, includes monetary incentive, voluntary participation)Unclear: one firm, recruitment not reportedYes: via telephone registerUnclear: exclusion and drop-out unclear

5. Are characteristics of subjects and workplace considered for discussion and implication of findings? (age, gender, type of work, firm size, prior inspections)No: subjects' age etc. not described, industry described, firm size and previous inspections missingNo: except for type of industry all descriptions missingNo: age, gender, ethnicity, firm size and previous inspection experiences not assessedNo: firm size and previous inspections missing (author withheld information to keep firm identity confidential)No: none reportedNo: age, gender, ethnicity, firm size and previous inspection experiences not assessed

6. Is the research ethical according to current criteria OR for recent studies is

there evidence of ethical approval by an appropriate body?
Unclear: not reportedUnclear: not reportedYesYes: ethical approval not reported but anonymity addressedUnclear: not reportedUnclear: not reported

Analyses and conclusion - transferability

7. Is/are the specific tool(s) of enforcement described? Are different tools

analysed in separate categories?
No: interventions not specifiedYes: inspection onlyNo: no specification of uniform enforcement toolsYes (outcome of interest only about planned inspection)No: enforcement in generalNo: types specified as intervention but combined in analyses (inspection with or without penalty)

8. Is a range of methods used to draw similar conclusions (triangulation)?Unclear: questionnaire and personal interviews done, not reported from which source the opinions derived and if similar conclusionsNo: questionnaire onlyUnclear: focus group and observation used but conclusions not reportedNo: only observationNo: description of firm size and previous inspections missingNo triangulation

9. Does the representation of data fit the views of the participants studied

(e.g. minimum 2 researchers independently analysed the data, or outside

auditors, or participants validate the findings)?
Unclear: not reportedUnclear: not reportedYesNo: no validating of findings, interpretation of data by one researcherUnclear: not reportedUnclear: not reported

Total score (Yes) out of 9015411

Total quality *

High quality: at least 4 YES in the 1st group and 2 YES in the 2nd group

Moderate quality: at least 2 YES in the 1st group and 3 YES in the 2nd group

Low quality: less than 2 YES in the 1st group
LowLowModerateLowLowLow

 *Questions are categorised into 1st group (no. 4, 5, 7, 8 and 9) and 2nd group questions (no. 1, 2, 3 and 6) according to the likelihood of influencing the outcome.