Presented at the Department of Emergency Medicine Research Day, Department of Emergency Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada, June 16, 2009.
Improving Child Protection in the Emergency Department: A Systematic Review of Professional Interventions for Health Care Providers
Version of Record online: 27 JAN 2010
© 2010 by the Society for Academic Emergency Medicine
Academic Emergency Medicine
Volume 17, Issue 2, pages 117–125, February 2010
How to Cite
Newton, A. S., Zou, B., Hamm, M. P., Curran, J., Gupta, S., Dumonceaux, C. and Lewis, M. (2010), Improving Child Protection in the Emergency Department: A Systematic Review of Professional Interventions for Health Care Providers. Academic Emergency Medicine, 17: 117–125. doi: 10.1111/j.1553-2712.2009.00640.x
Funding for this project was partially provided by a Knowledge Synthesis grant awarded to the principal author from the Canadian Institutes of Health Research (CIHR). Total amount awarded: $94,655. Dates of grant support: November 2009 to 2010. Dr. Newton and Ms. Curran are salary supported by the Canadian Child Health Clinician Scientist Program, in partnership with SickKids Foundation, Child & Family Research Institute (British Columbia), Women & Children’s Health Research Institute (Alberta), Manitoba Institute of Child Health.
- Issue online: 27 JAN 2010
- Version of Record online: 27 JAN 2010
- Received June 17, 2009; revision received September 4, 2009; accepted September 8, 2009.
- child abuse;
- health care delivery;
- systematic review
Objectives: This systematic review evaluated the effectiveness of professional and organizational interventions aimed at improving medical processes, such as documentation or clinical assessments by health care providers, in the care of pediatric emergency department (ED) patients where abuse was suspected.
Methods: A search of electronic databases, references, key journals, and conference proceedings was conducted and primary authors were contacted. Studies whose purpose was to evaluate a strategy aimed at improving ED clinical care of suspected abuse were included. Study methodologic quality was assessed by two independent reviewers. One reviewer extracted the data, and a second checked for completeness and accuracy.
Results: Six studies met the inclusion criteria: one randomized controlled trial (RCT), one quasi-RCT, and four observational studies. Study quality ranged from modest (observational studies) to good (trials). Variation in study interventions and outcomes limited between-study comparisons. The quasi-RCT supported self-instructional education kits as a means to improve physician knowledge for both physical abuse (mean ± standard deviation [SD] pretest score = 13.12 ± 2.36; mean ± SD posttest score = 18.16 ± 1.64) and sexual abuse (mean ± SD pretest score = 10.81 ± 3.20; mean ± SD posttest score = 18.45 ± 1.79). Modest-quality observational studies evaluated reminder systems for physician documentation with similar results across studies. Compared to standard practice, chart checklists paired with an educational program increased physician consideration of nonaccidental burns in burn cases (59% increase), documentation of time of injury (36% increase), and documentation of consistency (53% increase) and compatibility (55% increase) of reported histories. Decisional flow charts for suspected physical abuse also increased documentation of nonaccidental physical injury (69.5% increase; p < 0.0001) and had a similar significant effect as checklists on increasing documentation of history consistency and compatibility (69.5 and 70.0% increases, respectively; p < 0.0001) when compared to standard practice. No improvements were noted in these studies for documentation of consultations or current status with child protective services. The introduction of a specialized team and crisis center to standardize practice had little effect on physician documentation, but did increase documentation of child protective services involvement (22.7% increase; p < 0.005) and discharge status (23.7% increase; p < 0.02). Referral to social services increased in one study following the introduction of a chart checklist (8.6% increase; p = 0.018). A recently conducted multisite RCT did not support observational findings, reporting no significant effect of educational sessions and/or a chart checklist on ED practices.
Conclusions: The small number of studies identified in this review highlights the need for future quality studies that address care of a vulnerable clinical population. While moderate-quality observational studies suggest that education and reminder systems increase clinical knowledge and documentation, these findings are not supported by a multisite randomized trial. The limited theoretical base for conceptualizing change in health care providers and the influence of the ED environment on clinical practice are limitations to this current evidence base.
ACADEMIC EMERGENCY MEDICINE 2010; 17:117–125 © 2010 by the Society for Academic Emergency Medicine
In 2006, almost 3.6 million American children and youth received a child protection service investigation or assessment, with 28.6% of the investigations determining abuse or neglect.1 Canadian data indicate that, in 2003, just under a quarter of a million investigations of alleged child abuse and neglect were conducted, with 50% of those investigations being substantiated.2,3 These statistics have been critiqued as underrepresentative of the magnitude of abuse,1,2,4,5 and there remains an opportunity to not only improve the detection of abuse, but also initiate protective services. Abuse often consists of a series of incidents rather than a single event,6 and cases are reported after several medical evaluations. There is evidence to suggest that 20% to 30% of children and youth who die from abuse have been seen by health care providers for abusive sequelae before abuse was formally identified.7,8
The emergency department (ED) represents a critical system entry point for crisis-based health care visits. For children and youth who present to the hospital for accidents related to neglect or inflicted injuries, ED personnel may be their first contact and opportunity for initiating protective services.9–13 Abused children and youth have higher rates of ED use before diagnosis compared to other patients, but do not necessarily have distinguishable features to identify them as a “high risk” population for screening.14 Health care providers in the ED are therefore required to rely on their clinical skills and judgment to identify signs and symptoms consistent with abuse. Prior hospital visits that often meet criteria for suspicion of abuse, however, can go unreported or unidentified or be poorly documented.6,15–17 Factors known to facilitate or impede the assessment and management of child abuse include perceived time demands, obstacles to an effective health care system response, training gaps, and concerns with clinical abilities.18–20 Strategies aimed at improving ED clinical practices have been evaluated, but to date, have not been synthesized to formulate quality improvement recommendations. This systematic review evaluated the effectiveness of professional (e.g., educational initiatives) and organizational (e.g., implementation of specialized teams) interventions aimed at improving medical processes, such as documentation or clinical assessments by health care providers, in the care of pediatric ED patients where abuse was suspected.
A research librarian, with input from the clinical research team, developed and implemented systematic search strategies using language (English and French) and year (1985 to 2009) restrictions. The search was conducted in 14 high-yield electronic databases: MEDLINE, Ovid MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, Cochrane Central Register of Controlled Trials, OVID HealthStar, Cochrane Database of Systematic Reviews, Health Technology Assessment Database, Database of Abstracts of Reviews of Effects, ACP Journal Club, PsycINFO, CINAHL, SocIndex, ProQuest Theses and Dissertations, and Child Welfare Information Gateway. Reference lists, key journals, and conference proceedings (Canadian Association of Emergency Physicians, Society for Academic Emergency Medicine, American College of Emergency Physicians, and the Canadian Paediatric Society) were also reviewed. The initial search was conducted in January 2008 and was updated in January 2009. Search terms included abuse, violence, risk or risk factors or risk assessment, emergency health service, and emergency care or emergency treatment. Comprehensive strategies used in each database are available from the corresponding author on request.
Two reviewers independently screened the search results. Experimental and observational studies were eligible for review inclusion.21 Studies with the primary purpose of evaluating a strategy aimed at improving processes involved in the clinical care of suspected physical or sexual abuse in the ED were included. Health care providers targeted in the studies had to work with children and youth (≤18 years), and a clinically relevant outcome (e.g., identification of abuse, documentation, referral/involvement of protective services) needed to be reported.
The Jadad five-point scale was used to assess control of bias in randomized controlled trials (RCTs). Points were allocated as follows: randomization (0–2 points), double blinding (0–2 points), and withdrawals/dropouts (0–1 point).22 Concealment of allocation was assessed as adequate, inadequate, or unclear using criteria from Schulz et al.23 All other study designs were assessed using methodologic criteria developed by Downs and Black.24 Quality was measured by study reporting, external and internal validity, and power, with an assigned maximal quality index (QI) of 29. QI scores of >20 were considered good, 11 to 20 moderate, and <11 poor. Two reviewers (BZ, MPH) independently analyzed each study’s methodologic quality, and agreement was quantified with the kappa statistic.25
Data Extraction and Analysis
Data from the final six studies were extracted using a standardized form that encompassed elements of study characteristics (e.g., language of publication, country), characteristics of the study population, study setting, description of the intervention and comparisons, outcome measures and measurements tools, and results. Data were extracted by one reviewer and checked for accuracy and completeness by a second reviewer. In the case of unclear or unreported information in the original studies, primary authors were contacted. Heterogeneity in the interventions, outcomes, and unit of analysis precluded the use of meta-analysis to pool and interpret study results; therefore, a descriptive analysis of our findings based on the quality assessment is presented.
The search strategy identified 282 studies as potentially relevant to the review, all of which were assessed for study inclusion by two independent reviewers. As shown in Figure 1, six studies were confirmed for inclusion: one RCT,14 one quasi-RCT,26 and four observational studies of various designs.27–30 One study was excluded because we could not confirm if it targeted ED-based health care providers.31
Study characteristics and overall study quality are outlined in Table 1. All studies examined the impact of physician-oriented strategies to improve assessment and/or care. Studies took place in either the United States (n = 4) or United Kingdom (n = 2). The age of children and youth cared for by study participants varied. All studies focused on the assessment and treatment of physical abuse; one study also focused on sexual abuse. Studies reviewed the clinical care of children and youth with 1) any nonaccidental injury,28,30 2) nonaccidental burns,27,29 or 3) unknown.14
|First Author, Year (Country)||Study Design||Target Patient Population||Quality Assessment Score|
|Sample Size (n): Intervention, Comparison Group(s)||Age||Type of Abuse|
|Guenther, 200914 (USA)||RCT||Total = 1,575*||<36 months||Physical||3/5|
|Full intervention = 543|
|Partial intervention = 473|
|Comparison = 559|
|Showers, 199126 (USA)||Quasi-RCT||Total = 191†||NR||Physical, sexual||16/29|
|Group 1 (physical abuse kit) = 82|
|Group 2 (sexual abuse kit) = 109|
|Clark, 199727 (USA)||Controlled before–after||Total = 302*‡||1 months–18 yr||Physical||19/29|
|Intervention = 215|
|Comparison = 87|
|Limbos, 199830 (USA)||Nonconcurrent (historical) cohort||Total = 73*‡||<18 yr||Physical||17/29|
|Intervention = 29|
|Comparison = 44|
|Benger, 200129 (UK)||Controlled before–after||Total = 200*‡||0–6 yr||Physical||21/29|
|Intervention = 100|
|Comparison = 100|
|Benger, 200228 (UK)||Controlled before–after||Total = 2,000*‡||0–5 yr||Physical||19/29|
|Intervention = 1,000|
|Comparison = 1,000|
Methodologic quality of the studies varied (Table 1). Overall, reviewer agreement on quality was good (κ = 0.76)29 with reviewer disagreements resolved through consensus. While a randomized design was an overt strength in reducing risk of bias in the review’s one RCT,14 because it is impossible to double-blind studies of provider-based interventions, the quality score was limited to a modest maximum of three on the Jadad scale (rather than 5; range = 1 to 3). Concealment of allocation was assessed as unclear.23 Using the maximal QI, scores for the remaining studies on the Downs and Black checklist ranged from 16 (moderate quality) to 21 (good quality) out of a possible 29.26–30 Key methodologic strengths for higher-quality studies included detailed descriptions of the study design and analyses, inclusion of a representative sample, and adjustment for confounding factors. Lower scores were the result of insufficient study population description, small sample sizes, lack of a control group, and minimal reporting of statistical analyses (Table 1).
Description of Health Care Provider Interventions and Outcomes
An equal number of studies evaluated the effect of a single intervention on changing behaviors27,28 and knowledge26 for physicians compared to a multimodal intervention.14,29,30 Studies that evaluated educational interventions for physicians examined their impact on knowledge26 and documentation.14,29 While modality differed between these studies, the educational content provided was comparable and included abuse recognition and management (Table 2). The effect on physician behaviors was evaluated using chart-appended reminder systems that included checklists14,27,29 and a decisional flow chart.28 The effects of specialized services (child abuse and neglect team, child crisis center) and resident training (a workshop and optional elective) were examined in one study30 with a similar focus on practice behaviors. No studies evaluated the effect of interventions on clinical outcomes (e.g., psychosocial, physical) in children, youth, or their families (Table 2).
|First Author, Year||Intervention||Comparison||Main Outcome Domain(s)||Findings|
|Guenther, 200914||Three 1-hour didactic educational sessions (full intervention): • 45-minute presentation on recognition and documentation • 15-minute case discussion Screening checklist: • Inserted into charts of target population for injury presentations • Reminder to consider abuse diagnosis||1. No intervention (standard care) 2. One 1-hour educational session (partial intervention): • 45-minute presentation on recognition and documentation • 15-minute case discussion||Documentation of: 1. Time between injury and ED care 2. History consistency and compatibility 3. Injury description 4. Anterior fontanel exam in patients <12 months 5. Skin exam 6. Skeletal survey 7. CT scan 8. Referral to protection services or subspecialty||GEE models indicated no evidence of significant ED practice change or trend across hospitals (p > 0.2).|
|Showers, 199126||Self-instructional (educational) kit on physical (Group 1) and sexual (Group 2) abuse: • Learner objectives • Summary instructional and clinical information with case illustrations and supplementary article abstracts • Self-evaluation activities • Answers to test questions and suggested interpretations for case illustrations||Pretest on knowledge, case recognition, and attitudes about abuse||Physician knowledge: 1. Case recognition 2. Educational kit content 3. Program evaluation||Significant differences in knowledge score for both groups: Group 1 (mean ± SD pretest score 13.12 ± 2.36 vs. mean ± SD posttest score 18.16 ± 1.64). Group 2 (mean ± SD pretest score 10.81 ± 3.20 vs. mean ± SD posttest score 18.45 ± 1.79).|
|Clark, 199727||Burn profile checklist: • Placed of chart of all burn injuries (person responsible for placement not reported) • Reminder points for history and physical exam of burns||Standard practice (ND)||Referral to social services||Documentation of burn cases referred to social services in intervention (26/215; 12.1%) vs. comparison (3/87; 3.4%) groups (p = 0.018).|
|Limbos, 199830||Standard practice plus: 1. Introduction of suspected child abuse and neglect team 2. Development of child crisis center 3. Increased resident training||Standard practice 1. Structured and unstructured forms to report past and present historical information, diagnostic procedures and results, disposition, and drawings to illustrate physical findings||Physician documentation: 1. History (including time of arrival, accompaniment, time and nature of injury, witnesses to injury, history of past injuries, developmental history) 2. Physical examination (including location/size/color of injuries, drawings and photographs) 3. Supplemental diagnostic procedures (skeletal survey and lab tests) 4. Disposition (child protection services, discharge status)||Significant improvement in the documentation of the involvement of child protective services—intervention (29/29; 100%) vs. comparison (34/44; 77.2%) groups (p < 0.005)—and disposition status—intervention (26/29; 89.6%) vs. comparison (29/44; 65.9%) groups (p < 0.02). Significant decreases in documentation of time of arrival—intervention (12/29; 41.3%) vs. comparison (31/44; 70.4%)—and radiographs taken—intervention (11/29; 37.9%) vs. comparison (37/44; 84.1%). No improvements in the documentation of injury descriptions or history.|
|Benger, 200129||Educational program: • Presentation on recognition and management of abuse (length unknown) Chart checklist (sticker): • Sticker placed on the front of charts (person responsible for placement not reported) • Reminder checklist to consider abuse diagnosis for burns/scalds||Standard practice (ND)||Physician documentation: 1. Current status in child protection register 2. Time injury occurred 3. History consistency and compatibility 4. Possibility of nonaccidental injury being considered 5. General state and behavior of patient 6. Presence of other injuries 7. Tetanus status 8. Consultation with other colleagues regarding possibility of nonaccidental injury||Improved frequency (97% of flagged charts) and adequacy of physician documentation after intervention: Documentation of time of injury (36% increase), consistency of history and compatibility of injury (53% and 55% increase, respectively), and suspicion of nonaccidental event (59% increase). No improvements in physician documentation of child protection register status, tetanus status, or consultation with colleagues.|
|Benger, 200228||Reminder flow chart (sticker): • Sticker placed on front of charts by nurses after initial assessment for injury • Decisional flow chart to consider abuse diagnosis||Standard practice (ND)||Physician documentation: 1. Time between injury and ED care 2. History consistency and compatibility 3. Possibility of nonaccidental injury being considered 4. General state and behavior of patient 5. Presence of other injuries 6. Consultation with protection services and/or other colleagues||Improvement in the documentation of consideration of nonaccidental injury—intervention (711/1000) vs. comparison (16/1000) groups (p < 0.0001) and across other documentation standards (p < 0.0001) with the exception of consultation with colleagues including child protective services.|
Intervention Effects on Clinical Knowledge and Practice
Educational programs with14,29 and without26 reminder systems for patient charts demonstrated a mixed impact (Table 2). In their evaluation of the effectiveness of two self-instructional programs, Showers and Laird26 reported a significant increase in mean scores of physician knowledge of both physical and sexual abuse (p < 0.0001). Benger and McCabe29 reported significant improvements in physician documentation through the introduction of an educational program and chart reminder for pediatric burn and scald patients. The intervention resulted in a 59% increase in the consideration of nonaccidental injury and also improved physician documentation of time of injury (36% increase in documentation), as well as consistency of history and compatibility of injury (53 and 55% increase in documentation, respectively). A recently published RCT by Guenther et al.14 contradicts both the Showers and Laird26 and Benger and McCabe29 study findings. This multisite trial found that multiple educational interventions along with a reminder chart did not significantly improve ED documentation practices of cases of suspected physical abuse.14
Studies evaluating the effect of reminder systems on physician documentation28 and referral to social services27 also yielded mixed results (Table 2). Benger and Pearce28 found that, compared to standard practice, flow charts placed on patient charts consistently increased physician documentation on all expected assessment facets, while the use of a checklist by Clark et al.27 improved physician reporting of nonaccidental burns to social services.
As shown in Table 2, only one study, of modest quality, examined the effect of the introduction of specialized, in-house services and training on physician documentation.30 Significant improvements were noted in documentation of time of arrival to the ED, the involvement of child protective services, and disposition status. Improvements were not observed in the documentation of history and physical examinations.30
One study reported the response rate to their mail-out educational intervention, and three studies evaluated adherence to their ED-based interventions. Showers and Laird26 reported a 22 and 30% response rate for return of test booklets on physical and sexual abuse, respectively. Adherence to ED-based interventions was similar across the three reporting studies: 60% inclusion of a documentation reminder in eligible charts,29 72% inclusion of a documentation reminder in eligible charts,28 and 78% inclusion of a chart checklist in eligible charts.27
This systematic review evaluated the effectiveness of interventions aimed at improving medical processes, such as documentation or clinical assessments, in the care of pediatric ED patients where abuse was suspected. While moderate-quality studies suggested education and reminder systems increased various aspects of clinical knowledge and physician documentation,26–30 these findings were not supported by a higher-quality randomized trial, which reported limited to no effect on clinical practice trends.14 The lack of a theoretical base for conceptualizing the change in health care provider behavior and the effect of the ED environment on clinical practice makes it difficult to determine what influenced the medical process changes in the studies. Noted variability in documentation (termed “sporadic”) in the trial by Guenther et al.14 highlights the possibility of unaccounted-for factors that influence clinical practices.
There are no criterion standards for developing and evaluating educational, behavioral, or organizational interventions designed to improve health care professional practices. Modest and mixed effects of these interventions are noted in comprehensive reviews.31–37 Objectives remain for ED researchers to better understand factors that facilitate or impede physician behavior change, and the ideal form interventions and health care systems need to take to effectively address these factors and influence clinical practices.38–41 While the scope of this discussion is beyond the purpose of this paper, systematic reviews have demonstrated that a range of professional interventions such as audit and feedback,37 continuing medical education (CME),42–45 printed educational materials (including clinical practice guidelines and peer-reviewed publications),33 and educational outreach31 have varied effects in different clinical practice settings and emphasizes the need for theoretically driven and evaluated interventions. This tenet also holds true for improving medical processes for suspected child abuse. Socolar et al.,46 for example, used an audit and feedback strategy with physicians in the Child Medical Evaluation Program to target physician documentation and knowledge of child sexual abuse. While discrete changes were noted in physician history taking, tailored feedback with directed educational resources did not improve documentation or knowledge in this large RCT. Results indicated that CME for child sexual abuse, use of a structured record for documentation, and physician sex (female) were significantly associated with better history documentation. Factors such as physician attitudes and beliefs were not measured,46 and the use of a theoretical framework to link intervention strategies with evaluation outcomes would have strengthened interpretation of study results.
In future studies of ED medical processes for suspected child abuse, we propose exploring a comprehensive set of effect mediators and moderators across practice behaviors and measuring barriers and facilitators to ED practice change. As a first step, study designs must acknowledge the complexity of health care provider–focused interventions and should include the following: 1) increased sample sizes to detect the small to modest effects health care provider interventions can have, 2) independent comparison groups (trial designs versus observational designs), and 3) measurement of the extent to which interventions were implemented and adhered to in the ED. The nature of behaviors targeted for improvement also requires consideration. An asset of the studies in this review, irrespective of their quality, was the inclusion of evidence-based criteria from which to evaluate documentation practices. However, these behaviors are complex and likely require a number of cognitive and behavioral steps before a change in documentation behavior is realized (such as a cascade of decision-making). Future studies must be theoretically driven to rigorously conceptualize these steps.
To optimize practice change, future interventions aimed at improving medical processes need to account for the complexity of the ED clinical practice setting and how this can affect care and documentation practices. Effect mediators and moderators of various practice behaviors need to be identified and explored, as they can account for how different interventions impact ED care quality and delivery. Factors known to facilitate or impede the assessment and management of child abuse also need to be measured (e.g., concerns with personal abilities, time demands, perceived obstacles to system responses, training gaps, human resources). Factors inherent in the ED practice setting also need to be accounted for, including the high patient volume, high levels of diagnostic uncertainty, limited clinician performance feedback, the multitude of interruptions and distractions for clinicians, and the complexity of care—all of which contribute to a clinical practice environment that is prone to clinical error and high stress.47 Accounting for the effects of these factors when measuring practice change will assist in the detection of true intervention effects.
Accounting for the multiple roles involved in delivering care may help to better delineate processes required to improve care and patient outcomes. The evaluation of professional interventions targeting other emergency health care providers, such as nurses and social workers, would be of value. Also, the evaluation of organizational strategies aimed at optimizing the role of multidisciplinary teams would be of value, and in particular, the role of physicians alongside child protection services and the police. Such studies would help answer questions of how best to carry out comprehensive child protection in the ED.
This review is limited by the small number of studies available meeting our review objective. Other limitations stem primarily from the studies themselves. Confidence in the observational studies’ findings is tempered by the studies’ modest methodologic quality, as well as inconsistent results among the included studies. Heterogeneity in the interventions, outcomes, and unit of analysis limited our ability to make between-study comparisons. The studies also did not assess clinical outcomes in patients or families (e.g., emotional distress) associated with efforts such as charting and documentation.
These limitations are compounded by the lack of a theoretical base for conceptualizing behavior change and the influence of clinical environment on clinical practice across all studies. To date, one of the major limitations of provider-focused intervention studies has been the lack of underlying theory to support the choice of intervention.32 Research in this area has been “an expensive version of trial and error.”48 The limitations of atheoretical interventions have been duly noted by Grimshaw and colleagues,49,50 which have resulted in a number of well-executed, theoretically driven studies.51–56 Social cognitive theories, such as the theory of planned behavior, for example, have aided interventions to account for the cognitive mechanisms that underlie behavior change in provider-oriented interventions. Attention to the ED provider–specific barriers and facilitators while considering theoretical perspectives to develop and evaluate future provider-focused interventions may result in more effective (and tailored) interventions in emergency medicine41 and may improve the modest adherence noted in several studies.26–29
The small number of studies identified in this review highlights the need for future studies aimed at improving child abuse care in the ED setting. While moderate-quality observational studies suggest that education and reminder systems increase clinical knowledge and documentation, these findings are not supported by a single randomized trial. More theory-based studies with rigorous designs and well-defined clinical scopes are needed to better evaluate the effect of professional and organizational interventions on clinical practice and patient and family outcomes.
The authors acknowledge the important contributions from Ms. Lisa Tjosvold (Alberta Research Centre for Health Evidence, University of Alberta) for conducting the initial literature search and Mses. Justina Tuli and Carla Farnesi (Department of Pediatrics, University of Alberta) for assisting with initial screening and data extraction, respectively.
- 1U.S. Department of Health and Human Services, Administration on Children, Youth and Families. Child Maltreatment 2006. Washington, DC: U.S. Government Printing Office, 2008.
- 2Canadian Incidence Study of Reported Child Abuse and Neglect-2003: Major Findings. Ottawa, ON: Minister of Public Works and Government Services Canada, 2005., , , et al.
- 3for the National Clearinghouse on Family Violence. Child Maltreatment in Canada: Overview Paper. Ottawa, ON: Public Health Agency of Canada, 2006., , , ,
- 21Evaluating non-randomised intervention studies. Health Technol Assess. 2003; 7:iii–x, 1–173., , , et al.
- 25Practical Statistics for Medical Research. London, UK: Chapman and Hall, 1991..
- 41Cognitive and social issues in emergency medicine knowledge translation: a research agenda. Acad Emerg Med. 2007; 97:984–90., , , , , .
- 51Looking inside the black box: a theory-based process evaluation alongside a randomised controlled trial of printed educational materials (the Ontario printed educational message, OPEM) to improve referral and prescribing practices in primary care in Ontario, Canada. Implement Sci. 2007; 2:38., , , et al.
- 53Evidence-based selection of theories for designing behavior change interventions: using methods based on theoretical construct domains to understand clinicians’ blood transfusion behavior. Br J Health Psychol. 2009; 14(Pt 4):625–46., , , et al.
- 55An intervention modeling experiment to change GPs’ intentions to implement evidence-based practice: using theory-based interventions to promote GP management of upper respiratory tract infection without prescribing antibiotics #2. BMC Health Serv Res. 2008; 8:10., , , et al.