Description of the condition
Substantial numbers of children worldwide are receiving out of home care. In the United States (US) in September 2010, 408,000 children were receiving out of home care (AFCARS data 2010); in England in March 2011, 65,520 children (DfE SFR 2011); in Scotland in July 2010, 15,892 children (National Statistics Publication for Scotland 2011); in Wales in March 2011, 5419 children (National Statistics 2011); in Australia in June 2011, 37,648 children (AIHW 2012) and in Norway in December 2009, 6603 children and young people (from birth to17 years of age) (Statistics Norway 2011). Most children are in out of home care programmes as a result of substantiated maltreatment, such as neglect, physical abuse, sexual abuse and/or emotional abuse. Other reasons for placement in out of home care include inability of the family to cope, parental illness, death and homelessness (AAP 2002; DfE SFR 2011; National Statistics 2011). Children in out of home care may be in foster care placements, kinship placements or other alternative care placements such as residential care homes. Worldwide, studies have consistently shown that children in out of home care have increased rates of physical, developmental, mental health and behavioural needs compared with their peers. The reasons for this are multi-layered and include increased likelihood of exposure to child abuse and neglect, disadvantage and poverty, poor antenatal care and poor parental health, especially mental health problems, and substance misuse (AAP 2002; RACP 2006).
Physical problems experienced by children in out of home care include acute and chronic health problems, impaired vision and hearing, poor oral and dental health, and lower rates of immunisation uptake. Ninety per cent of North American children in out of home care were found to have an abnormality in one or more body system; 15% failed a hearing screening test, 25% did not pass a vision screen, and the children were both lighter and shorter than their peers (Chernoff 1994). During initial placement, a North American study found that 27% of children had an upper respiratory illness, 21% had skin conditions, 12% had poor oral and dental health, and approximately one-third failed a vision screen (Takayama 1998). A longitudinal study from the United Kingdom (UK) of children in out of home care found that 52% had a health problem that required outpatient treatment and 26% had more than one problem requiring treatment (Skuse 2001). In this study, it was estimated that 15% of children were likely to have required treatment from a specialist (Skuse 2001). An Australian study in an out of home care clinic situated in a tertiary paediatric hospital found that only 3% of children who were assessed had no health problems, 30% failed a vision screen, and 28% failed a hearing test. Concern arose regarding abnormal growth and infection for 14% and 12%, respectively, and 24% of children had incomplete immunisation records (Nathanson 2007).
High rates of developmental, behavioural, and emotional problems are seen in children in out of home care. In the US, more than one-fifth of children younger than five years of age failed a developmental screening test (Chernoff 1994;Takayama 1998), and mental health screening of children older than three years of age found that almost 15% had suicidal ideation, 7% had homicidal ideation, and 36% had a history of behavioural problems (Chernoff 1994). In Australia, 60% failed a developmental screening test and 54% were identified as having behavioural problems (Nathanson 2007). Another Australian study that examined the mental health of children in court-ordered out of home care placements, using two standardised carer-report questionnaires, found that the children had high levels of poor mental health and socialisation: The range and severity of problems were reported to be similar to those of US clinic-referred groups (Tarren-Sweeney 2006). Difficulties included high rates of social problems, distorted thought patterns, attention difficulties, rule breaking or delinquent behaviour, and aggression. Roughly one-third were found to have some degree of age-inappropriate sexual behaviour, and many were reported to show behaviour suggestive of insecure relationships (Tarren-Sweeney 2006. Fifty-four per cent of Aboriginal children in out of home care in Australia were found to have a speech and language delay, and one-third had fine motor skill delay; 64% of school-aged children were found to have educational problems (Raman 2007).
Description of the intervention
It is recognised that children in out of home care have increased rates of physical, developmental, mental health, and behavioural needs compared with their peers; therefore clinical practice policy guidelines have been introduced in some countries to assist with the management of these needs. These guidelines have emphasised the need for comprehensive paediatric assessments to actively screen for problems associated with this group (AAP 2002; RACP 2006; DoH 2009). Before guidelines were introduced, 'usual care' for children in out of home care meant that health assessments would occur only when specific problems were suspected, and these assessments may not have been comprehensive but might have focused on a specific health issue instead.
Current clinical practice policy guidelines from the US, the UK, and Australia indicate that all children in out of home care should receive a comprehensive paediatric assessment upon entering care (within 30 days) and should undergo follow-up assessments (AAP 2002; RACP 2006; DoH 2009). Assessments should be comprehensive, and should be performed by staff experienced in meeting needs and managing care for children living in out of home care. They should include a careful history, during which available information on the child is collated: medical assessment of the child's physical health and growth, developmental and mental health screening, the formulation of a healthcare plan, and identification of a healthcare co-ordinator. In some countries, registration in a national scheme is a prerequisite for accessing public health services, including services provided for children in public care.
No standardised or universally accepted model has been used to deliver comprehensive paediatric assessments for this vulnerable group of children. Published data are based on differing assessment models, for example, multi-disciplinary assessment in a designated community-based foster care clinic setting (McCue Horwitz 2000), multi-disciplinary assessment at a health screening clinic for children in out of home care at a tertiary paediatric hospital Child Protection Unit (Nathanson 2007), and multi-disciplinary assessment in a paediatric outpatient department setting at a university hospital (Blatt 1997). Other ways in which models may differ include the choice of staff members who perform the assessments (e.g. individual paediatrician, nurse practitioner, multi-disciplinary team); the assessment location (e.g. hospital-based service, community health setting, foster agency setting) and the clinic structure itself (e.g. specialist out of home care clinics (seamless care) compared with standard outpatient clinics or private rooms). Assessment facilities may differ in the degree to which they can provide follow-up and case management.
How the intervention might work
The American Academy of Pediatrics has reported that the healthcare of children in out of home care is often compromised, and most do not receive a comprehensive developmental or psychological assessment at any time during their placement. Factors responsible for this include inadequate funding, poor planning, lack of access, prolonged waits for services, poor service co-ordination, and poor communication (AAP 2002). Barriers to adequate intervention for children in out of home care also include lack of permanency plans for these children, multiple placements with resultant changes in key people involved in the care of the child (e.g. carer, caseworker, local doctor, school teacher), and inadequate transfer of information when the placement changes; often there is ambiguity as to who is responsible for co-ordinating healthcare services for the child (AAP 2002; RACP 2006).
Children in out of home care are likely to benefit from comprehensive paediatric assessments because these assessments identify problems and areas of need for a given child (which otherwise may not have been noted), facilitate care through appropriate referrals (Hill 2003), and identify a healthcare co-ordinator for the child (RACP 2006). Although different models of comprehensive paediatric assessment have been proposed, it is unclear which models are most effective. Studies in the US have shown that uptake of comprehensive paediatric assessments for children in out of home care has been poor, and that although assessments have been helpful in identifying health problems and needs, little action is being taken on the basis of recommendations that have been put forth (AAP 2002).
Why it is important to do this review
To date, no systematic review has examined the effectiveness of comprehensive paediatric assessments for children entering out of home care in identifying physical health, developmental, mental and behavioural problems, and facilitating access to appropriate interventions.
No standardised model is available for the delivery of comprehensive paediatric assessments for children entering out of home care, and the published data reflect this. Providing comprehensive paediatric assessments is a complex intervention, and it is very unlikely that it has been studied in the format of a randomised trial. Therefore it is important for the authors of this systematic review to look at both randomised and non-randomised study design types when assessing the effectiveness of the intervention, and to systematically report the findings and limitations of available study design types.
To assess the effectiveness of comprehensive paediatric assessments in identifying physical, developmental, mental health, and behavioural needs, and in facilitating access to appropriate interventions compared with usual care (partial assessment) or no assessment for children entering out of home care.
Criteria for considering studies for this review
Types of studies
Randomised controlled trials (RCTs), quasi-randomised controlled trials, non-randomised controlled trials (NRCTs), and prospective cohort studies will be included.
Types of participants
Children from birth to 18 years of age who have entered out of home care. We will exclude children who have undergone a previous comprehensive paediatric assessment and children who are not in a statutory out of home care placement, for example, children in an informally arranged voluntary placement.
Types of interventions
Studies comparing children entering out of home care who receive a 'comprehensive paediatric assessment' with children who receive usual care or no assessment or who are on the waiting list for assessment. For this review, a 'comprehensive paediatric assessment' must include the following components.
- A general health assessment.
- Developmental screening (history and/or standardised tool).
- Mental health screening (history and/or standardised tool).
Comprehensive paediatric assessments can be offered through different models of care, including the following.
- Multi-disciplinary team assessment.
- Specialised clinics for assessment of children in out of home care.
- Assessment with and without case management.
- Assessment by a clinician who has received specific training for the role.
Children receiving usual care will include those who undergo partial or non-comprehensive assessment, for example, those who have one or two of the components mentioned above.
Types of outcome measures
Health needs identified include the following.
- Physical health needs.
- Developmental needs.
- Mental health needs (including attachment problems in infants and toddlers).
Pathways to care consist of the following.
- Formulation of a healthcare plan for the child after assessment.
- Potential harm associated with the assessment, including identification of problems but no access to service.
Health needs identified include the following.
- Dental health needs.
- Immunisation status (up-to-date, catch-up required).
Pathways to care consist of the following:
- Public health service access.
- 'Timely' access to services and therapies (therapies and services commence within three months of referral).
- Ongoing engagement with services at follow-up (e.g. at 6- and 12-month follow-up).
- Carer satisfaction with the assessment.
- Assessment done but not acted upon at follow-up (e.g. at 6- and 12-month follow-up).
Cost of service
We will report the following outcomes in a 'Summary of findings' table: identification of physical health, developmental and mental health needs (including attachment problems in infants and toddlers); potential harm associated with the assessment, including identification of problems but no access to services; 'timely' access to services and therapies; and ongoing engagement with services at follow-up.
Search methods for identification of studies
We will search the following databases. No date or language restrictions will be applied, and relevant articles will be translated as required.
- Cochrane Central Register of Controlled Trials (CENTRAL), part of The Cocrane Library
- Ovid MEDLINE
- Social Policy and Practice (Ovid)
- Social Science Citation Index (Web of Science)
- Science Citation Index (Web of Science)
- Conference Proceedings Citation Index Science
- Conference Proceedings Citation Index Social Sciences & Humanities
- ProQuest Dissertations & Theses: UK and Ireland
- WorldCat (limited to theses and dissertations)
- ClinicalTrials.gov (clinicaltrials.gov/)
- International Clinical Trials Registry Platform (www.who.int/ictrp/en/)
- metaRegister of Controlled Trials (www.controlled-trials.com/)
The search strategy for Ovid MEDLINE is shown below and will be modified as required for other databases.
1 Residential facilities/
2 Group homes/
4 Child, Institutionalized/
5 Child custody/
6 Foster Home Care/
7 ((kin or kinship) adj3 (care$ or caring or foster$ or placement$)).tw.
8 ((family or families) adj3 (placement$ or substitut$)).tw.
9 (out of home or away from home or looked after).tw.
10 ((resident$ or institut$ or group$ or congregate$) adj3 (care or setting$ or program$)).tw.
11 (foster$ adj3 (care$ or famil$ or placement$ or program$)).tw.
12 (foster$ adj3 (child$ or teen$ or adolescen$ or youth$ or young people)).tw.
13 (local authorit$ adj3 care).tw.
14 ("in care" adj3 (child$ or teen$ or adolescen$ or youth$ or young people)).tw.
15 (care system adj3 (child$ or teen$ or adolescen$ or youth$ or young people)).tw.
16 ((alternat$ or substitut$) adj3 care$ adj3 (child$ or teen$ or adolescen$ or youth$ or young people)).tw.
18 child health services/
19 adolescent health services/
20 community health services/
21 mental health services/
22 Health Services Accessibility/
23 diagnostic tests, routine/
24 diagnostic services/
25 Comprehensive Health Care/
26 Child welfare/
27 health status/
28 Health Status Indicators/
29 "Health Services Needs and Demand"/
30 needs assessment/
31 psychological tests/
32 Physical Examination/
33 medical history taking/
34 ((comprehensiv$ or p?ediatric or multi-disciplin$ or inter-disciplin$ or multidisciplin$ or interdisciplin$) adj1 (assess$ or screen$ or test$)).tw. (6450)
35 (battery adj1 (assess$ or test$)).tw.
36 (need? assessment or assessment of need?).tw.
37 ((health or mental or psycholog$ or psychiatric$ or developmental or physical) adj3 (assess$ or evaluat$ or diagnos$ or monitor$ or screen$ or examin$ or measur$ or test$)).tw.
39 exp infant/
40 exp child/
42 (baby or babies or infant$ or child$ or pre-school$ or preschool$ or teen$ or adolescen$ or young people or youth$).tw.
44 17 and 38 and 43
Searching other resources
We will search the websites of child protection and paediatric organisations to identify potentially relevant studies. These will include Child Family Community Australia (which incorporates the National Child Protection Clearinghouse), NAPCAN, and the CORE INFO: Cardiff Child Protection Systematic Reviews. We will search the reference lists of relevant studies and we will contact study authors for information about other potentially relevant published or unpublished work.
Data collection and analysis
Selection of studies
Two review authors will independently read titles and abstracts of studies identified by the literature searches to determine their relevance to this review. Studies will be obtained for further consideration if either review author considers it appropriate, or if its relevance is unclear. A study will be included in the review if both review authors agree that it meets the inclusion criteria. As labels used by researchers to describe non-randomised study design types may be interpreted and applied variably, the two review authors will use Table 13.2a and Table 13.2b in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011) to determine whether a study meets the inclusion criteria. Any disagreements will first be resolved by discussion; when this is not possible, the study will be considered by a third review author.
Data extraction and management
Two review authors will independently extract data from all identified relevant studies using a pre-designed data extraction sheet. Extracted data will include information regarding: study design; participant characteristics; type of out of home care placement, and duration of time in care and current placement; details of the paediatric assessment setting, assessment process, team and tools used; outcome measures and description of 'usual care' for comparison groups. Extracted data from both review authors will be checked, and any discrepancy will first be resolved by discussion. If this is not possible, discrepancies will be resolved by a third review author. When data are missing, all reasonable attempts will be made to contact the relevant authors to retrieve it.
Assessment of risk of bias in included studies
This review will consider both randomised and non-randomised study design types, and two review authors will independently assess all identified relevant studies. Given that different non-randomised study types vary in their susceptibility to different biases, several tools for assessing risk of bias will be utilised in this review, as per the guidance provided in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).
The Cochrane Collaboration's tool for assessing risk of bias will be used for RCTs, quasi-randomised RCTs, NRCTs, and prospective cohort studies (Higgins 2011).
Table 8.5.a: The Cochrane Collaboration’s tool for assessing risk of bias
In addition, we will use the Newcastle-Ottawa Quality Assessment Scale to assess the risk of bias for prospective cohort studies (Wells 2013).
Measures of treatment effect
Outcome measures may be dichotomous, ordinal, and continuous. When measures are ordinal, for example, a satisfaction scale, we will convert these outcome measures to dichotomous data. We will clearly define where the cut-off points have been taken and we will state why these points have been chosen. For dichotomous data, we will calculate the 95% confidence intervals and risk ratios for each outcome in all relevant studies. For continuous data, when the means and standard deviations are known or can be calculated from available data, we will analyse outcomes using the mean difference (for identical outcomes measured by the same scale) and the standardised mean difference (for the same outcome measured by different scales). When standard deviations or standard errors are not available, data on P values, t values, and confidence intervals will be extracted, so that standard deviations and standard errors can be imputed. Dichotomous and continuous data will be handled separately unless it is clinically meaningful for continuous data to be converted to dichotomous data. Any data conversion will be clearly described.
Unit of analysis issues
For studies that have used cluster randomisation, we will contact authors if it is unclear whether appropriate controls (e.g. robust standardised errors, hierarchical linear models) have been used for clustering. When appropriate controls have not been used, we will request individual participant data and we will estimate the intraclass correlation coefficient. The data will be reanalysed using multi-level models that control for clustering. After this is done, effect sizes and standard errors will be meta-analysed in (Review Manager 2012) using the generic inverse method (Higgins 2011). If individual participant data are not available, we will seek statistical advice from the Cochrane Statistical Methods Group and external experts as to which methods used to control for clustering would best apply.
Dealing with missing data
We will contact study authors, whenever possible, to obtain further information regarding missing data and dropouts when this has not been clearly provided in the original study. We will clearly state the study authors' assumptions underpinning any methods that have been used to deal with missing data, for example, that the data are assumed to be missing at random, or that missing values were assumed to have a particular value such as a poor outcome. We will perform sensitivity analyses to assess how sensitive results are to reasonable changes in the assumptions made. In the Discussion section, we will address the potential impact of missing data on the findings of the review. If meta-analysis is possible using data obtained, then data on all participants in a given study will be used; when it is not possible to obtain this information, meta-analysis will not be performed.
Assessment of heterogeneity
Studies will be assessed on the basis of their methodological (e.g. study design) and clinical (e.g. participant age, type of assessment model used) heterogeneity. We will assess the extent of between-study differences and the consistency of results of any meta-analysis in three ways.
- By visual inspection of the forest plots.
- By performing the Chi
2test of heterogeneity (when a significance level of less than 0.1 will be interpreted as evidence of heterogeneity).
- By examining the I
Assessment of reporting biases
If we identify sufficient studies (i.e. 10 or more), we will assess funnel plots to explore the possibility of small study bias. If funnel plot asymmetry is found, we will consider different explanations such as poor study design, the effect of different study sizes, and publication bias.
If two or more studies of the same study type are found that are suitable for inclusion, and the studies measure the same construct, we will perform a meta-analysis of the results. We will not perform a meta-analysis of results that measure the same construct but are derived from different study types.
If high heterogeneity is found within meta-analyses of the same study design, we will explore this by using clinically relevant subgroup analyses, as described below. If differences in outcomes due to clinically important subgroup differences are found, we will not perform an overall meta-analysis. Instead, we will perform subgroup meta-analyses. If subgroup analyses provide no clarity as to the source of the high heterogeneity, we will present the forest plot without a meta-analysis.
We will perform both fixed-effect and random-effects analyses as part of a sensitivity analysis. If summary estimates do not substantially differ, we will present random-effects analysis results.
Subgroup analysis and investigation of heterogeneity
If data are sufficient, we will perform a subgroup analysis to assess effects by reason for out of home care (maltreatment vs other reason); types of abuse; ethnicity (indigenous vs non-indigenous); placement type (residential or group home vs foster family, and kinship vs non-kinship placement); types of abuse; and assessment models.
We will present a 'Summary of findings' table, and the quality of evidence will be assessed by using the GRADE system (GRADE 2004). The assumed risk for the 'Summary of findings' tables will come from the placebo group summary estimate.
If possible, we will perform sensitivity analyses to determine the influence of study design (e.g. NRCT vs prospective cohort study) on results. We will also perform sensitivity analyses to investigate the robustness of results to publication status (published vs unpublished studies), the risk of study bias (excluding ‘high-risk’ studies), statistical methods used (fixed-effect model vs random-effects model), and the effects of missing data.
The authors would like to acknowledge and thank Laura MacDonald and Geraldine Macdonald from the Cochrane Developmental, Psychosocial and Learning Problems Group for their feedback and assistance during the preparation of this protocol, and Margaret Anderson for her help with the search strategy.
Contributions of authors
All review authors contributed to the writing of this protocol. GG and SW will independently screen titles and abstracts from the literature search. Any potentially relevant studies will be obtained; GG and SW will review these studies against eligibility criteria, and will independently perform risk of bias assessments and extract data from included studies. GG will draft the review with input from all authors. All authors will read and edit the review before submission.
Declarations of interest
Glenys Griffiths - none known.
Andrew Hayen - none known.
Susan Woolfenden - none known.
Katrina Williams - none known.
Shanti Raman - none known.
Dimitra Tzioumi - none known.