Summary of findings
Description of the condition
The number of street-connected children and young people worldwide has been estimated at around 100 million (UNICEF 2002) although this figure is widely contested. It is recognised that exact numbers are unknown and estimates vary, due in part to political motivations (Thomas de Benitez 2011). Numbers differ depending on whether estimated by governments or non-government organisations (NGOs). The definition and status of the problem has traditionally differed for Europe and other high income countries, although some of the structural antecedents such as inequalities or social exclusion may be similar. For example, a minimum of 66,000 first-time runaways per year is recorded in England (CSC 2009), and Canada’s street-connected children and young people may be runaways who have escaped sexual or physical abuse. Data for the US estimate 1 to 2 million ‘street involved youth’. The difficulty in estimating numbers is in part due to wide variations in definitions of which young people are included and the lack of formal identity papers for most street-connected children and young people.
In the historic United Nations International Children's Emergency Fund (UNICEF) definition, ‘children of the street’ are homeless children who live and sleep on the streets in predominantly urban areas, living with other street-connected children and young people or homeless adults. ‘Children on the street’ earn their living or beg for money on the street and may return home at night and maintain contact with their families. Such definitions may include children who are stateless or migrating, with or without their families. The definition of ‘street-connected children and young people’ can also overlap with categories such as runaways and homeless youth, children who have been trafficked, child labourers, children who live in slums, and children living in institutions (Ennew 2003; UNICEF 2005). Many commentators argue that the issues prevalent in the lives of street-connected children, including the risks, do not differ for other children living in urban or rural poverty, and that approaches to the issue of street-connected children and young people should not be disconnected from approaches to ameliorate poverty and social exclusion more generally (Panter-Brick 2002; Thomas de Benitez 2011). This review, however, focuses on street-connected children.
Definitions too are much debated, with varying emphasis on young people’s agency and resilience (Beazley 2003; Van Blerk 2006). Agency is typically conceptualised as an element of young people's resilience, enabling street-connected children and youth, for example, to negotiate for their basic needs, draw on social support networks, and explore pathways to achieve their personal goals in a resourceful manner (for example Theron 2010). In an overview by the Consortium for Street Children (CSC), de Thomas Benitez states: "street children are recognized to be young people who experience a combination of multiple deprivations and street-connectedness" (Thomas de Benitez 2011b). Children and young people may live and work on the street or in public spaces, work on the street and return to family homes or hostels at night, or a combination of these at different time periods.
In our systematic review, the term ‘street-connected children and young people’ is used to refer to children who work or sleep, or both, on the streets and may or may not necessarily be adequately supervised or directed by responsible adults. It includes (but not exclusively) the coexisting categories referred to by UNICEF as those ‘on the street’ and those ‘of the street’, children for whom the street is a reference point and has a central role in their lives (Redes Rio Crianca 2007; Thomas de Benitez 2011; UNICEF 2001a). Current thinking sees this process as non-linear, with many street-connected children and young people transitioning off the streets (Panter-Brick 2002). This definition opens the door to studies of young people living in slums, squatter settlements or in hostels who are also working on the street.
Important risks faced by street-connected children and young people are physical, psychological and sexual exploitation, violence, economic exploitation, social exclusion, no skills-based employment, substance misuse, widespread addiction and HIV (Ennew 2000; West 2003). Many street-connected children and young people experience health difficulties, coercion and control by adult gangs, criminality, and lack of education (West 2003). However, street-connected children and young people are not a homogenous group. Current research demonstrates that girls and young women may experience risks differently to boys and young men (Beazley 2003; Van Blerk 2006). Other groups, such as disabled youth or those from minority ethnic groups, may also have different experiences. Children live and work on the streets in different ways and for different reasons (UNICEF 2005). Most street-connected children and young people are not orphaned but are in contact with their families and may augment the household income (UNICEF 2005). Current research also emphasises the resilience of street-connected children and young people and the fact of children and young people’s agency and citizenship, making their own decisions and with a need for participation not solely protection (Panter-Brick 2002; Thomas de Benitez 2011).
Description of the intervention
Interventions aiming to improve the situation of street-connected children and young people include educational projects (Ouma 2004), vocational training (Ali 2004), harm-reduction (Poland 2002), HIV prevention (Rotheram-Borus 2003) and family therapeutic programmes (Roberts 2010). They often take the form of single projects, drop-in centres or peer education interventions, and many of these projects will be underpinned by the ‘children’s rights’ discourse, more recently taking a holistic approach to the needs of the young people (Ennew 2000; Thomas de Benitez 2011). Indeed, it has been argued that some interventions may not succeed if they ignore children’s voices and do not include their participation in planning and management (Panter-Brick 2002).
Educational projects offer street-connected children opportunities to break out of the cycle of poverty. Occasionally these projects help children and youth to sit formal examinations and obtain recognised certificates (Ouma 2004), while vocational training aims to develop skills to lead children and youth into the world of non-exploitative work. Often these programmes aim, through health and nutrition programmes, to increase the 'educatability’ of children and youth before or while they are attending schools. They can also take the form of non-formal education, consisting of any form of systematic learning activity outside the framework of the formal system. Such provision may be run alongside formal schooling, or separately.
Several considerations are relevant to the intervention and programs with reference to the relevant population. So far, we have particularly identified gender, ethnicity, religion, disability, citizenship, legal status and age of the street-connected children and young people as relevant individual factors that may impact on outcomes of interventions. There are also relevant contextual factors, which include the experience of sexual abuse, violence, addiction, low literacy, migration (including rural-to-urban), poverty and mechanisms of exclusion (such as negative community responses to the children's migratory or refugee status, and labelling them as ‘vagrants’, ‘illegal vendors’ or ‘truants’).
It is also important to consider the nature of strategies for engaging young people at street level that, according to a wealth of qualitative literature drawing on ethnographic data and practitioner perspectives, form the basis of successful intervention programmes (Ennew 2000; Karabanow 2004; Panter-Brick 2002; Thomas de Benitez 2011). "To determine the ‘type’ of intervention needed, engagement enables a relationship and trust to be built. Participatory models of engagement ensure that sufficient time and space is given to children to demonstrate to outsiders why they came to the street, and what their background is. Participatory engagement allows children themselves to tell their histories rather than have to directly answer questions about their past" (Walker 2011 [pers comm]).
How the intervention might work
As Anderson 2010 argues, logic models offer a particularly useful tool in the analysis of complex interventions that operate at individual, group and social system levels. We found the logic model a useful tool in capturing on one hand the heterogeneity of intervention types, background variables and research contexts relevant to the review topic, and the core elements of successful interventions on the other. In our primary intervention logic model (Figure 1), we divide such intervention components (second column) into micro-, meso- and exo-level factors, drawing on Bronfenbrenner's analytical model (Bronfenbrenner 1979). These interact with factors relevant to recruitment and engagement (first column), again with features relevant at different levels of analysis, including macro-level factors such as culture and religion. The third column indicates potential intermediate outcomes at these four levels, followed by longer-term outcomes in the fourth column. Our generalisability logic model (Figure 2) provides a more concise model for assessing the generalisability of a particular intervention across socio-economic and cultural contexts.
|Figure 1. Intervention and context logic model|
|Figure 2. Generalisability logic model|
Some outcomes identified in the literature include negative effects of poorly planned or forced interventions (Thomas de Benitez 2011) and detrimental outcomes frequently documented in association with reintegration of children in non-family care into their families of origin (Thoburn 2009). A possible adverse outcome that may, however, not easily be captured in study evaluations is an increase in street-connected children and young people’s mistrust of adults in the context of interventions that may be ad hoc and short-lived due to lack of funding and other structural support. Some researchers consider that study designs that do not provide genuine opportunities for children and young people’s participation throughout the research process are most likely to fail in assessing the full range of outcomes of an intervention (Panter-Brick 2002; Slesnick 2009).
A final point to be made is that the circumstances of street-connected children and young people, as noted above, may be non-linear and young people may continue to live and work on the streets whilst engaging with interventions, and may take many years to reintegrate fully or become reincluded within mainstream society.
Why it is important to do this review
The rationale for this review is to assess the effectiveness of interventions for improving outcomes for street-connected children and young people, and reducing the risks of the most adverse outcomes; to promote access to and integration into education, training and employment opportunities and more healthy and settled lifestyles. Such lifestyles include access to universal human rights such as survival, development, participation and inclusion, although these may be difficult to measure.
By addressing the above-mentioned outcomes, we explicitly aimed to synthesise the evidence on reintegration approaches, including harm-reduction programmes. We propose to focus on inclusion, reintegration and harm-reduction interventions targeted at children and young people while they are living on, or closely connected to, the streets.
We used the World Health Organization (WHO) definition of inclusion. The primary aims of policies and action aimed at reversing exclusionary processes should be to:
- promote full and equal inclusion in social systems;
- provide universal access to living standards which are socially acceptable to all members of a society, including access to the same level and quality of health and educational services, safe water, sanitation and ‘decent work’, as defined by the International Labour Organization (ILO);
- respect and promote cultural diversity;
- address unequal inclusion as well as situations of extreme exclusion (WHO SEKN 2008).
We believe that the results of this systematic review are relevant to a large number of street-connected children and young people worldwide. The review examined interventions that enable children to live safe and healthy lives that promote their rights and support their pathways to adulthood. It also highlighted gaps in the current evidence base.
For the purposes of this review, we define reintegration as the children and young people entering a residential or educational environment that has the potential to provide them with elements of physical safety, medical care, nutrition, counselling, education, inclusion in social and economic opportunities, and room for recreation and personal and spiritual growth that may impact positively on longer term life chances. Reintegration does not mean returning the children to the situations from which they may have escaped. Family reintegration is potentially a highly valuable outcome for many street-connected children and young people. However, the effectiveness as well as the ethical implementation of interventions aimed at family reintegration are premised on access to appropriate resources for assessment, support and follow-up, in recognition of the potentially significant risks associated with processes of family reintegration (Thoburn 2009).
‘Harm-reduction’ is an umbrella term to describe the interventions aimed at reducing harms associated with lifestyles of street-connected children and young people including, for example, those associated with early or risky sexual activity and substance use (UNICEF 2001b). Expressed in general terms, these would be interventions aimed at street-connected children and young people, and aimed to protect and promote both their welfare and their well-being while they are on the street so that they are able to benefit from more focused reintegration approaches when it is appropriate and possible for them to do so. All the long term recommendations we found at the UNICEF evaluation database are structural. However, the short term recommendations by UNICEF are based on principles of child protection that can be described as matching the harm-reduction approach. This is open to interpretation but seems to be in line with the opinion of people working with street-connected children and young people who were consulted by members of our team; protection may be a necessary stage on the path to reintegration, alongside development and participation.
We identified few rigorous reviews on the effectiveness of interventions to support street-connected children and young people through a scoping search. Descriptive reviews of interventions which include literature on lower middle income and low income countries include Dybicz 2005; Karabanow 2004; Peters 2004; Slesnick 2009; and Thomas de Benitez 2011. Moore 2005 and Sanabria 2006 present descriptive reviews focusing exclusively on US-based interventions. While these reviews provide useful analyses and classifications of the literature, their search strategies are often poorly described or limited in scope. Furthermore, they do not contain rigorous evaluations of studies.
We identified one review which included interventions for ‘homeless youth’, described as systematic (Altena 2010), where studies were reported to have been systematically rated for study quality using a consistent tool. This review is recent and aimed to be inclusive of literature in developing countries (language criteria not specified). It searched the following databases: PsycINFO, ERIC, MEDLINE, The Cochrane Library, Google Scholar, EMBASE and CINAHL for studies conducted between 1985 and 2008. Out of 557 unique search results, 12 studies were included for final evaluation, none of which were conducted in low and middle income countries (LMICs). In comparison, the current systematic review was considerably broader in scope, both in terms of the number of databases searched and the breadth of our search terms. However, to avoid duplication our systematic review takes into account the existence of a Cochrane review on HIV/AIDS prevention with homeless youth (Naranbhai 2011), as discussed below.
Primary research objectives
To evaluate and summarise the effectiveness of interventions for street-connected children and young people that aim to:
- promote inclusion and reintegration;
- increase literacy and numeracy;
- increase access to education and employment;
- promote mental health, including self-esteem; and
- reduce harms associated with early sexual activity and substance misuse.
Furthermore, to explore what can be known about the processes of successful intervention and models of change in this area, and understand how intervention effectiveness may vary in different contexts.
Secondary research objectives
- To explore whether effects of the intervention differ within and between populations, and whether an equity gradient impacts on these effects including, and importantly, extrapolating from all findings relevance for LMICs (Peters 2004).
- To describe other health, educational, psycho-social, and behavioural effects, where appropriate outcomes are available.
- To explore the influence of context in the design, delivery, and outcomes of the interventions.
- To explore the relationship between the number of components, duration, and effects of the interventions.
- To highlight implications for further research and research methods to improve knowledge of the interventions in relation to the primary research objective.
This review also aimed to consider potential adverse or unintended outcomes.
Criteria for considering studies for this review
Types of studies
Interventions targeting (and measuring) outcomes for street-connected children and young people have used a variety of approaches and designs. We included randomised controlled trials (RCTs), clinical controlled trials (CCTs), controlled before-and-after trials (CBA) and quasi-randomised trials. Quasi-randomised trials refer to studies which allocate the children and young people to treatment or control conditions depending on methods determined as not truly randomised, for example, on their date of birth or the day of the month they enter the intervention site. Some other quasi-randomised designs, such as regression discontinuity designs, were eligible for inclusion in the review.
We did not include qualitative data in our outcomes synthesis. However, we used qualitative intervention evaluations in order to design the original logic model and continued developing the logic model with the help of qualitative data and the identified included studies in the progress of the review. We also sought qualitative data, including sibling or companion studies of included quantitative studies, to illuminate the impact of context and also mechanisms of change and any process factors. We did not conduct separate searches for qualitative literature other than for companion studies of included studies and those needed to highlight any particular questions arising in relation to context, mechanisms, and process, etc., according to themes outlined in the logic models. Some such materials were sought from studies retrieved in the search not included in the review, and these were used to discuss process and contextual factors as well as issues of generalisability of findings to LMIC contexts.
For this review, the included studies required a comparator, either groups who did not receive an intervention, who received standard practice interventions, or who received a different type of intervention.
Types of participants
We included all studies focusing on street-connected children and young people between the ages of 0 and 24 years (inclusive), consistent with the United Nations (UN) definition of youth as including those aged 15 to 24 years regardless of location, reason for street connectedness or gender. Potential research participants included: street-connected children and young people, their families and carers, professionals working with children, young people and their families, the police and employers.
Street-connected children and young people, and in the cases of family focused interventions their families and carers were the intervention recipients. We did not include any studies that did not report separate outcomes data on street-connected children and young people in the context of systemic interventions.
Families and carers, the community, employers and professionals can be an important part of the ‘input’ component of the intervention to the extent that they are needed to support the intervention and are part of it. Our definition of professionals and community included non-government organisations (NGOs), faith-based organisations, orphanages, social workers and police.
For the purposes of this review we defined street-connected children as in the Description of the intervention above: children and young people may live and work on the street or in public spaces, work on the street and return to family homes or hostels at night, or a combination of these at different time periods. For the most part, they experience complex social and economic circumstances that ‘defy easy definition’ (Thomas de Benitez 2011). Current thinking sees this process as non-linear, with many street-connected children and young people transitioning off the streets, more than once, with this also a non-linear process.
Types of interventions
The intention was to include any interventions that:
- involved harm-reduction, inclusion or reintegration programmes for street-connected children and young people, were intended to reduce harms associated with risky sexual activity and substance misuse, and promoted inclusion and reintegration;
- increased literacy, numeracy and self-esteem;
- increased participation in education and skills-based employment;
- provided shelter, housing and drop-in support.
We planned to include any type of intervention including behavioural, social, policy, structural or other interventions explicitly aimed at reducing risky sexual activity and substance misuse. Interventions may be delivered to individuals, families, small groups or entire communities. Furthermore, recognising the complexity of the issues facing many street-connected young people, there has been a developing focus on multifaceted interventions that incorporate a range of approaches including housing, education, training and health (Thomas de Benitez 2008).
Types of outcome measures
Since a recent Cochrane review and a systematic review conducted for the WHO have evaluated AIDS and HIV as target outcomes (Naranbhai 2011; Ross 2006) we did not include AIDS and HIV risks as outcome variables. However, we assessed the degree to which the included studies of these reviews overlapped with our scope and population and, where relevant, considered the trends in the results of these reviews when interpreting the results of our review.
The primary outcomes were inclusion and reintegration. We define reintegration as the children and young people entering a residential and/or educational environment that has the potential to provide them with elements of physical safety, medical care, nutrition, counselling, education, inclusion in social and economic opportunities, and room for recreation and personal and spiritual growth that may impact positively on longer term life chances. According to this definition, reintegration does not mean returning children to situations from which they may have escaped, though may include family reintegration.
We also extracted the following analysable data of other related measures of health, well-being, and educational and occupational achievement.
- Safer or reduced sexual activity.
- Safer or reduced substance use (e.g. reduced sharing of injecting equipment).
- Increased use of hostel or shelter type services.
- Participation in education.
- Participation in skills-based (rather than exploitative) employment.
- Reduced use of violence.
- Increased contact with family.
- Participation in intervention planning and delivery.
We included intervention studies if they aimed to achieve any one of the listed primary or secondary outcomes, or both. Secondary objectives were found to be particularly relevant as most interventions were administered within an existing service setting.
We extracted measures relating to the process of implementing an intervention and intervention approaches, where reported. We also extracted information consistent with the process characteristics listed in the original logic model in order to develop an explanatory framework.
We have included a descriptive map of all studies considered for eligibility for inclusion in the review, in order to present as fully as possible a description of the existing evidence base on this topic. This is included as an adjunctive to the main review in the interests of completeness of data, rather than being used as a tool for narrowing the review focus (Appendix 1).
Search methods for identification of studies
We have worked with information specialists from Campbell’s International Development Co-ordinating Group and the Cochrane Musculoskeletal Group, which is co-located with the Cochrane Campbell Equity Methods Group and informed by their search expertise, to develop a search strategy. We used guidance from Chapter 6 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011) and methods from the Campbell Collaboration's Information Retrieval Methods Group’s guide to information retrieval for systematic reviews (Hammerstrøm 2010). No language restriction was applied.
The search was developed in Ovid MEDLINE and modified for use in other databases (see Appendix 2: Search results summary and search strategies). We identified 29,151 items from all the relevant databases (See Appendix 2). All references were imported into RefWorks and tagged with the name of the database. Duplicates were removed within RefWorks, leaving the final total of results from the electronic databases at 15,995. See the PRISMA diagram in Figure 3.
|Figure 3. Study flow diagram.|
We searched the following bibliographic databases for eligible empirical studies:
Cochrane Central Register of Controlled Trials (CENTRAL) (database inception to search date);
MEDLINE and PreMEDLINE (1948 to search date);
EMBASE and EMBASE Classic (1947 to search date);
CINAHL (1966 to search date);
PsycINFO (1806 to search date);
ERIC (1950 to search date);
Sociological Abstracts (1952 to search date);
Social Services Abstracts (1979 to search date);
Social Work Abstracts (1977 to search date);
Healthstar (1966 to search date);
LILACS (database inception to search date);
System for Grey literature in Europe (OpenGrey) (database inception to search date);
ProQuest Dissertations and Theses (database inception to search date);
EconLit (1969 to search date);
IDEAS Economics and Finance Research (database inception to search date);
JOLIS Library Catalog of the holdings of the World Bank Group and IMF Libraries (database inception to search date);
BLDS (British Library for Development Studies) (1987 to search date);
Google, Google Scholar.
Searching other resources
We screened items suggested by experts, advisory group members, and authors of included studies, including companion studies. We also checked reference lists of included studies from the electronic database search and contacted all authors of included studies to ask about unpublished or ongoing studies. We used search terms from the electronic search which described our population, and adapted them as appropriate to search the Internet-based resources. We used included studies to perform a citing studies search using SCOPUS or Web of Science and PubMed’s related article function to track references to the included articles, relevant reviews and annotated bibliographies.
We conducted a targeted Internet search on the following relevant sites:
§ J-PAL website
§ UNICEF database of evaluations
§ Eldis http://www.eldis.org/
§ Department for International Development http://www.dfid.gov.uk/
§ Inter-American Development Bank http://www.iadb.org
§ Asian Development Bank http://www.adb.org
§ African Development Bank http://www.afdb.org
Data collection and analysis
Selection of studies
The results of the search were all screened in EROS software according to the following categories: Effectiveness study: probability of inclusion; Evaluation study with other study designs; Ethnography or other qualitative studies; Excluded: related to street children but not evaluating effectiveness; Narrative review; Excluded: not related with street children; Non-English language studies (which were assessed for inclusion separately). The majority of the studies were excluded for clearly not meeting eligibility criteria for the review. Fifty- seven of the total number of studies screened in EROS were assessed as potentially eligible (the first category) by at least one review author and their full-text articles (where available) screened by at least two review authors according to the criteria specified in the protocol. Of these, 10 studies were included in the review.
Additionally, a total of 50 out of the original 15,995 records were classified by the review authors as narrative reviews. Full-text documents for these were obtained and scanned for relevant references by two review authors. A total of 108 references were identified as potentially eligible through this process. After comparison with the existing database, 40 records were reviewed by full text. One of these was included in the review.
The PRISMA flowchart displays this process visually (Figure 3).
The full text for the majority of MA and PhD theses could not be obtained, and these were screened by abstract only.
We also undertook a separate search of the databases specified in the review for qualitative and quantitative studies that were associated with the 11 studies included in the review. The search strategy consisted of the following.
i) A search for qualitative studies on solely subject (street children) and topic terms (evaluation of interventions) without specifying the study type(s) of interest.
ii) Looking for qualitative or quantitative studies by authors and co-authors of included studies on ‘street children’ to find directly related studies e.g. same study but measuring different outcomes, follow-up and on-going studies.
Data extraction and management
We included all studies considered eligible for the review. Two independent review authors (two of RH, HH, AM, MV) extracted the data from eligible studies on to standardised data collection forms and these were entered in Review Manager 5 using double-data entry (RevMan 2011). We tailored the data extraction to the requirements of the review, using the PROGRESS II checklist as developed by the Cochrane-Campbell Equity Methods Group (Kavanagh 2008), working to the logic model. We assembled and compared multiple reports and publications of the same study for completeness and possible contradictions. No companion studies were found that reported findings on the process evaluation of the intervention. Three review authors piloted the data extraction form to assess its ability to capture study data and inform assessment of study quality. We resolved any identified problems through discussion and revised the form accordingly.
Assessment of risk of bias in included studies
Two review authors (RH, HH, MV, AM) assessed the risk of bias using the retrieved reports of the studies and raising additional queries with authors where further information was required. Any disagreements between review authors’ risk of bias assessments were resolved by discussion. We assessed the risk of selection, performance, attrition and detection bias. We evaluated and rated as ‘high’, ‘low’ or ‘unclear’ the sequence generation; allocation concealment; blinding of participants, personnel and outcomes; incomplete outcome data; selective outcome reporting; and other sources of bias.
Measures of treatment effect
No dichotomous outcomes were reported in the included studies. We used the weighted mean difference (WMD) between the post-test values of the intervention and control groups to analyse the size of intervention effects for continuous outcomes. For outcomes measured on different scales, we used the standardised mean difference (SMD).
Where possible, we reported continuous outcomes on the original scale. We standardised outcomes measured on different scales as required for the analysis. We only conducted a meta-analysis where the data were sufficiently similar. Where data were available, sufficiently similar in outcomes and time points and of sufficient quality, we performed statistical analyses using Review Manager 5 software (RevMan 2011). We did not combine evidence from differing study designs and outcome types in the same forest plot.
Unit of analysis issues
In order to avoid double-counting where studies presented results for several periods of follow-up, we undertook separate meta-analyses for the various time points: immediate post-test, six month follow-up and 12 month follow-up. Where a study presented data from a different time point to the other studies, we presented these data separately.
Where multiple treatment and control group types were presented in study reports, we aimed to present the data from each study as consistently as possible with the primary comparison of treatment compared with the control group. Where a study compared two interventions against one control group, the control group number was halved for each comparison to avoid double-counting of participants. No eligible cluster designs were found in the searches for this review.
Dealing with missing data
Due to the fluctuating nature of attendance at likely programmes, we did not exclude studies according to degree of incomplete data for assessment. We incorporated this both narratively and in the risk of bias assessment. At data extraction stage, if missing data were unclear or were not fully reported, we contacted the authors. In general, we reported the occurrence of missing data both in the data extraction form and in the risk of bias table, while the data extraction form also captured where missing data were retrieved.
Assessment of heterogeneity
Heterogenety was mainly assessed through extensive knowledge of the characteristics of the included studies. Statistical heterogeneity was assessed using the I
Assessment of reporting biases
We have narratively addressed the imbalance within the included studies in both the conduct of evaluations and publication of reports between high income and low and middle income countries. Insufficient studies were found for a funnel plot to be useful so this was not performed.
Our study selection included RCTs as well as controlled before-and-after (CBA) studies and other non-randomised designs that included a control or comparison group (but not those with a convenience comparison group; all control groups were randomised or propensity scores were used to balance baseline differences). Risk of bias was assessed using the Cochrane risk of bias tool. Confounding aspects of populations, interventions or settings are discussed in the discussion section of the review.
Where possible, we reported continuous outcomes on the original scale. We standardised outcomes measured on different scales, as required for the analysis. No binary data were included in the review.
Where data were available, and outcomes were measured in similar ways and of sufficient quality, we performed statistical analyses using Review Manager 5 software (RevMan 2011) and a random-effects model. We did not combine evidence from differing study designs and outcome types in the same forest plot. Similarity of data was assessed according to types of outcomes measured and at which time points. For this review we included all interventions in the same meta-analyses.
We analysed the data from all studies, including those not included in the meta-analysis, according to features of the logic model, extracted through the data extraction process. We grouped data according to the outcomes of the interventions as well as discussing contexts, particularly regarding income status and cultural environment of the different countries included in the review. We further considered groupings around age, gender, ethnicity and, where possible, the reasons for children and young people being street-connected (for example migration status, economic activity, history of abuse).
We made a decision to include all endpoint data up to six months with data from more than one study across outcomes. Most frequently, these data were collected at three and six months from the start of the intervention. Longer term follow-up data, measured across the studies at the different time points of 9, 12, or 15 months following the intervention, were reported narratively as there was an insufficient number of studies to include in the meta-analysis.
Where two intervention groups were compared against one control group, we halved the number of participants in the control group for each outcome.
Where the same scale was used, we performed a random-effects model analysis of mean difference (MD). Where different scales were used, the effect size was based on a random effects model analysis of standardised mean difference (SMD). Details of included outcomes, including measures used and time points measured, are summarised in Table 1.
In addition, we performed a change from baseline calculation for each included outcome at each included time point by subtracting the group mean at follow-up from the group mean at baseline for the intervention and control groups. All change scores reported in the review (Appendix 3) were calculated by the review authors rather than being calculated by the study authors. These figures should be interpreted with caution since they do not account for standard deviation.
Additionally, although the majority of studies presented relevant outcome data for the same number of participants at each time point, including baseline (that is missing cases were excluded from the analyses), in some studies the numbers varied betwen time points. Our change scores do not account for these discrepancies. Nevertheless, we believe they give a reasonable indication of certain important effects not captured by comparisons of means and standard deviations at fixed follow-up time points. In particular, they help to demonstrate that in many cases the scores for both groups improved from baseline, which may appear as no difference between the groups in a meta-analysis of endpoint data, or indeed a benefit to the control group.
Outcomes not included in meta-analysis
A number of outcomes reported in the included studies were not included in meta-analysis due to differences in type of measurement, time point or inability to access the data, and they are listed under outcomes in the results section of this review and in Appendix 4. Where possible, we have added these outcomes to the narrative report of the outcome data included in the review to enable better cross referencing of more synthesised results across studies.
Subgroup analysis and investigation of heterogeneity
We hoped to include subgroups, analysing by age, gender, location of studies, high and low and middle income countries, and intervention approaches, to inform logic models and the development of possible theories arising from the review. However, insufficient data were included to make this possible.
Insufficient data were included in the meta-analysis to make sensitivity analysis possible.
Description of studies
Results of the search
Our search yielded a total of 29,151 records, leaving 15,995 records after removal of duplicates. Search results are included at Appendix 2.
Fifty-seven studies were independently classified by at least one review author in the first classification category in EROS ('probability of inclusion'), according to the categories described in Selection of studies. These were reviewed by full text, where possible, and 10 studies (11 references) were included in the review. A further 108 records were identified from 68 records classified in EROS as narrative reviews. After removing duplicates, screening by abstract and checking against the search database, the full text was sought for 40 of these. Of these, one study was included in the review and one was included as an ongoing study (see Ongoing studies). No relevant further references for ongoing studies were obtained from the authors contacted. A further 230 non-English language records were reviewed, and the full text sought for nine. None of these were eligible for inclusion in the review (see also PRISMA flow diagram at Figure 3).
All of the records reviewed by full text were also considered for eligibility for a descriptive map (Appendix 1), with selection criteria allowing for a broader range of study designs. Unlike the review, this mapping included a number of studies from LMICs. Out of 60 references considered potentially eligible, 30 studies were included in the mapping exercise (this figure includes the 11 studies included in the review) and 30 excluded from it. Out of the 60, 48 references were excluded from this review (see Characteristics of excluded studies). In effect, the descriptive map described in more detail the included and excluded studies in this review.
In total, 11 studies (12 papers; 12 interventions) were included in the review. Data from eight studies were available for meta-analysis and findings from the three remaining studies have been described narratively.
The search for companion studies identified two relevant publications (Slesnick 2006; Slesnick 2006c). Both papers present post hoc quantitative analyses of combinations of data from Slesnick 2005; and Slesnick 2009 EBFT; Slesnick 2009 FFT.
Eleven studies (12 publications) met the inclusion criteria of the review (Baer 2007; Cauce 1994; Hyun 2005; Milburn 2012; Peterson 2006; Rew 2007; Rotheram-Borus 2003; Slesnick 2005; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT; Tischler 2002). Of these, eight studies were classed as RCTs; two as CBAs (Rotheram-Borus 2003; Tischler 2002) and one as a quasi-RCT (Rew 2007). There was a gap of eight years between the first two published studies. The remaining studies have been published at relatively even intervals between 2002 and 2012, with the highest number of publications (three) in 2007.
All first authors were successfully contacted. However, authors for the Rotheram-Borus 2003 and Rew 2007 studies were unable to provide relevant raw data on the outcomes measured. The outcomes measured in Tischler 2002 (primarily children's SDQ scores) were unique to their study, and were reported differently from other studies (that is as a change from baseline score as opposed to mean and standard deviation scores). Therefore, data from these three studies were not included in our analysis but relevant findings, as presented in these three papers, were referred to in our discussion. Also to note is the fact that three of the included studies were conducted by research teams headed by Prof N Slesnick (Slesnick 2005; Slesnick 2007/08; and Slesnick 2009 EBFT; Slesnick 2009 FFT). Since one of these (Slesnick 2009 EBFT; Slesnick 2009 FFT) involved two separate intervention groups, four of the twelve intervention groups included in the analysis were from studies directed by Slesnick.
All the studies were conducted in the US with the exception of Tischler 2002, which was undertaken in the UK, and Hyun 2005, which was a Korean study. We were unable to identify any studies in LMICs meeting our inclusion criteria. All the Slesnick interventions were located in the same city (Albuquerque, New Mexico) (Slesnick 2005; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT). Other US studies were located in Seattle, Washington (Cauce 1994; Peterson 2006), Los Angeles and San Bernardino, California (Milburn 2012), Texas (Rew 2007) and New York (Rotheram-Borus 2003). No location was specified in Baer 2007, but similar to Peterson 2006 the study authors (who included the first author of Peterson 2006) were based at the University of Washington. Tischler 2002 was conducted in Birmingham, UK and Hyun 2005 in Seoul, Korea.
Interventions consisted of individual-oriented (Baer 2007; Cauce 1994; Peterson 2006; Rew 2007; Slesnick 2007/08; Tischler 2002), group-based (Hyun 2005; Rotheram-Borus 2003) and family-based (Milburn 2012; Slesnick 2005; and Slesnick 2009 EBFT; Slesnick 2009 FFT) approaches. Baer 2007 and Peterson 2006 adopted a motivational framework; Slesnick 2007/08 a community reinforcement framework (incorporating behavioural, motivational and systemic approaches) in combination with a cognitive-behavioural HIV prevention intervention; Hyun 2005 a CBT framework; Cauce 1994 and Tischler 2002 a multicomponent case management framework including individual therapy sessions; Slesnick 2005, Slesnick 2009 EBFT, Slesnick 2009 FFT and Milburn 2012 different forms of behavioural family intervention frameworks (ecologically-based family therapy (EBFT), functional family therapy (FFT), and a cognitive-behavioural family intervention, respectively); Rew 2007 a social cognitive framework; and Rotheram-Borus 2003 a social cognitive, multicomponent framework. In total, five interventions (Cauce 1994; Rotheram-Borus 2003; Tischler 2002; and the EBFT intervention in Slesnick 2005 and Slesnick 2009 EBFT) were multicomponent and involved liaison with external service providers. Intervention length ranged from a single brief session to ‘on-going’. In many cases, considerable variation between participants was noted for treatment attendance or duration of the intervention, or both.
The majority of the studies recruited participants through a shelter (Hyun 2005; Slesnick 2005; Slesnick 2009 EBFT; Slesnick 2009 FFT), drop-in service (Baer 2007; Cauce 1994; Slesnick 2007/08) or hostel (Tischler 2002). Three studies employed multiple strategies in order to engage a more representative population of street-connected children and youth: Milburn 2012 recruited newly homeless youth from community-based organisations (for example shelters and schools) as well as through direct recruitment (for example by flyers); Rew 2007 recruited participants via a street outreach centre and ‘word-of-mouth’, started by youth with a connection to the service; and Peterson 2006 recruited participants from street intercept locations (38%), through agencies (58%) and methods such as flyers or ‘word-of mouth’ (8%). For the Tischler 2002 CBA, recruitment was undertaken via posters in three homeless hostels and a letter and information sheet which was posted to each unit within the hostel (Tischler 2012 [pers comm]). Rotheram-Borus 2003 did not provide information on their recruitment method.
In the two CBAs (Rotheram-Borus 2003; Tischler 2002), the control group was based in an ‘equivalent’ setting (two shelters and a hostel, respectively) providing similar services to the agency or agencies from which the intervention population was recruited. In six studies (Baer 2007; Cauce 1994; Hyun 2005; Slesnick 2005; Slesnick 2007/08; and Slesnick 2009 EBFT; Slesnick 2009 FFT), the control group was drawn from the same agency-based population as the intervention group, and the control condition consisted of ‘service as usual’ provided by the agency. Furthermore, the agency also served as the intervention setting in all of the above studies with the exception of Slesnick 2009 EBFT; Slesnick 2009 FFT (the two interventions in this study comprised either functional family therapy (FFT), which was provided in an office location; or ecologically-based family therapy (EBFT), which typically took place in the participant's parental home).
In the three studies which recruited participants from mixed settings (Milburn 2012; Peterson 2006; Rew 2007), the intervention took place in ‘field-site offices’ (Peterson 2006); a site selected by the family, usually their home (Milburn 2012); or an unspecified location organised through the street outreach programme (Rew 2007). The studies did not specify the control condition in any detail. Relevant background data, for example on recruitment type, may have been collected in some of the pre-test and post-test assessments but they were not systematically analysed or accounted for in the papers.
The total number of participants randomised in the included studies were (in ascending order): 32 (Hyun 2005); 76 (Tischler 2002, child participants only)*; 115 (Cauce 1994)**; 117 (Baer 2007); 119 (Slesnick 2009 EBFT; Slesnick 2009 FFT ); 124 (Slesnick 2005); 151 (Milburn 2012); 180 (Slesnick 2007/08); 285 (Peterson 2006); 311 (Rotheram-Borus 2003)*; and 805 (Rew 2007)*.
*These participants were selected through quasi-randomised methods. ** This figure represents the number of participants included in the analysis, since a total number was not available for this study.
Study populations were described in the studies as homeless families with children (Tischler 2002), newly homeless youth (Milburn 2012), substance or alcohol abusing runaway adolescents (Slesnick 2005; Slesnick 2009 EBFT; Slesnick 2009 FFT), runaways (Rotheram-Borus 2003), runaway young men (Hyun 2005), and homeless adolescents or youth (Baer 2007; Cauce 1994; Peterson 2006; Rew 2007; Slesnick 2007/08). A diversity of inclusion and exclusion criteria were used across the studies. For example, the study populations in Slesnick 2005 and Slesnick 2009 EBFT; Slesnick 2009 FFT were similar except that the former selected a population with a primary drug abuse profile and the latter a population with a primary alcohol abuse profile.
Participant ages ranged from 3 to 23 years. Mean ages for participants were (in ascending order): 14.8 (Milburn 2012); 14.8 (Slesnick 2005); 15.5 (Hyun 2005); 15.6 (Rotheram-Borus 2003); 16.5 (Cauce 1994); 17.4 (Peterson 2006); 17.9 (Baer 2007); 19.2 (Slesnick 2007/08); and 19.47 (Rew 2007). Participants in Slesnick 2009 EBFT; Slesnick 2009 FFT were between 12 and 17 years old and participating children in Tischler 2002 between 3 and 16 years old.
The total percentages of male participants in the studies were (in ascending order): 33.8% (Milburn 2012); 41.1% (Slesnick 2005); 51% (Rotheram-Borus 2003); 54.7% (Peterson 2006); 55% (Slesnick 2009 EBFT; Slesnick 2009 FFT); 56% (Baer 2007); 57% (Cauce 1994); 61% (Rew 2007); 66% (Slesnick 2007/08); and 100% (Hyun 2005). The gender of the child participants in Tischler 2002 was not specified. In summary, many studies had a majority population of young men. However, the intervention in Rew 2007 was gender specific.
The largest ethnic groups in each study, as described by the authors, were: 58% Caucasian (Baer 2007); 59% white (Cauce 1994); 74% Korean Christian (Hyun 2005); 61.6% Hispanic (Milburn 2012); 72.3% Caucasian (Peterson 2006); 58% white (Rew 2007); 59% African American (Rotheram-Borus 2003); 41.1% Hispanic (Slesnick 2005); 41.1% Anglo-American (Slesnick 2007/08); and 44% Hispanic (Slesnick 2009 EBFT; Slesnick 2009 FFT). Thus, in five studies the largest ethnic group was described as white, Caucasian, Anglo-American; in three studies the largest ethnic group was described as Hispanic; in one study the largest ethnic group was described as African American; and in one study the majority represented a religious minority of majority ethnicity. Ethnicity data were not reported in Tischler 2002.
Other background information collected at baseline included abuse history, length of time on the streets or number of runaway episodes, and reasons for leaving home.
Assessment of heterogeneity
Contrary to expectation, the included studies were considerably homogeneous in terms of location (with nine out of 11 based in the US), study design (eight out of 11 being RCTs) and outcome categories. In contrast, the included studies were considerably heterogeneous in terms of outcome measures and time points, confounders controlled for and, to a lesser degree, types of study populations. Meta-analysis was possible for eight included studies (for which raw data were available), although the number of studies considered under individual outcome items varied greatly (see Table 1 for a summary). The outcomes for which most data were available were depression and delinquent behaviours at three months (data from five studies, six interventions for both).
The number of studies measuring the primary and secondary outcomes as defined in our protocol were as follows.
No studies measured the primary outcomes of inclusion and reintegration.
2. Safer or reduced substance use (e.g. reduced sharing of injecting equipment): measured in eight studies (Baer 2007; Cauce 1994; Milburn 2012; Peterson 2006; Slesnick 2005; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT; Rotheram-Borus 2003).
3. Increased use of hostel or shelter type services: measured in one study (Baer 2007).
4. Literacy: not measured in included studies.
5. Numeracy: not measured in included studies.
8. Participation in education: not measured in included studies.
9. Participation in skills-based (rather than exploitative) employment: not measured in included studies.
12. Participation in intervention planning and delivery: not measured in included studies.
Other outcomes (not included in the above)
None of the studies explicitly measured adverse outcomes.
Outcome items included in meta-analyses
Due to the heterogeneity of outcome definitions, measures and time points for follow-up used in the studies, we were only able to include a selection of the reported outcome items in the meta-analysis. Some of these represented individual items on a measure, for example an item on a questionnaire, while others represented total scores, for example an aggregate of individual items on a particular scale or subscale. For 23 such items, data were available for more than one intervention. Numbers of participants included in each meta-analysis ranged from 75 to 404. For a list of the outcome items included in the meta-analysis, including study reference, time points and measures used, see Table 1.
Outcome items not included in meta-analyses
We excluded 66 outcome items which were reported in the included studies from the meta-analysis either because they were measured in only one study (for example due to differences in measures used) or at a time point not overlapping with any other study; because we did not have access to relevant data (mean and standard deviation scores); or because they were not within the remit of this review. A full list of these outcomes according to study is presented in Appendix 4. In the Effects of interventions section, we have reported relevant outcomes narratively (according to authors' own analyses, where data were unavailable).
The following time points were reported in the studies (in ascending order): three and six weeks (Rew 2007); eight weeks (Hyun 2005); one and three months (Baer 2007; Peterson 2006); three months (Cauce 1994); three and six months (Slesnick 2007/08); six months (Tischler 2002); three, six and 12 months (Milburn 2012; Slesnick 2005); three, nine and 15 months (Slesnick 2009 EBFT; Slesnick 2009 FFT); three, six, 12, 18 and 24 months (Rotheram-Borus 2003). See also Table 1 for time points included in meta-analyses.
Confounders and process factors
Of demographic confounders, age, gender and ethnicity were most frequently accounted for in outcome analyses, and are detailed in Effects of interventions. Sexual and physical abuse history was examined separately (Slesnick 2006) in relation to Slesnick 2005; Slesnick 2009 EBFT; Slesnick 2009 FFT and primary alcohol versus primary drug abuse (in interaction with gender) in relation to Slesnick 2005 and Slesnick 2009 EBFT (Slesnick 2006c).
Potentially relevant service delivery factors included engagement of young people, treatment attendance, length of intervention, therapeutic relationship and compliance. Some of the included interventions were manualised, and observer ratings or transcript records were employed to ensure treatment fidelity by intervention providers. Process data collected in the studies included service-user satisfaction (for intervention condition only) (Baer 2007; Peterson 2006 (quantitative); Rew 2007; Tischler 2002 (qualitative)), counsellor-rated 'level of engagement' (Baer 2007; Peterson 2006), 'stage of change' (Peterson 2006), counsellor effects and treatment attendance (Slesnick 2007/08; and Slesnick 2009 EBFT; Slesnick 2009 FFT), and family resettlement (Tischler 2002). We did not report on all of the data relevant to service delivery confounders, due to the heterogeneity of the interventions and the measures used in this area, but refer to analyses conducted by study authors in the discussions below.
Follow-up rates at longest follow-up were as follows (in ascending order): 43% (intervention), 49% (control) at 12 months (Milburn 2012); 62% (EBFT), 65% (FFT), 62% (control) at 15 months (Slesnick 2009 EBFT; Slesnick 2009 FFT); 66% (intervention), 74% (control) at 24 months (Rotheram-Borus 2003); 71% (total included in analysis) at T3 (Rew 2007); 78% (intervention), 58% (control) at 6 months (Tischler 2002); 80% (total) at 3 months (Peterson 2006); 84% (control), 88% (intervention) at 6 months (Slesnick 2007/08); 88% (intervention), 81% (control) at 6 weeks (Hyun 2005); 89% (intervention), 88% (control) at 12 months (Slesnick 2005), and 92% (total) at 3 months (Baer 2007) (no attrition reported in Baer 2007; 10 participants were excluded from the analysis due to exclusion criteria).
In the following, we give a brief overview of measures used for outcome items included in the meta-analysis. Further information on scales used in the included studies, including scoring, was researched and compiled into a large table, which can be found at Appendix 5.
Sexual health and risk behaviour related outcomes were the most varied and we could only include two outcome items in our meta-analyses, both measured in Milburn 2012 and Slesnick 2007/08. Outcomes in this category were measured in Slesnick 2007/08 using the Homeless Youth Questionnaire and in Milburn 2012 using the authors' own research instrument, which was similar to the one used in Slesnick though more limited in the scope and precision of the questions asked.
Substance use related outcomes were also varied. Frequency of substance use was measured in Peterson 2006 and Baer 2007 using a Time-Limited Follow-Back interview (TLFB) developed by Sobell 1992, and in Slesnick 2005; Slesnick 2007/08; and Slesnick 2009 EBFT; Slesnick 2009 FFT using a derivation of the TLFB, Form 90, which was developed for ‘Project MATCH’, a large-scale RCT by the US National Institute on Alcohol Abuse and Alcoholism. The time scale in the TLFB is the last 30 days, as opposed to the last 90 days in Form 90; they also differ in how they quantify days of substance use. The most recent study (Milburn 2012) did not use a standardised tool of measurement. It employed the same time scale as Form 90 while the unit of measurement was similar to the TLFB. Two different measures were also used for 'problem consequences': Peterson 2006 administered the 23-item Rutgers Alcohol Problem Index (RAPI) with revised instructions (to cover any relevant form of substance abuse), while the Slesnick studies used the Problem Oriented Screening Instrument for Teenagers (POSIT) targeting substance abuse in general. The Adolescent Drinking Index (ADI), for which no detailed scoring guidance was available, was used in only one study (Slesnick 2009 EBFT; Slesnick 2009 FFT).
Depression, self-esteem and other psychological functioning outcomes were usually measured using standardised tools. Depression was measured in Slesnick 2005; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT; and Hyun 2005 using the Beck Depression Index (BDI) (score range 0 to 63), and by Cauce 1994 using the Reynolds Adolescent Depression Scale (RADS) (score range 30 to 120). Internalising and externalising behaviours were measured in Slesnick 2005; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT; and Cauce 1994 using Youth Self-Report (YSR). Self-esteem was measured in Hyun 2005 using the Self-Esteem Inventory, and Cauce 1994 used the Rosenberg Self-Esteem Scale (RSES). Numbers of psychiatric diagnoses were measured in Slesnick 2005; Slesnick 2009 EBFT; Slesnick 2009 FFT using the Computerized Diagnostic Interview Schedule (CDISC).
Social functioning was measured in six studies. The only outcome item in this category included in a meta-analysis was delinquent behaviours, measured by the Youth Self-Report (YSR) (Cauce 1994), DSM-IV criteria (Milburn 2012), and the National Youth Survey Delinquency Scale (NYSDS) (Slesnick 2005; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT).
Family functioning was measured in two studies (Slesnick 2005; and Slesnick 2009 EBFT; Slesnick 2009 FFT) using the outcomes of verbal aggression (youth) and family violence (youth) as measured on relevant subscales of the Conflict Tactics Scales (CTS); family cohesion and family conflict as measured on relevant subscales of the Family Environment Scale (FES); and parental care and parental overprotectiveness as measured on the Parental Bonding Instrument (PBI). Scores reported reflected the youths' own perceptions of, for example, the use of aggression to resolve family conflict.
We excluded 15,984 studies in a two-step process described under Selection of studies and Results of the search (see also Figure 3). The majority were excluded following screening by title and abstract.
Reasons for excluding 47 studies initially classified as 'potentially eligible' are listed in the Characteristics of excluded studies table. The most common reason was the lack of a comparable control group, for example due to convenience sampling. Nine studies did not include any control or comparison group and were excluded from the review for this reason.
Thirty studies were selected for meeting most of the objectives of the review even if failing to meet all the criteria specified under research design. A descriptive overview of these studies is presented in Appendix 1.
Another 616 records were classified in screening software as ethnographic or descriptive studies on street-connected children and youth and were excluded from the review for not evaluating effectiveness, 117 were categorised as about street children but not evaluating effectiveness, and 230 as non-English language. The first two categories, and if feasible the non-English language studies also, will be included in a larger mapping exercise.
Risk of bias in included studies
We assessed risk of bias for sequence generation, allocation concealment, blinding of participants and personnel including outcome analysis, incomplete outcome data, and selective reporting. Risk of bias assessments are presented graphically in Figure 4 and Figure 5 (see also Characteristics of included studies).
|Figure 4. Risk of bias summary: review authors' judgements about each risk of bias item for each included study.|
|Figure 5. Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.|
We did not gain sufficient information on potential randomisation or allocation methods in Rotheram-Borus 2003 but allocation in this study occurred at the level of shelters rather than individual participants. Propensity scores were used at a later stage to render intervention and control groups comparable. The authors described their study design as 'quasi-experimental'. Similarly for Rew 2007, data on randomisation and allocation procedures were not available and the authors described their study design as a 'quasi-experimental repeated measures design'. These have been classed as high risk both on sequence generation and allocation.
The sequence generation for seven studies (eight interventions) were classed as low risk. In Baer 2007; Milburn 2012; Peterson 2006; Slesnick 2005; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT randomisation was by computer (computerised coin toss was used in Milburn 2012; a computerised URN randomisation programme was utilised in the remaining five studies); Cauce 1994 utilised sequential envelopes. The sequence generation for two studies (Hyun 2005; Tischler 2002) was classed as high risk, being based on consecutive recruitment and even and odd number allocation, respectively.
For allocation concealment, six studies (seven interventions) were classified as low risk (Baer 2007; Milburn 2012; Peterson 2006; Slesnick 2005; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT). In these, allocation took place in a location separate from recruitment and by a different person, commonly the study director or a member of the intervention team. Four studies were classified as high risk (Cauce 1994; Hyun 2005; Rew 2007; Tischler 2002). In these, randomisation took place at the assessment site or not at all (the Rew 2007 study used a 'quasi-experimental' study design where participants were assigned to three different groups but we were unable to gain a clear picture of the allocation procedure).
All studies showed a high risk of bias in relation to blinding as it was not possible to blind participants in such interventions. Most outcome items used self-report mechanisms so were subjective and thus the inability to blind participants or intervention staff added a potentially high risk of bias. On blinding of outcome assessment, five studies (six interventions) were considered to represent high risk, four low risk, and two unclear risk of bias.
Incomplete outcome data
We considered the attrition rates good to very good considering the typical characteristics of the research populations, their life styles and the drop-out rates for interventions in general. While relevant information was available for each study, generally the drop-out rates appeared similar for the intervention and control groups (with the exception of Tischler 2002). However, high attrition rates may reflect the drop-out of self-selecting subsets of the population and as such add a strong risk of biasing outcome measurements. In the population included in this review, who typically had chaotic unsettled lives, it was possible that the participants retained might be more representative of the young people ready to make changes in their lives than those who dropped out. For a small number of studies the authors presented attrition analyses, as detailed below. Of these, only one study (Slesnick 2009 EBFT; Slesnick 2009 FFT) found no differences between the demographic profiles of drop-outs and retained participants. However, motivation to change was not assessed. Consequently, we rated the majority of studies as high risk in this area, with some rated unclear.
All studies appeared to report on all outcomes, while descriptive data were sometimes provided in the studies only for statistically significant or favourable outcomes. With the exception of Rew 2007; Rotheram-Borus 2003; and Tischler 2002, study authors were able to provide us with their unreported raw data on outcome items reported in their papers. No additional outcome data were provided for any of the studies. However, there was some variation between the individual outcome items reported across the Slesnick studies, even where the same research tools appeared to have been used. For example, while all of these outcome items were measured on Form 90, the number of categories of drugs used was reported for the EBFT intervention in Slesnick 2005 but not in Slesnick 2009 EBFT; number of days living at home was reported in Slesnick 2009 EBFT; Slesnick 2009 FFT but not in Slesnick 2005; and social stability was only reported in Slesnick 2007/08. There was also variation in the way substance use was reported across the studies. This could have been due to a different study focus or progressive adjustment of the measure used. There was a possibility that this represented a reporting bias. Assessments of risk of this area of bias were unclear in all cases but this was based on information available and, as reported above, there appeared to be some discrepancies so risk may have been greater than originally assessed.
Other potential sources of bias
Four of the twelve included interventions were from studies conducted by one research team (Slesnick 2005; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT) and there are similarities in terms of study design, type of intervention, location and population characteristics.
Incentives for participation in assessment were reported for the majority of the included studies, with the exception of Hyun 2005; Rotheram-Borus 2003; and Tischler 2002. In Baer 2007, youth in the intervention condition received $10 vouchers for each completed session, and intervention participants in Rotheram-Borus 2003 received minor incentives (food, $1 notes and tokens of appreciation) as rewards for participation and other positive behaviour. Finding no positive intervention effects (in contrast to Peterson 2006), Baer 2007 speculated that payment for session attendance could have undermined intrinsic motivation for participation. However, the outcome trends across the interventions were generally mixed and thus we were unable to draw firm conclusions about the potential effects of incentives on outcome findings. For potential impact on follow-up rates, see discussion on attrition analyses below.
Effects of interventions
In this section, we have detailed intervention effects primarily based on our meta-analyses, data provided by the study and change score calculations. Following this, we have highlighted heterogeneity and process factors in the included studies, drawing on both our meta-analyses and statistical analyses provided by study authors. Readers should note that here was great variation in the types of measurement tools used by studies, making comparison difficult for some outcomes. Readers are recommended to look into the original included studies for more detail on some aspects of measurement where it has not been possible to describe in detail in this review.
None of the studies included in the review reported the relative effects of different intervention components. In our meta-analyses we compared the (standardised) mean difference between intervention and control groups for each included outcome at the same time point (three, six or 12 months) (see also Measures of treatment effect). One study (Hyun 2005) used a shorter endpoint of eight weeks but was included in the meta-analysis. Additionally, we presented other relevant data for outcome items not included in the meta-analyses. These data were reported here as presented by the study authors.
We also calculated change scores for outcome items included in the meta-analysis, outcome items not included in the meta-analysis, and time points not included in the meta-analysis (utilising the longest follow-up data available). Importantly, the figures do not account for standard deviation or standard error. We used these change scores as indicators, primarily to illustrate the fact that for most outcomes a positive change (as opposed to deterioration) was evident in both intervention and comparison intervention groups. The change scores with comments can be found in Appendix 3, while brief summaries are provided in the section below.
Not measured in included studies.
Not measured in included studies.
1. Safer or reduced sexual activity
Overall, the results for this outcome were uncertain, with sexual behaviour in both intervention and control groups changing at different time points. From the data below it was unclear whether either intervention or service as usual (SAU) services impacted on sexual behaviour. In summary, the results across the studies were mixed with regard to changing sexual risk behaviour practices. Raw data for Rew 2007 and Rotheram-Borus 2003 were not available and we referred instead to the data reported by the authors.
The following outcome items were included in the meta-analysis:
- number of times had sex in last 90 days (three, six months);
- number of sexual partners in last 90 days (three, six months).
1.1 Number of times had sex in last 90 days
Three-month data were included in the meta-analysis for Slesnick 2007/08 and Milburn 2012 ( Analysis 1.1). No important or statistically significant effects were found (combined mean difference (MD) -0.04-0.25 to 0.17). Six-month data were included for Slesnick 2007/08 and Milburn 2012 ( Analysis 1.2). Again, the effects were small and not statistically significant (combined MD -0.04, 95% CI -0.22 to 0.13).
1.2 Number of sexual partners in last 90 days
Three-month data were included in the meta-analysis for Slesnick 2007/08 and Milburn 2012 ( Analysis 1.3). In Milburn 2012 the data were highly skewed (MD 0.27, 95% CI -4.55 to 5.09). In Slesnick 2007/08 there was a small, statistically significant effect in favour of the intervention group (MD -0.57, 95% CI -1.14 to 0.00). The combined MD was -0.56 (95% CI -1.13 to 0.01). Six-month data were included for Slesnick 2007/08 and Milburn 2012 ( Analysis 1.4). At six months, the data in Milburn 2012 were less skewed and favoured the control group, possibly reflecting an evening out of patterns of sexual behaviour within the group over time, though in our comparative analysis no statistically significant effects were present for either study (combined MD 0.73, 95% CI -2.97 to 4.43).
In Rotheram-Borus 2003, as reported by the authors, the number of recent sexual partners among young women had increased in the control group compared to the intervention group at 24 months (odds ratio (OR) 0.68, 90% CI 0.47 to 0.98; P = 0.084). Among young men, the number of recent sexual partners was similar at 24 months (OR 0.96, 90% CI 0.56 to 1.66) with a marginal decrease between 12 and 24 months in the intervention group and a marginal increase between 12 and 24 months in the control group (these were references to figures presented by the authors).
1.3 Unprotected sex, condom use
Data in the following sections were presented differently in the different studies and raw data were not made available to the review team. As such we have presented the data as reported in the studies.
In Rotheram-Borus 2003, frequency of unprotected sexual acts among young women was lower in the intervention group compared to the control group at 24 months (OR 0.35, 90% CI 0.17 to 0.71; P = 0.018). Among young men, the number of unprotected sexual acts was similar in both groups across all time points (OR 1.62, 90% CI 0.53 to 4.96).
In Rew 2007, data as presented by the authors on self-efficacy to use condoms showed statistically significant (P < 0.001) changes in mean scores over time, but the trends within both groups were mixed. Study authors also reported data on intention to use condoms, similarly indicating statistically significant (P = 0.25) changes in mean scores. No further evidence was presented to support the study hypothesis, so the results must be seen as unclear.
1.4 Other sexual risk behaviour
In Rotheram-Borus 2003, rates of abstinence from vaginal and anal sex among young women were higher in the intervention group at 24 months (OR 2.41, 90% CI 0.77 to 7.62; P = 0.088). Rates of abstinence from vaginal and anal sex among young men showed an increase at 12 months (somewhat sharper in the control group) followed by an equivalent decrease at 24 months (OR 1.28, 90% CI 0.24 to 6.99). Again, these findings reflected uncertainty and did not show a clear trend.
In Rew 2007, AIDS and sexually transmitted diseases (STD) knowledge scores indicated a statistically significant difference between T1 and T3 (1.309, 95% CI 0.575 to 2.042; P < 0.001) and T2 and T3 (1.658, 95% CI 0.873 to 2.443; P < 0.001) in the intervention condition, showing a stable pattern between T1 and T2 followed by a decrease by T3. In the control condition, there was a statistically significant difference between T1 and T2 (1.217, 95% CI 0.650 to 1.785; P < 0.001) and T1 and T3 (1.553, 95% CI 0.899 to 2.207; P < 0.001), showing a decrease followed by a stable pattern. At T2, there was a statistically significant difference between the groups (P = 0.003) with the intervention mean being higher than the control. For sexual self-care behaviour, there was an overall pattern of decrease in both groups. For safe sex behaviour, there was no statistically significant time effect in the intervention group (P = 0.598) while in the control group there was a statistically significant (P = 0.010) time effect for a decrease in levels of safe sex behaviour. There were no statistically significant changes in mean scores for sexual risk-taking behaviour over time, overall (P = 0.167) or in either group (P > 0.5 for both). Again, these findings reflected uncertainty and did not show a clear trend.
Summary of change scores (Appendix 3): the change scores indicated very mixed results. Overall, this category of outcomes demonstrated very limited intervention effects across studies, if any. However, it should also be noted that the baseline levels of sexual activity varied considerably across studies. Some trends favouring the control group were indicated in the Slesnick 2007/08 study.
2. Safer or reduced substance use (for example reduced sharing of injecting equipment)
Results on this outcome were also uncertain and of mixed direction.
The following outcome items were included in the meta-analysis, representing a wide range of different types of measurement with different interpretations and different reference points:
- number of days of alcohol use in last 30 days (one, three months);
- percentage days of alcohol use in last 90 days (three months);
- number of standard drinks in last 90 days (three months);
- adolescent drinking index score (three months);
- percentage days of alcohol/ drug use in last 90 days (three months);
- percentage days of only drug use in last 90 days (three months);
- number of categories of drug use in last 90 days (six months);
- number of days of marijuana use in last 30 days (one, three months);
- number of days of illicit drug use other than marijuana in last 30 days (one, three months);
- number of problem consequences (three, six months);
- number of substance use diagnoses (three months).
2.1 Number of days of alcohol use in last 30 days
One-month data were included for Baer 2007 and Peterson 2006 ( Analysis 2.1). No statistically significant or important effect was found and the mixed findings reflected uncertainty (total MD -0.3, 95% CI -2.25 to 1.59). Three-month data were included for Baer 2007 and Peterson 2006 ( Analysis 2.2). The combined MD was 1.10 (95% CI -0.67 to 2.88) favouring the comparison intervention.
2.2 Percentage days of alcohol use in last 90 days
Three-month data were included for Slesnick 2005; and Slesnick 2009 EBFT; Slesnick 2009 FFT ( Analysis 2.3). The combined MD was -0.34 (95% CI -2.34 to 1.75), that is clinically small and not statistically significant. However, there was a larger effect in favour of the FFT experimental group compared to EBFT, but still relatively small, in Slesnick 2009 EBFT; Slesnick 2009 FFT (MD -3.00, 95% CI -9.89 to 3.89), and this remained non-statistically significant.
In Rotheram-Borus 2003, results were uncertain and may have reflected a short term positive change but no maintenance of gains in the longer term. Alcohol use among young women was lower in the intervention group at 12 months (OR 0.43, P = 0.053) but slightly higher at 24 months (OR 1.72, 90% CI 0.54 to 5.49). Alcohol use among young men was lower in the intervention group at three months (OR 0.25, P = 0.1000) but similar at 24 months (OR 1.16, 90% CI 0.24 to 6.99).
2.3 Number of standard drinks in last 90 days
Three-month data were included for Slesnick 2009 EBFT; Slesnick 2009 FFT ( Analysis 2.4). There was a small non-statistically significant effect in favour of both the EBFT intervention group (MD -3.05, 95% CI -7.26 to 1.16) and the FFT intervention group (MD -2.73, 95% CI -6.49 to 1.03). The combined MD was similarly small but statistically significant and favoured the intervention group (MD -2.87, 95% CI -5.68 to -0.07).
2.4 Adolescent drinking index (ADI) score:
3-month data were included for Slesnick 2009 EBFT; Slesnick 2009 FFT ( Analysis 2.5). There was a small, statistically non-significant and uncertain effect in favour of the control group for the EBFT intervention (2.97, [-4.48, 10.42]). For FFT, there was a more negligible impact which favoured the intervention group (-1.19, [-9.43, 6.96]). The combined MD was 1.08 [-4.42, 6.57].
2.5 Percentage days of alcohol/ drug use in last 90 days
The difference in findings from this scale compared to the others used in studies in the review may have reflected the fact that this scale aggregated drug and alcohol use data rather than keeping them separate. Furthermore, the precise definition used in this scale of drug and alcohol use was unclear.
Three-month data were included for Slesnick 2009 EBFT; Slesnick 2009 FFT ( Analysis 2.6). There was a larger effect in favour of the control group compared to the EBFT intervention (MD 8.00, 95% CI -12.89 to 28.89) but this was not statistically significant and therefore reflected uncertainty. Similarly, but in the opposite direction, there was a larger but non-statistically significant effect in favour of the FFT intervention (MD -10.00, 95% CI -26.72 to 6.72). The combined MD was -2.97 (95% CI -16.02 to 10.08).
2.6 Percentage days of only drug use in last 90 days
Three-month data were included for Slesnick 2009 EBFT and Slesnick 2009 FFT ( Analysis 2.7). Small and non-statistically significant effects were found. The MD for the EBFT intervention was 2.00 (95% CI -18.07 to 22.07) slightly favouring the control group. The MD for the FFT intervention was -7.00 (95% CI -23.72 to 9.72) with the mean favouring the intervention group. The combined MD was -3.31 (95% CI -16.16 to 9.53).
2.7 Number of categories of drug use in last 90 days:
In Rotheram-Borus 2003, using the only data available from the study authors, the number of drugs used among young women was lower in the intervention group at 12 months (OR 0.36, P = 0.019) but similar at 24 months (OR 1.34, 90% CI 0.59 to 3.05). Among young men, the number of drugs used was lower in the intervention group at 3 months (OR 0.59, P = 0.085) but similar at 24 months (OR 0.90, 90% CI 0.38 to 2.12). This may have indicate that in this population any changes in substance use behaviour may not be sustained over time.
2.8 Number of days of marijuana use in last 30 days
One-month data were included for Baer 2007 and Peterson 2006 ( Analysis 2.9). There were small and statistically non-significant effects favouring the control group in Baer 2007 and the intervention group for Peterson 2006. The combined MD was -0.52 (95% CI -3.65 to 2.62). Three-month data for Baer 2007 and Peterson 2006 ( Analysis 2.10) showed a slightly larger effect favouring the control group in Baer 2007 (MD 1.60, 95% CI -3.60 to 6.80) and a decreased (statistically non-significant) effect slightly in favour of the intervention group in Peterson 2006. The combined MD was 0.37 (95% CI -2.73 to 3.47). These results showed mixed direction of effects and reflected uncertainty.
In Rotheram-Borus 2003, in data as reported by the authors, marijuana use among young women was lower in the intervention group at 12 months (OR 0.19, P = 0.005) but higher at 24 months (OR 2.51, 90% CI 0.61 to 10.38). Among young men, marijuana use was lower in the intervention group at 3 months (OR 0.31, P = 0.050) but identical at 24 months (OR 1.08, 90% CI 0.17 to 6.93). These results showed that gains at one month did not appear to be maintained over time.
2.9 Number of days of illicit drug use other than marijuana in last 30 days
One-month data were included for Baer 2007 and Peterson 2006 ( Analysis 2.11). There was a small, statistically non-significant effect favouring the control group in Baer 2007 (MD 1.50, 95% CI -0.70 to 3.70) but no statistically significant effect in Peterson 2006. The combined MD was 1.21 (95% CI -0.68 to 3.10). Three-month data for Baer 2007 and Peterson 2006 showed no statistically significant effects. The combined MD was 0.22 (95% CI -1.84 to 2.28). These results were again mixed and reflected uncertainty.
2.10 Number of problem consequences (POSIT)
Three-month data were included for Slesnick 2005 and Slesnick 2009 EBFT; Slesnick 2009 FFT ( Analysis 2.13). There were small effects favouring the control group in all three studies. The combined MD was 1.51 (95% CI 0.56 to 2.47), which was statistically significant showing overall benefit for the control group. The largest effect in favour of the control group was for the EBFT intervention (MD 2.21, 95% CI -0.02 to 3.68). Six-month data were included for Slesnick 2005 and Slesnick 2007/08 ( Analysis 2.14). No statistically significant effect was found at this this time point. The combined MD was 0.34 (95% CI -0.67 to 1.34).
For Peterson 2006, data on drug use consequences (RAPI) were not available. The authors reported that they found no reduction in drug use consequences at three months across the groups. Further, there was no evidence to suggest that the intervention had changed the consequences of drug use.
2.11 Number of substance use diagnoses (CDISC)
Three-month data were included for Slesnick 2009 EBFT; Slesnick 2009 FFT. The effects were marginal and statistically non-significant, although the combined MD reached statistical significance (MD -0.70, 95% CI -1.27 to -0.14), a very small benefit in favour of the intervention group.
Summary of change scores (Appendix 3): the change scores in this category were also mixed, as discussed in more detail in the summary of results. Some longer term change scores (without SD and thus with no estimate of precision) favouring the intervention group were found in Slesnick 2009 EBFT; Slesnick 2009 FFT for per cent days of alcohol use in the last three months, number of standard drinks, per cent days of alcohol or drug use, and per cent days of only drug use. Longer term change scores in Milburn 2012 were also mixed but appeared to favour intervention for 'times had alcohol' in the last three months. Results for drug and alcohol outcomes across the remaining studies were mixed, with few showing more than marginal changes that were mostly not sustained over time. Benefits for control groups (for example number of days of abstinence in Baer 2007; percentage days used tobacco in Slesnick 2005) may have reflected benefit of the comparison SAU intervention being greater than that for the index intervention being evaluated.
3. Increased use of hostel or shelter type services
Not measured in included studies.
Not measured in included studies.
Summary: results for self-esteem demonstrated no statistically significant benefit.
Self-esteem: endpoint data were included for Cauce 1994 (three months) and Hyun 2005 (eight weeks) ( Analysis 3.1). No statistically significant effect or clinically important effect was found (combined SMD 0.11, 95% CI -0.22 to 0.44).
Summary of change scores (Appendix 3): the trends based on change scores appeared similar for all groups, indicating marginal to slight improvement.
Summary: results for depression were mixed with no clear demonstration of benefit.
Depression: three-month data were included for Cauce 1994, Hyun 2005, Slesnick 2005 and Slesnick 2009 EBFT; Slesnick 2009 FFT ( Analysis 4.1). There was no demonstrated combined effect for depression at three months and the results from individual studies were mixed (combined SMD -0.04, 95% CI -0.40 to 0.31). Six-month data were included for Slesnick 2005 and Slesnick 2007/08 ( Analysis 2.14). The combined MD revealed no statistical significance and marginal benefit favouring the intervention group (MD -0.43, 95% CI -2.83 to 1.98).
8. Participation in education
Not measured in included studies (but see aggregate measure of 'social stability' under 'social functioning').
9. Participation in skills-based (rather than exploitative) employment
Not measured in included studies.
10. Reduced use of violence
Summary: no statistically significant effects were demonstrated.
The following outcome items were included in the meta-analysis:
- verbal aggression (youth) (three months);
- family violence (youth) (three months).
10.1 Verbal aggression (youth) (CTS)
No effect was found on this measure (combined MD -0.00, 95% CI -0.07 to 0.06) ( Analysis 5.1).
10.2 Family violence (youth) (CTS)
No effect was found on this measure (combined MD -0.00, 95% CI -0.02 to 0.02) ( Analysis 5.2).
Summary of change scores (Appendix 3): the baseline mean scores on these measure were low in all groups and there appeared to be similarly small reductions for both outcome items across the groups.
11. Increased contact with family
Summary: no statistically significant effects were demonstrated.
11.1 Percentage days living at home
Three-month data were included for Slesnick 2009 EBFT; Slesnick 2009 FFT ( Analysis 6.1). There were no statistically significant effects on this measure due to considerably wide CIs, possibly due to small numbers of participants or to uncertain effect. The MD for the EBFT comparison was -2.00 (95% CI -28.09 to 24.09) favouring the control group, and for the FFT comparison -17.00 (95% CI -43.22 to 9.22) again favouring the control group. The combined MD was -9.46 (95% CI -27.96 to 9.03) in favour of the control condition. This may have indicated benefit to the comparison condition.
Summary of change scores (Appendix 3): the trend for the intervention groups were mixed at different time points but overall the trends appeared to favour the control group while indicating improvements across all groups at 12 months.
12. Participation in intervention planning and delivery
Not measured in included studies.
The following outcomes were relevant to the secondary outcomes as stated in our protocol but did not correspond to predefined secondary outcome measures. We have grouped them into the following categories: social functioning, psychological functioning and family functioning. As above, we have reported on data included in the meta-analysis, change scores (calculated by the review authors) and other data (as presented by study authors).
13. Social functioning
The following outcome item was included in meta-analysis.
Delinquent behaviours (at 3, 6 and 12 months)
Summary: for this outcome item some statistically significant effects were demonstrated, but overall the results were inconclusive.
Three-month data were included for Cauce 1994, Milburn 2012, Slesnick 2005, and Slesnick 2009 EBFT; Slesnick 2009 FFT ( Analysis 7.1). The combined point estimate showed a marginal effect that was statistically significant (combined SMD -0.29, 95% CI -0.54 to -0.03). Of the individual studies, only the Milburn 2012 results showed statistical significance, although the effect was very small. Six-month data were included for Milburn 2012, Slesnick 2005, and Slesnick 2007/08 ( Analysis 7.2). A marginal and statistically non-significant effect was found (combined SMD -0.07, 95% CI -0.52 to 0.37), although again Milburn 2012 showed a statistically significant but small result. Twelve-month data were included for Milburn 2012 and Slesnick 2005 ( Analysis 7.3). The combined SMD was -0.16 (95% CI -1.05 to 0.72). However, a small statistically significant effect was found favouring the intervention group in Milburn 2012 (MD -0.63, 95% CI -1.11 to -0.14).
Summary of change scores (Appendix 3): the change scores for delinquent behaviours were mixed and difficult to interpret because of limited information on the various scales used. In one study (Slesnick 2007/08) there appeared to be an increase in the intervention group against a reduction in the control group. In other studies there were reductions in both groups, slightly larger in the intervention groups in Cauce 1994 and Peterson 2006 and larger in the control group in the Slesnick studies. Some benefit to the intervention group was suggested for social stability as measured in Slesnick 2007/08 and peer relations as measured in Tischler 2002.
14. Psychological functioning
Summary: for this outcome category, no statistically significant effects were found in meta-analyses.
The following outcome items were included in meta-analysis:
- internalising problems (at three, six months);
- externalising problems (at three, six months);
- number of psychiatric diagnoses (three months).
14.1 Internalising problems
Three-month data were included for Cauce 1994, Slesnick 2005, and Slesnick 2009 EBFT; Slesnick 2009 FFT ( Analysis 8.1). There were no clinically or statistically significant effects for internalising problems at three months (combined SMD 0.09, 95% CI -0.14 to 0.32). Six-month data were included for Slesnick 2005 and Slesnick 2007/08 ( Analysis 8.2). The forest plot showed little evidence of effect with a marginal non-statistically significant effect in favour of the experimental groups, with the combined MD being -1.14 (95% CI -3.36 to 2.10).
14.2 Externalising problems
Three-month data were included for Cauce 1994, Slesnick 2005, and Slesnick 2009 EBFT; Slesnick 2009 FFT ( Analysis 8.3). There were no statistically significant effects for externalising problems at three months (combined SMD 0.23, 95% CI -0.14 to 0.60). Six-month data were included for Slesnick 2005 and Slesnick 2007/08 ( Analysis 8.4). The combined MD was 0.02 (95% CI -2.21 to 2.25). For Slesnick 2005, there was a small MD of 2.59 (95% CI -1.34 to 6.52) favouring the control group. In contrast, for Slesnick 2007/08 there was a slight but similarly statistically non-significant effect (MD -1.21, 95% CI -3.93 to 1.51) favouring the intervention group.
14.3 Number of psychiatric diagnoses (CDISC)
Summary of change scores (Appendix 3): the change scores in this category suggested either little change or improvements in all groups, with the scale of the change appearing to favour different groups in different studies.
15. Family functioning
Summary: no statistically significant effects were found apart from a small effect favouring the intervention group for family cohesion at three months.
The following outcome items were included in the meta-analysis:
- family cohesion (three months);
- family conflict (three months);
- parental care (three months);
- parental overprotectiveness (three months).
15.1 Family cohesion (FES)
A small, clinically marginal but statistically significant effect was found on this measure (combined MD 0.88, 95% CI 0.23 to 1.54) ( Analysis 9.1). There was a slight non-significant effect in favour of the intervention groups (EBFT and FFT) in Slesnick 2009 EBFT; Slesnick 2009 FFT (MD 1.12, 95% CI -0.01 to 2.25; and MD 1.30, 95% CI -0.06 to 2.66, respectively).
15.2 Family conflict (FES)
No effect was found on this measure (combined MD -0.05, 95% CI -0.91 to 0.81) ( Analysis 9.2).
15.3 Parental care (PBI)
No statistically significant effect was found on this measure (combined MD 1.68, 95% CI -0.63 to 4.00) ( Analysis 9.3) but there were marginal effects for all intervention groups: the MD was 1.45 (95% CI -1.77 to 4.67) in Slesnick 2005, 2.16 (95% CI -2.41 to 6.73) for Slesnick 2009 EBFT, and 1.67 (95% CI -3.18 to 6.52) for Slesnick 2009 FFT.
15.4 Parental overprotectiveness (PBI)
No statistically significant effect was found on this measure (combined MD -0.13, 95% CI -2.58 to 2.43) ( Analysis 9.4) but there were small non-statistically significant effects in differential directions: the MD was 2.34 (95% CI -0.99 to 5.67) in Slesnick 2005 favouring the control group, -3.39 (95% CI -8.52 to 1.74) for Slesnick 2009 EBFT, and -2.69 (95% CI -7.78 to 2.40) for Slesnick 2009 FFT in favour of the intervention groups. Results for this outcome were mixed and thus uncertain, with some benefit for control groups, thus indicating potential benefit of the comparison SAU condition.
Summary of change scores (Appendix 3): according to change scores (longest follow-up 15 months),trends appeared similar for all groups, indicating improvement, apart from an increase in parental overprotection at three months in the control group in Slesnick 2009 FFT. The magnitude of change appeared either to be similar or favour different groups at different time points.
Service evaluations were conducted in four studies. Participants in Peterson 2006 and Baer 2007 rated their satisfaction with the intervention on a 5-point scale. Evaluations were mainly positive (even where actual outcomes were mixed); Peterson 2006 reported their scores to have been so consistently positive that the data were excluded from the analysis. Rew 2007 and Tischler 2002 also reported positive feedback from their qualitative evaluations.
The majority of the statistical meta-analyses in this review show 0% in the I
Two analyses showed high I
Overall, however, the populations did differ between the studies, which might explain some of the variation (see below).
The studies collected a wide range of demographic data; however, the data collected were not consistent across studies and were not consistently utilised in data analyses. In this section we have discussed the most commonly assessed demographic confounders. As we did not gain access to raw data specific to subpopulations, we relied here on authors' own analyses.
Overall demographic analyses
According to Baer 2007, demographic factors and treatment exposure did not moderate outcomes. Slesnick 2009 EBFT; Slesnick 2009 FFT found that the level of (HIV related) high-risk behaviours in which participants engaged in at baseline were stronger predictors of change in HIV risk behaviours than treatment condition, Beck Depression Inventory rated depression and other demographic variables (those with a higher score were more likely to show statistically significant improvement).
For practical reasons, transient children and youth were sometimes excluded from participation or were lost to follow-up, although some studies actively tried to minimise attrition rates (see Incomplete outcome data (attrition bias)), for example by contacting absent participants. Incentives for assessment attendance for both groups were used in the majority of the studies and ranged from practical items such as toothbrushes to vouchers or money. Participants in all studies by Slesnick were paid the most in absolute value ($50 at each follow-up). Participants in a similar study by Milburn 2012 received $30 to $40 at follow-up. Participation rates varied across these studies, with Milburn 2012 having the lowest follow-up rate among all the included studies. Whilst no assessment incentives were reported in Rotheram-Borus 2003, their follow-up rates were relatively good even at 24 months (as above). On the whole, follow-up rates in the included studies were relatively good (see Description of studies).
In Slesnick 2007/08, drop-outs were found to differ on alcohol abuse, marijuana dependence and HIV risk for the past three months. Youth who completed all assessments had greater prevalence of marijuana dependence, lower prevalence of alcohol abuse and lower HIV risk score. They did not differ significantly by gender, ethnicity, treatment modality or baseline depression. This may be contrasted with the authors' finding that baseline HIV risk behaviour scores were the strongest predictor of improvement on the same outcome measure.
In Slesnick 2009 EBFT; Slesnick 2009 FFT, those lost to follow-up did not differ in demographic or dependent variables compared to participants retained in the study. Attrition also did not differ by treatment modality.
In Peterson 2006, attrition was associated with age, recruitment during spring or summer, recruitment area and frequency of drug use. Among intervention and assessment-only groups, having been on the street longer, being male, having been recruited during the spring or summer and using alcohol more frequently increased the likelihood of missing one or more follow-up interviews.
In summary, the profile for drop-out participants varied across studies, possibly depending on recruitment methods, engagement strategies and type of intervention (see also Patton 2011). The available data were too limited for drawing overall conclusions.
In Slesnick 2005, no treatment interactions by gender were found. In Slesnick 2009 EBFT; Slesnick 2009 FFT, some gender differences at baseline were reported. EBFT was found to be effective for both young men and young women in reducing substance use, while FFT was reported to decrease substance use for young men (and older adolescents) only. Neither young men nor young women in the SAU group significantly reduced their substance use, and young men in SAU were reported to have increased their alcohol use by 50% by 15 months. However, an analysis in Slesnick 2006c, apparently combining data from Slesnick 2005 and Slesnick 2009 EBFT, maintained that primary drug using young men (Slesnick 2005 sample) in the intervention group increased their use of alcohol by 30% while decreasing their drug use, as did participants in both the intervention and control groups regardless of gender or being a primary alcohol or drug user (Slesnick 2005 and Slesnick 2009 EBFT sample).
In Slesnick 2007/08, young women reported higher overall HIV risk behaviours at baseline. Also, young women were more likely to engage in HIV risk behaviours (main effect). No gender effects were reported in Slesnick 2007/08.
In Slesnick 2005, 47% of female participants reported having ever been sexually abused, compared to only 8% among male participants. Abuse history was found to moderate certain outcomes (problem consequences and number of drugs used) in favour of youth assigned to the EBFT intervention. Slesnick 2006 combined data from the included Slesnick 2005 and Slesnick 2009 EBFT studies (which involved the same intervention) for analysis with a focus on history of abuse. Contrary to expectations, no association was found between abuse history and level of substance use or abuse history and treatment outcome.
In Rew 2007, young women scored higher on a number of positive sexual health measures (for example AIDS and STD knowledge). The study results also suggested that young women benefited from the gender-specific intervention by increasing their confidence and self-care behaviours compared to those in the control condition.
In Rotheram-Borus 2003, the number of sexual partners and the number of unprotected sexual acts were lower at 24 months for young women in the intervention group, according to the authors significantly so (raw data were unavailable for this study). The authors suggested that young women were more likely to find a degree of stability and protection in romantic relationships limiting the need to participate in harmful sexual and drug use behaviours for survival, whereas young men were more continually exposed to high-risk environments and events such as incarceration.
Slesnick 2007/08 was the only study to assess whether participants had engaged in 'survival sex', defined as 'trading sex for money, food or shelter' (p5). In the intervention group, the percentage of participants who had engaged with 'survival sex' remained at 3.1% to 3.7% of the population across the three time points (baseline, 3 months and 6 months). In the SAU group, the percentage dropped from 8.3% at baseline to 3.0% at 3 months and 0% at 6 months. While the numbers were small, this result could be interpreted in different ways depending on whether participants in SAU reduced their engagement in 'survival sex' or simply dropped out from the study.
Ethnicity analyses were conducted in the Slesnick studies (Slesnick 2005; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT) comparing Anglo-American with non-Anglo-American or Hispanic participants. They found few differences between ethnic groups at baseline. In Slesnick 2005, Anglo-American youth were more likely to report at least one previous suicide attempt; more Anglo-American youths reported having had sex with more than one partner in the last 24 hours in Slesnick 2007/08; and Anglo-American youth reported higher conflict tactics with verbal aggression in Slesnick 2009 EBFT; Slesnick 2009 FFT. The only treatment interaction by ethnicity was found for the number of DSM-IV Axis I diagnoses in Slesnick 2005, which showed an opposing pattern for the two groups for changes at three and six months. At 12 months both groups obtained similar scores. The primary treatment outcomes were not moderated by ethnicity.
Age analyses were conducted in the Slesnick studies. In Slesnick 2007/08, older (19 to 22 year-old) participants reported higher means on the overall HIV risk behaviour score, had higher HIV knowledge, and were more likely to report intravenous drug use than younger (14 to 18 year-old) participants. A slight increase in condom use was the only positive outcome related to reduction in sexual risk behaviours in this study. This was observed for all participants in the intervention group, as well as older participants in SAU, whereas younger participants in SAU decreased their use of condoms. The change patterns at three and six months differed according to age group within each treatment condition. Further, for age moderated treatment outcome for depression, in the intervention group participants of all ages significantly reduced their depression, and younger youth in SAU also reduced their depression in contrast to older youth in SAU who did not.
Other demographic factors
No study exclusively compared young people receiving a service with those on the street and not engaged in any way with services. Due to the study methodologies most commonly used, the research participants in the intervention and SAU groups may have represented a self-selected sample with relatively high levels of help-seeking attitudes and motivation to change, as many outcomes measured obtained a positive change from baseline in both groups (see change scores in Appendix 3). The included studies provided very limited evidence that could be used to test this hypothesis.
In Peterson 2006, the study counsellor rated intervention participants' 'level of engagement'. This was not found to differ by age, gender, length of time on the street, baseline drug use, history of injection drug use, sexual and physical abuse history, or recruitment method, which included recruitment of some participants directly from the street. Some degree of correlation was found with 'stage of change' (see below).
Peterson 2006 was also the only study to measure participants' 'stage of change'. Their measure was based on Prochaska et al's conceptualisation (1992) (cited in Peterson 2006) and was assessed at baseline only using an algorithm based on intention to change alcohol or drug use, whether changes had already been made, and the time frame of those intentions or changes. Stage of change was found to moderate outcome results for drug use ('summed drug use other than marijuana') but not alcohol or marijuana use. No analysis of correlation with other demographic variables was presented for this variable.
As reported above under the section on gender analyses, Slesnick 2006, examined the relationship between abuse history, substance abuse at baseline and family functioning as a treatment outcome, apparently combining data from Slesnick 2005 and Slesnick 2009 EBFT; Slesnick 2009 FFT. Slesnick 2006c, combining data from Slesnick 2005 and Slesnick 2009 EBFT examined the relationship between primary drug versus primary alcohol substance abuse profile at baseline, gender and substance abuse as a treatment outcome (also reported above under 'gender analyses').
None of the included studies reported on a separate process evaluation component. However, some service delivery factors were accounted for in individual analyses.
In Peterson 2006, participants were rated by the study counsellor for their perceived 'level of engagement'. For analyses, intervention participants were grouped into those with 'high' and those with 'low' level of engagement. Participants classed as ‘high engagement’ as opposed to 'low engagement' had significantly lower scores on 'summed drug use other than marijuana' (but not on the other two outcomes) at one month, although the contrast had reduced to non-significant levels at three months. Mean scores for control group participants' were in-between these two groups. The result was not replicated in a later study (Baer 2007).
Comparing treatment attendance rates in their two family therapy interventions, Slesnick 2009 EBFT; Slesnick 2009 FFT speculated that the physical setting of the family therapy intervention (for example home rather than an office) rather than the particular style of therapy (that is EBFT versus FFT) may have been a critical factor for successful treatment engagement (defined as responsiveness to the therapy approach, leading to more positive outcomes). The moderating effects of gender and age on some outcomes were hypothetically linked to treatment engagement.
Treatment attendance was easier to measure than the more qualitative concept of treatment engagement. According to statistical analyses in Slesnick 2009 EBFT; Slesnick 2009 FFT, two demographic variables (higher externalising behaviours and sexual abuse) were associated with higher treatment attendance in the EBFT condition only. However, no moderating effects were reported for treatment attendance. In another Slesnick study (Slesnick 2008; not included in the review), sexual abuse history and a history of suicide attempts were found to predict higher levels of treatment attendance, and higher treatment attendance in turn was associated with a higher reduction in alcohol use but not with other substance use outcomes.
While some interventions did appear effective on certain outcome measures, the reasons for their effectiveness remained unclear. For example, Peterson 2006 (p259 to 60) reported that while the use of illicit drugs (other than marijuana) was reduced in the intervention group relative to the control group, "there was nothing in the data to suggest that the [motivational enhancement] intervention had even a small effect on drug use through influencing stage of change". The most consistently positive results for substance use outcomes were found for the two types of family-therapy interventions evaluated in Slesnick 2005, and Slesnick 2009 EBFT; Slesnick 2009 FFT. In contrast, none of these interventions had a significant differential impact on family functioning, which improved for both groups, or for percentage days living at home (only reported in Slesnick 2009 EBFT; Slesnick 2009 FFT) for which there appeared to have been a relatively greater increase in the control group.
According to the composite analysis in Slesnick 2006, family cohesion, number of diagnoses other than substance related ones at time point one, and number of drugs used at baseline emerged as the three potentially most significant predictors of change in substance use in the intervention group. Together they accounted for around 39% of the variability in change across time. Self-reported family cohesion was the only treatment relevant factor, defined as perceived commitment, mutual help and support within the family. While family cohesion also improved in the SAU groups, it was not associated with change in substance use for SAU participants according to this analysis.
Although these findings may have partly reflected inadequacies of outcome measures, they also suggested that interventions did not necessarily or primarily achieve change in the anticipated manner, that is by enhancing motivation to change or improving family functioning. Population characteristics, pre-existing motivation to change, level of engagement or treatment attendance, setting, counsellor style, and length or intensity of intervention emerged as some candidates that could contribute to the findings obtained. For example, Slesnick 2007/08 reported on significant therapist effects on reductions in substance use, although what distinguished the more successful therapist styles was not examined. Whilst some of these factors were controlled for in individual analyses, the studies did not provide sufficient evidence for robust overall conclusions. Overall, the range of potentially confounding factors makes such analyses very difficult.
Summary of main results
Overview of studies
The vast majority of the 11 included studies were comparisons of two different interventions, a specialised, therapeutic intervention compared with service as usual (SAU) (three studies did not detail comparison conditions but research populations were largely recruited from shelters rather than the street). In most cases, the comparison intervention could also be considered a co-intervention since intervention participants were not excluded from taking part in SAU.
All of the specialised interventions were based on therapeutic models, including social, emotional, cognitive, behavioural, and systemic orientations. Four interventions represented a multicomponent approach also including liaison with external service providers (for example housing departments, legal bodies) and six were delivered within a peer or family group setting. Participatory methods were not utilised in any study. Not all of the included studies provided information on SAU characteristics. It is clear, however, that some of the usual services were provided to a high standard (see Quality of the evidence section). None of the included studies had long term residential settings as a control condition.
While the study populations varied somewhat between studies and the interventions were reasonably heterogeneous, the selected outcomes were notably homogeneous across the included studies, falling primarily into the categories of substance abuse (drug or alcohol), individual psychological functioning, social functioning and sexual health behaviours. In contrast, outcome measures and, consequently, individual outcome items were considerably varied. Little information was available to interpret the clinical or subjective significance of the results for the study population in question.
In terms of the aims of the review, there were no studies explicitly targeting the primary outcomes defined in our protocol, which were inclusion and reintegration. In summary, all of the included interventions were relevant mainly to our secondary outcomes, although only a small proportion of the elements covered in our secondary outcomes or logic models were covered by any of the included interventions. In contrast, comparison interventions, that is SAU, often appeared broader in scope.
The secondary outcomes for which data were included in the meta-analysis were safer or reduced sexual activity; safer or reduced substance use; self-esteem; depression; reduced use of violence; and increased contact with family. Data from one study were presented for increased use of hostel or shelter type services. No data were available for literacy; numeracy; participation in education; participation in skills-based (rather than exploitative) employment; or participation in intervention planning and delivery. Other outcomes measured in the included studies fell into the categories of social functioning, psychological functioning and family functioning. None of the studies reported on adverse outcomes.
Only a limited number of studies could be included in meta-analyses due to the extreme heterogeneity of outcome measures and time points used. To complement the meta-analyses, we also calculated change scores for all outcome items for which we had raw data (see Appendix 3), primarily to highlight the fact that in many cases intervention effects appeared to be paralleled by positive changes of similar scale in the control group. Instances where the control group improved more than the intervention group, or improved in opposition to a deteriorating trend in the intervention group, were rarely highlighted by study authors. We wished to draw attention to such instances as they may provide some evidence of the comparative strengths of the SAU condition. However, due to the nature of the data available to us, we could not calculate standard deviations or standard errors for the change scores. We therefore refer to them as indications only.
Even in cases where some statistically significant effects were indicated through meta-analyses (or where change scores suggested statistically or clinically significant changes), most interventions achieved mixed results in relation to different outcome categories or items and different time points. No consistent pattern was found for these differential impacts across studies. Also, the clinical and practical significance of the findings was often unclear.
Methodological limitations in the included studies are detailed elsewhere in the review, but tentatively the data appears to support the conclusion that services need not be highly specialised or technical in order to foster some degree of positive change among street-connected children and youth recruited through shelters or drop-ins. A specialised intervention offering some therapeutic programme has not been proved consistently better than usual shelter or drop-in service in these studies. On the basis of longer term change scores from four comparable studies (Milburn 2012; Slesnick 2005; Slesnick 2009 EBFT; Slesnick 2009 FFT), benefits achieved in the intervention and control groups appeared in some cases substantial and relatively long-lasting (12 to 15 months).
Due to the limitations of the study designs employed, generic maturational effects underlying positive trends cannot be ruled out. As noted in one study, substance use patterns are more often characterised by change rather than stability, and 'there appear to be natural developmental processes toward moderation of use' (Baer et al 1998, cited in Peterson 2006, p261). Longitudinal data on homeless young people living on the street in the US offers some support for this trend (Whitbeck 2009, Chapter 15). However, the latter authors also argue that on the level of individual diagnoses, longitudinal data demonstrate long term stability across a range of mental health indicators including diagnoses for substance abuse, as well as continued social marginalisation. The findings support the notion of complementing overall analyses with individual-level analyses, as discussed below in the section on Quality of the evidence (statistical analyses).
In our analyses, some of the most consistent improvements (in either the intervention or the control group) took place in the context of substance abuse among participants residing at runaway shelters. Runaways are a distinct subpopulation of street-connected children and youth who may never have lived on the street (Robertson and Toro 1999, cited in Slesnick 2007/08). Thus, they are likely to have relatively stronger family ties and limited engagement with street life. Participant scores on a range of risk measures (for example HIV risk behaviours, sexual activity and family violence) were usually relatively low on average in the included studies. However, even with these populations the results were mixed on the level of individual outcome items, as discussed below. Overall, we have limited knowledge of the process factors contributing to positive outcomes.
According to our meta-analyses, statistically significant changes in either direction were observed for five outcome items. These were number of standard drinks at three months (favouring the intervention) (see Analysis 2.4), number of problem consequences at three months (favouring the comparison intervention) (see Analysis 2.13), number of substance use diagnoses at three months (favouring the intervention) (see Analysis 2.15), delinquent behaviours at three months (favouring the intervention) (see Analysis 7.1) and family cohesion at three months (favouring the intervention) (see Analysis 9.1). Each analysis relates to a small number of studies (among these analyses, the highest number of interventions was included for 'delinquent behaviours at three months', which also shows the smallest effect). Overall, these findings appear mixed and inconsistent.
As noted above, a limitation of our primary analyses is that we were unable to include much of the relevant data in the meta-analysis due to different measurement types and time points. Therefore, for the evaluation summary below, we also draw on change scores (Appendix 3) and authors' own analyses.
Safer or reduced sexual activity: outcome measures as well as findings in this category were very mixed, with limited to no statistically significant or consistent intervention effects apparent. Moreover, we did not gain access to raw data from two out of four relevant studies in this category, and data for some of the outcome items were considerably skewed. In Rotheram-Borus 2003, which had the longest follow-up period in any study, the authors report that despite initial improvements in certain areas, in particular substance use, relapse had occurred by between three and six months for young men and by 12 months for young women. The only long term effect found in their study was that for young women, both the number of sexual partners and the number of unprotected sexual acts were lower at 24 months. Among young men, the intervention and control groups followed a broadly similar pattern, while among women the patterns were more mixed.
Safer or reduced substance use: this was the category for which the most data were available for comparison and therefore we report on the outcomes at some length.
In two studies (Baer 2007; Peterson 2006), change scores indicated improvements in both groups with some appearing to favour the control group in terms of the scale of the change (for example number of days of alcohol use in Peterson 2006). The authors only report a statistically significant intervention effect on 'summed drug use other than marijuana' at one month but not three months in Peterson 2006.
According to the authors of three studies, family therapy interventions for runaway adolescents appear to have achieved some statistically significant and lasting (12 to 15 month) benefits in reducing alcohol or drug use, somewhat above the similarly positive benefits for participants receiving SAU (Milburn 2012; Slesnick 2005; Slesnick 2009 EBFT; Slesnick 2009 FFT). The changes in both groups also appear clinically significant. Overall, however, the results from these studies were mixed, as discussed below. It should also be noted that Milburn 2012 had the highest rate of attrition among the included studies; approximately half of the participants were missing at 12 months. Furthermore, research participants in all three studies were shelter-residing runaways with some family contact, and the majority of the research participants were of Hispanic background.
Data even from the relatively most successful interventions (Milburn 2012; Slesnick 2009 EBFT; Slesnick 2009 FFT) suggest that interventions may to some degree change the pattern of substance abuse rather than reduce it. For example, in Milburn 2012 intervention participants (with a primarily alcohol using profile) increased their use of marijuana while reducing their use of alcohol and hard drugs. Similarly, though demonstrating an opposite trend, an analysis in Slesnick 2006c, combining data from Slesnick 2005 and Slesnick 2009 EBFT, suggested that unlike primary alcohol abusing participants (both young men and women) primary drug abusing young men in the EBFT intervention group increased their use of alcohol by 32% while decreasing their drug use similarly to other groups (including the control group). Among the studies by Slesnick, separate data on marijuana use were only available for one study (Slesnick 2005), where marijuana use followed a similar downward trend as other drug use in both groups. However, change scores on tobacco use in this study indicated a small increase in the intervention group as opposed to a decrease in the control group.
Some contradictions in the data on substance abuse related outcomes were also apparent in Slesnick 2009 EBFT; Slesnick 2009 FFT if looking at change scores (mean scores only). For example, while the intervention groups had a significantly greater reduction in mean scores for days of alcohol and drug use compared to SAU, problem consequences of substance use (POSIT) showed a similar downward trend for all three groups, and the largest longer term reduction in substance use diagnoses and the largest longer term increase in percentage of days living at home according to the change scores appeared to be in the SAU group (however, see also above meta-analyses on number of substance use diagnoses ( Analysis 2.15) and number of problem consequences ( Analysis 2.13), favouring different groups at three months).
Self-esteem and depression: in this category also, participants in both groups appeared to improve in all studies apart from an increase in depression among control participants in Hyun 2005 (this study had a very small study population). On reducing depression, change scores offered some support for the intervention in Hyun 2005, Slesnick 2007/08, and Slesnick 2009 EBFT; Slesnick 2009 FFT; however our meta-analysis did not indicate any statistically significant differences at three or six months. In contrast to Slesnick 2009 EBFT; Slesnick 2009 FFT, the Slesnick 2005 control participants appeared to improve slightly more on psychological outcome measures, including depression, according to the change scores.
Reduced use of violence: this was only measured in two studies and baseline scores on this measure were low (Slesnick 2005; and Slesnick 2009 EBFT; Slesnick 2009 FFT). No significant differences were found in meta-analyses and our change scores indicated that self-reported verbal aggression and family violence reduced similarly in all groups at all time points included (longest follow-up 15 months).
Increased contact with family: data on this outcome were only reported in one study (with two intervention groups) (Slesnick 2009 EBFT; Slesnick 2009 FFT) and the results were mixed. While our meta-analysis shows no statistically significant change, the change scores indicate that percentage days living at home reduced in both intervention groups at three months while increasing in the control group. At 15 months, increases were evident across the three groups but the largest overall increase was in the control group.
Other outcomes: outcomes beyond those outlined in our protocol but included in meta-analyses and showing statistically significant effects in our meta-analyses were delinquent behaviours at three months (favouring the intervention) (see Analysis 7.1) and family cohesion (favouring the intervention) (see Analysis 9.1). The measures used for capturing delinquent behaviours were diverse and some of the data were skewed. The overall effect was small and no longer present at later time points (six and 12 months). The change scores for this outcome item were mixed. For family cohesion, three intervention groups and two control groups (from two studies) were included. There is a statistically significant (if small) effect in favour of the intervention groups receiving family therapy. However, according to the longer term change scores the differences between the three groups appear to have diminished by 12 to 15 months.
While longer term intervention effects for family-therapy interventions with runaway adolescents or newly homeless youth in Slesnick 2005, Slesnick 2009 EBFT; Slesnick 2009 FFT and Milburn 2012 appeared on average relatively strong for certain outcome items (mainly related to substance use), the overall findings of the review suggest that the use of structured services in itself predicts positive change on a range of outcomes. Further, the SAU conditions included in this review may have been more effective than specialised interventions in certain outcome categories, for example reduction of sexual risk behaviours and increased contact with family. However, due to methodological limitations discussed we draw this conclusion with caution.
In our primary logic model (Figure 1) we outline the broad components that appear to contribute most to intervention success with street-connected children and youth based on our review of the research literature including qualitative research. Most of the interventions included in this review focused on behaviour change related to participants' current lifestyle or reductions in associated harms, or both. Longer term outcomes beyond narrowly defined problem areas were not evaluated.
Overall completeness and applicability of evidence
Implications for generalisibility (in particular to LMIC contexts) and equity factors
In the following discussion we focus on the applicability of the evidence to other populations of street-connected children and youth, particularly in LMICs, drawing on a brief overview of comparative data. For the purposes of this discussion, the mechanisms for consideration of these questions centre on similarity of interventions, populations and context, and setting between LMICs and the high income country (HIC) populations, interventions and contexts of the studies included in the review, as discussed below. Similarly a discussion by Lavis 2009 on assessing applicability focuses on similarities and differences in populations, contexts and 'on the ground realities'.
The extrapolation tool promoted within Cochrane groups promotes examination of studies for generalisability in more disadvantaged populations according to the following questions: is there good reason to think that it would work with the disadvantaged; it might work in the disadvantaged; no idea if it would work in the disadvantaged; it might be harmful in the disadvantaged; good reason to think it will be harmful in the disadvantaged, where working (benefit) is defined as benefit outweighing harm, and harm is defined as harm outweighing benefit (Pottie 2010). For the purposes of this review we replace the more disadvantaged populations in this model with LMIC contexts.
In this section, we also examine equity related issues in the 11 included studies, focusing on ethnicity, socio-economic status, gender, sexual orientation and disability. We argue that all of the above factors are applicable across HIC and LMIC contexts, and that judgements of generalisability need to be made on a case-by-case basis (see our logic model for HIC and LMIC generalisability, Figure 2).
Lavis 2009, in discussing applicability, highlights differences in service delivery contexts as of crucial importance to considering generalisability. As noted here, several included studies' SAU and therapeutic interventions incorporate specialist referrals and multiagency approaches. An important recent UN report (UN High Commission on Human Rights 2012) on street children recommends many structural factors to governments to improve promotion of the rights of street children. Among these are coordinated child protection and welfare systems, consistent birth registration, multiagency partnership working, adequate financial provision for structures, services and coordination, and addressing stigma and discrimination of street children. Clearly contexts in which such structures currently exist differ in many ways from those contexts in which they do not. Many non-government organisation (NGO) street children interventions in LMICs currently incorporate similar interventions to the SAU services offered in the included studies, but caution should be exercised when applying the results of this review to contexts lacking protective structural arrangements, and generalisability should be assessed on a context-by-context basis.
The provision of a drop-in or shelter service is perhaps the most typical form of intervention available for this population in both HICs and LMICs and is the SAU received by comparison groups in the included studies. Such services commonly provide for basic physical and psychological needs, and sometimes facilitate access to specialist services. However, none of the included studies explicitly set out to examine the effectiveness of such services in themselves. Instead, they focused on highly specialised, time-limited interventions drawing on psychological therapies, which inevitably have higher costs and may not be typical of service provision in either HICs or LMICs though they are arguably more readily available for street-connected populations in the former. Other interventions available in both HICs and LMICs are longer term residential settings for street-connected children and youth. Such services were not represented in the included studies.
The highly specialised therapeutic interventions examined in the included studies are not typical of interventions offered in either HICs or LMICs to support street-connected children and young people and promote reintegration type outcomes. The SAU comparison interventions are more similar to services offered across the world. The overall finding that in many cases SAU participants improved from baseline on the measures used supports the use of these interventions, although a 'measurement effect' whereby the repeated assessments conducted in the studies may in themselves trigger greater awareness and contribute to the outcomes obtained cannot be ruled out (Godin 2008; Morwitz 2004).
Key issues in consideration of the generalisability of this review, which includes studies conducted in HICs, to populations of street-connected children and young people in LMICs focus on a number of issues. These are identified in the introductory background sections of this review as crucial, in particular issues of risk faced by street-connected children and young people, the role of resilience and reasons for young people being street-connected.
All included studies except for Peterson 2006, Milburn 2012 and Rew 2007 recruited participants exclusively through drop-in centres or shelters. Although drop-in youth may be considered more at risk than shelter-based youth (Slesnick 2007/08), comparison of baseline characteristics among participants in the Baer 2007 and Peterson 2006 studies indicated significant differences between a sample of drop-in recruits and a sample recruited from mixed locations, including the street (with the latter more likely to exhibit high-risk behaviours such as heroin use). Only a small minority in the included studies were recruited directly from the street, especially among studies included in the meta-analysis, of which only Milburn 2012 and Peterson 2006 recruited a portion of the research population directly from the street (42% in Peterson 2006; proportion not reported in Milburn 2012). These two studies differed in an important respect. In Peterson 2006, 'parental contact' was the most commonly used exclusion criterion (applying to 60% of youth screened for participation). Since the Milburn 2012 study employed a family intervention, not being away from home for more than six months and having the potential to return home were used as inclusion criteria. Other studies excluded young people with or without significant substance abuse problems. In summary, while the interventions served varied subpopulations of street-connected children and youth, only around 120 participants from one study (Peterson 2006) could be said to have represented a population who may not have had contact with either their families or an agency. Therefore, the results may not be broadly generalisable to children and youth who live on the streets and do not access services, whether in HIC or LMIC contexts.
Reasons for being on the street within the included studies
Of the 11 studies included in this review, only one study (Cauce 1994) explicitly asked participants to state their reasons for leaving home. A further five out of the 11 studies reported histories or experience of physical and sexual abuse and family conflict or violence prior to leaving home (Hyun 2005; Rew 2007; Slesnick 2005; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT). Four studies (Baer 2007; Peterson 2006; Rotheram-Borus 2003; Slesnick 2007/08) did not report on any historical factors such as physical or sexual abuse or family conflict. One study (Milburn 2012) specifically excluded participants who reported abuse, neglect at home or mental health problems. A further study (Tischler 2002) was about homeless families, mainly single mothers with children, among whom domestic violence was the most frequently cited reason for homelessness (44%). It may be that because reasons for leaving home are widespread throughout the broader literature base on youth who are homeless in HICs, studies that are focused on treatment programs for youth who are already street-connected, such as the 11 studies in this review, do not include this information in the relevance of their evaluations. Nevertheless, it must be acknowledged as a limitation of the review with regard to enabling the identification of specific therapies linked to specific psychosocial histories of the population in question (Rew 2007; Slesnick 2005).
Given the lack of specific and consistent reporting of factors contributing to young people leaving home within the included studies, it is difficult to generalise about what the risk factors are that cause or trigger exit from home. A limitation of these studies was the absence, apart from one study (Cauce 1994), of explicit questions regarding the reasons for leaving home. Nevertheless, from the group of participants whose family histories were reported in this study, order, physical abuse, family violence or conflict, and sexual abuse were reported in the lives of participants, in particular. This does correspond with the wider literature on homeless children and adolescents that has found higher rates of family conflict or violence, physical and sexual abuse among homeless and runaway youth compared to non-runaway and homeless populations (Hyun 2005; Rew 2007; Slesnick 2009 EBFT; Slesnick 2009 FFT; Tischler 2002). Higher rates of substance or alcohol use, and high-risk behaviours are also found within this population compared to the domiciled youth population, as is acknowledged within all 11 included studies. It is this aspect of homeless youth experience that is predominantly the focus of evaluation studies. Whilst addressing that the adverse impact of street life for young people is important for achieving good outcomes for this population, there is a significant need for research to focus on these family contexts and earlier prevention strategies that aim to re-engage young people with their families, as identified by Milburn 2012 and in Slesnick 2005: "Because research suggests that family disturbance is highly correlated to the act of running away (Finkelhor 1990, Kufeldt 1992) family therapy is identified as the most important first treatment to evaluate with this population" (p3).
The included studies all aimed to impact high-risk behaviour and life styles of street-connected children and young people. In those that report reasons for street-connectedness, family breakdown and abuse histories feature highly in the backgrounds of the included children and young people. As such, the populations in the included studies may be seen as comparable to the many street-connected children and young people in HMICs where family fragility and breakdown and abuse history are among the reasons for young people leaving home. There is evidence from some LMIC street children projects that supporting child runaways is high among their priorities. As such, there is clear comparability with this subset of LMIC street-connected children and young people with much of the population included in this review, as discussed in more detail below.
Reasons for being on the street in low income countries (LICs)
With regard to comparability of HIC and LICs, a sample of seven studies was drawn from a selection of excluded qualitative studies on street children in LICs. Selection criteria were that the studies specifically stated reasons for children leaving home within the abstract, were published in peer reviewed journals, and geographical locations included Africa, Asia and South America.
Whilst poverty is, arguably, a major trigger for children to come out onto the streets to work in LICs (Abebe 2008), it has been suggested that poverty as the primary reason for children being ‘pushed’ onto the street is not an accurate assessment (Conticini 2007). As with children and young people in HICs, reasons for leaving home reported in these studies state that it is family conflict, parental abuse and family disintegration which trigger a young person’s move onto the street (Henley 2010; Plummer 2007; Praharaj 2008; Raffaelli 2000; Tyler 1986). However, there is a significant difference in the populations of street-connected youth in HICs and LICs. While contested, some literature on homeless children and young people in LICs has distinguished between ‘working street children’ and ‘street children’ (Plummer 2007). ‘Working street children’ are children who have been pushed onto the street as a result of economic hardship but who return home at night after spending their days working on the streets, whilst ‘street children’ are those children and young people who both work and live on the streets (Abebe 2008; Plummer 2007). The Plummer 2007 study, which drew on a sample of 1217 working children and 432 street children in the Sudan, found that the reasons for being on streets were different between working children and street children. Working boys and girls reported poverty and financial hardship as the primary reasons for working whilst, in contrast, the initial qualitative research found that family dysfunction was very widely reported by street children. Moreover, her study found that substance use (glue sniffing) was more highly correlated with street boys and girls than with working boys and girls, and that street boys and girls generally reported more experience of war, familial abuse, and parental death or homelessness.
Similarly, the Henley 2010 study, which drew on a sample population of 1098 children and youth visible on the streets in northern Tanzania, noted a clear trend between ‘part-time’ and ‘full-time’ street children, with full-time street children having higher abusive scores than part-time street children. The Abebe 2008 study of 60 street working children in Addis Ababa found that approximately 80% of working street children returned home at night. This suggests that whilst poverty is a significant trigger for children’s migration to the street in LICs it is also more likely to mean that children are working but returning home. On the other hand, children who leave as a result of abuse or family disintegration tend to live on the street and do not return home. However, other researchers have considered such categorisations overly rigid in light of the complex and shifting circumstances characteristic of street-connected children and young people’s lives (Ennew 2003; Glauser 1997).
The Conticini 2007 study reports that their main finding is that the breakdown of social relationships within the household, and not economic poverty, is the primary cause of child migration to the street (p207), and this is supported by the other studies in LICs cited here. Thus the similarities between HICs and LICs do appear strong in relation to family dysfunction as a causal factor for children and young people leaving home. Studies in HICs do not foreground poverty as a factor for youth homelessness and there is clearly scope for research into the specific links economic hardship within families has as well as the economic activities of homeless youth. However, the common ground that is emerging between HICs and LICs does appear to lie within the sphere of family fragility and dysfunction as a trigger into homelessness and street life.
For those street-connected children and young people from LMICs whose reasons for being on the street include earning a living or contributing to family income, the intervention approach may need to be different, allowing for both ongoing economic support and skills training and education such as offered by some NGOs, for example conditional cash transfer schemes, as well as other support. Income deprivation may not be the usual focus for interventions in HICs but clearly has relevance in the context of insecure labour markets, growing (youth) unemployment and reduced welfare funding, which affect children and youth in HICs worldwide (see for example Karabanow 2010). Similarly, for children and young people who are on the streets for reasons of war, urbanisation or migration, particularly but not exclusively in LMIC contexts (see for example Altanis 2003), the intervention approach would optimally need to address outcomes related to these experiences in addition to harm reduction and reintegration and educational input (Figure 2).
Risks faced by street-connected children and young people in HICs and LMICs
The literature on street children in developing worlds and developed worlds are different. In HICs there are more systematic and scientific-based studies, reviews of interventions and support services for developed world street children, who are often referred to as ‘homeless children’. Conversely, there are many more ethnographic research studies conducted with and on developing world ‘street children’ (McAdam-Crisp 2005; Panter-Brick 2002). Given that the nature of these studies is different, and often looking for or at different things, it is interesting that the risks that street-connected children and young people face on the street are similar. That is, they are at greater risk of increased substance abuse (Towe 2009; Wanzela 2010), sexual exploitation, risky sexual behaviour and sexually transmitted infections (STIs) (Gaetz 2004; Kacker 2007; Kombarakaran 2004), mental health issues (Thabet 2010; Whitbeck 2004) and violence (Save the Children 2005). It should be noted here that according to the data collected, the study populations in the included studies generally represented street-connected children and youth with low to moderate risk profiles.
However there is something to be said about specific risks as a result of socio-geographical and political situations and contexts. In nations where the trafficking of children is evident, street-connected children and young people may be at a higher risk of being trafficked (see Adepoju 2005), although previous assumptions about the high prevalence of trafficking risk have been contested by some research (Thomas de Benitez 2011b). In places where the use of child soldiers is not uncommon, street children may be at risk of being recruited into warfare (Singer 2010). While the latter two examples are prevalent in the developing world, in the developed world street children are often homeless young people who ‘sleep rough’ in cars or with friends, and in different situations often do so without their families. This differs from many developing world spaces where children are on the streets with their families, working and living together; however in some developing world countries this is changing, potentially putting children at greater risk of greater exploitation than if with their families (Adepoju 2005).
Importantly, processes such as rapid urbanisation, slum clearance and rural-urban migration, more prevalent in the LMICs rather than HICs, particularly against backdrops of rapid economic growth and social change, can lead to situations where children and young people find themselves in street-connected circumstances (for example Xue 2009; Young 2004).0 It is beyond the scope of this review to consider mechanisms for prevention of 'street children creation' that arise from rapid urbanisation and population movements, although this could usefully be the focus of future research.
Ethnicity and ethnic minorities
All of the included studies were conducted in HICs, and only two were conducted outside the US (UK and South Korea). Nine out of the 11 included studies were conducted in the US, and two US locations in particular, Albuquerque in New Mexico (Slesnick 2005; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT) and Seattle and Washington (Baer 2007 (location inferred but not explicitly stated); Cauce 1994; Peterson 2006), were over-represented in the sample. Studies from the US represented a variety of ethnic populations, largely white and Hispanic. Whilst data on ethnic background of participants were commonly provided, they usually gave no information on how representative the study populations were of the general population of the study locality (for example the city or neighbourhood from which the research population was drawn) and which ethnic groups could be considered ethnic minorities in their local, as opposed to national, context. Participants in the Korean study reportedly represented a religious (Christian) minority.
Peterson 2006 mentions that participants who were not fluent in English were excluded from the study. No other study mentions this criterion, but they also do not indicate the use of translators or multilingual recruitment methods. Without relevant contextual information, it is difficult to judge how significant this may be. However, in some cities or areas from which participants were recruited (for example, in three studies (Milburn 2012; Slesnick 2005; Slesnick 2009 EBFT; Slesnick 2009 FFT) the dominant ethnic identity among participants was Hispanic), fluency in English as an inclusion criterion could potentially exclude a significant proportion of participants otherwise eligible and representative of street-connected children and youth in that particular area, including recent migrants. Language profiles and citizenship status may have important implications for outreach and service provision. For example, in HICs service users' lack of legal status may restrict the ability or willingness of some NGOs to provide them appropriate services.
The ethnic profile for the three family interventions (Milburn 2012; Slesnick 2005; and Slesnick 2009 EBFT; Slesnick 2009 FFT) was distinct from other types of interventions in that Hispanics represented the largest ethnic minority in each. In the Slesnick studies the second largest ethnic group was Anglo-American, and close in size to the Hispanic populations, whereas in Milburn 2012 the Hispanic population represented 62% and African Americans 21% of the total study population. One hypothesis as to why this might be is the high premium on family connectedness within this community compared to the Anglo-American population in general (see also Slesnick 2002).
Depending on the socio-cultural contexts of interventions, particular interventions may be more or less acceptable. Similarly, family focused interventions might specifically be less popular in some cultural contexts for the inverse of these reasons. Depending on the levels of stigma associated with substance use and sexual behaviour in some cultural contexts, assessments would have to be made as to whether interventions that were focused on these behaviours in particular might be more or less acceptable and appropriate. For example, family conflict involving social exclusion or stigmatisation of individuals with minority sexual identities may be better tackled at community level. This, however, is not a HIC and LMIC distinction but rather one that would need to be made on a context-by-context basis.
Among the included studies, the majority did not report on the socio-economic or educational status, social capital or acculturation indicators among participants or the communities they represented, despite research indicating the importance of such factors in predicting risk behaviours (for example Bantchevska 2008; Slesnick 2002). The only exceptions are Slesnick 2005 and Slesnick 2009 EBFT; Slesnick 2009 FFT, which report on mean family income at baseline. In the Slesnick 2005 study, there was no significant difference between the control and intervention groups. In the 2009 study, the mean income for the control group is relatively low compared to both intervention groups. Educational level was included in a discriminant analysis in Slesnick 2007/08. The absence of collecting and analysing socio-economic data in intervention evaluation research is prominent.
Gender and parenthood
There was a slight over-representation of young men in the studies. This may reflect greater visibility of young men than young women in street situations, and possibly greater likelihood of help-seeking.
Apart from engagement in survival sex, none of the included studies, including those involving street-based (as opposed to shelter-based) populations, appeared to measure potential harms predominantly though not exclusively (see for example Muhrisun 2004) affecting young women, such as unwanted pregnancies, abortions, miscarriages, intimate partner violence, sexual harassment or rape. There may be similar harms predominantly affecting young men, such as physical assault, incarceration or involvement in gang related violence, which confound intervention effects. Such outcomes were not explored in any study. This may be due to the relatively low risk profile of included study populations.
Forming intimate relationships was also not examined in the included studies. Some qualitative data suggest that intimate relationships and starting a family may have a stabilising influence on some street-connected youth (for example Karabanow 2008). Other studies (for example Whitbeck 2009) highlight the often mixed effects of relationships among street living homeless youth who often come from a background of dysfunctional family relations, including backgrounds of sexual and physical abuse. Even when meeting their protection needs, intimate relationships may have complex implications for young women in particular. For example, relationships may further prevent young women from pursuing different pathways of integration through educational and vocational opportunities.
Having children or being pregnant was also not examined in any of the included studies, although a paper by Slesnick 2006b appears to report on parenthood among a subsample of the Slesnick 2005 study. Within their sample of 201 adolescents, 24% either had children or were expecting (23 young men and 25 young women). According to longitudinal data on homeless adolescents in the US, 46.8% of young homeless women were or had been pregnant at the end of the first follow-up period, and 77% at the end of three years (Whitbeck 2009). The majority of the pregnancies reported at first follow-up were not carried to term. In one study (Slesnick 2006b), homeless adolescent parents reported more runaway episodes and engaged in more high-risk sexual and drug behaviours than did non-parents. Mothers engaged in more overall HIV risk behaviours than non-parents and fathers, while fathers engaged in more intravenous drug use.
No studies reported on any type of disability among participant populations.
Quality of the evidence
The majority of the included studies were RCTs and thus the study methodologies can be considered relatively robust. Another strength of the included studies were relatively high retention rates for this population across the studies. However, there were a number of limitations which we highlight below. One considerable limitation was the absence of comparison groups without regular access to services, since eight out of 11 studies recruited participants for both intervention and control groups exclusively from drop-in centres or shelters and the remaining three studies (Milburn 2012; Peterson 2006; Rew 2007) represented participants from mixed settings. Thus, no study in this review compares an intervention with 'nothing' or 'no service', which is the condition lived by most street-connected children and young people around the world. Considering the limited scope of such SAU services in most countries, including the US (for example at the time of the Rotheram-Borus 2003 study, only four shelters were identified in the New York area), the study populations are not necessarily widely representative of street-connected children and youth even in HICs.
Most studies provided limited information for assessing control conditions. Thus, while there was a suggestion of significant variance in the quality and quantity of the services offered by different types of agencies in the service as usual (SAU) conditions, the data provided were not sufficient for robust comparisons across interventions. For example, Milburn 2012, Peterson 2006 and Rew 2007, who recruited participants from mixed settings, including streets, did not specify the control condition.
In many cases, however, SAU appeared to be of relatively high quality. For instance, the drop-in centre which served as the SAU condition in Cauce 1994 offered street involved youth "a drop-in room, free meals, food and clothing banks, health services, a school program, and recreation programs" (p22). Additionally, the centre offered drug and alcohol counselling, group sessions on self-esteem, sexuality, parenting and job skills, as well as individual case management. Several of the SAU conditions provided counselling services. Considering the fact that interventions were usually narrow in scope, participants in the intervention groups were also likely to access SAU to meet other needs. However, few of the included studies reported having systematically controlled for similarity between the two groups in terms of using 'control' services, apart from limited data in some studies (for example number of counselling sessions). Furthermore, it is impossible to know to what extent positive outcomes in the intervention group were contingent upon simultaneously receiving SAU.
Furthermore, in so far as many of the interventions were time limited and specialised, as opposed to more permanent and comprehensive services which may have been more familiar to the research population, it could be argued that control conditions may have had distinct advantages vis a vis intervention conditions. However, service delivery related confounders (for example service satisfaction, level of engagement) were usually examined only in the context of the intervention condition. A further potentially confounding factor in some studies was the fact that, as in the case of a number of interventions, some of the agency-based services were contained within their location while others involved referrals or joined-up working with external service providers, depending on individual needs. The impact of external services were not examined in any of the relevant studies.
Finally, since interventions commonly took place in the shelter or drop-in centre from which participants were recruited, there was a high likelihood of contamination between intervention and control groups in most of the studies. Several of the authors draw attention to this fact. Participants in the intervention and control groups were likely to belong to the same peer network and could therefore affect each other's behaviours in either direction. In summary, it is very difficult to isolate intervention effects from SAU, especially in studies where the two conditions operated under the same roof. (Contamination has similarly been recognised as a problematic issue with street-based participants sharing living quarters (Rew 2007).)
Four studies had a follow-up period exceeding six months (Milburn 2012; Rotheram-Borus 2003, Slesnick 2005; Slesnick 2009 EBFT; Slesnick 2009 FFT), while five had a follow-up period of three months or below (Baer 2007; Cauce 1994; Hyun 2005; Peterson 2006; Rew 2007). The longest follow-up was 24 months (Rotheram-Borus 2003); however the longest follow-up for which raw data were available was 15 months (Slesnick 2009 EBFT; Slesnick 2009 FFT). The longest follow-up with raw data from more than one study and one outcome was six months, which further limits the robustness of any predictions on the basis of included data.
While outcome categories were considerably homogeneous across the studies, as noted above, there was a lack of consistency between type of intervention and type of outcomes measured. Added to the heterogeneity of outcome measures discussed above, this further limited the amount of data available for meaningful comparison. For example, among studies involving a social cognitive or behavioural intervention, two (Rew 2007; Rotheram-Borus 2003) measured exclusively cognitive-behavioural outcomes, one (Hyun 2005) exclusively psychological outcomes, and three (Baer 2007; Milburn 2012; Peterson 2006 ) exclusively behavioural outcomes. Studies by Slesnick measured both psychological and social functioning outcomes irrespective of type of intervention (that is family therapy, Community Reinforcement Approach and HIV prevention) and additionally either family functioning or cognitive-behavioural outcomes depending on the intervention. Cauce 1994 and Tischler 2002, who both evaluated a multicomponent intervention, measured psychological and social adjustment outcomes. Further, outcomes within these broader categories varied. For example, the family interventions focused on different behavioural outcomes (Milburn 2012 on substance use, delinquent behaviour and sexual risk behaviour; Slesnick 2005; Slesnick 2009 EBFT; Slesnick 2009 FFT on substance use, delinquent behaviour and family functioning). Thus, there is a lack of consistency in the choice of outcomes across the studies.
Irrespective of theoretical orientation, it is not obvious what the relevant measurable effects of an intervention should be. In addition, the line between outcomes and process factors is blurred. For example, although several interventions included a motivational element, defined as encouraging ‘readiness to change’ (Peterson 2006), they did not treat motivation to change as an outcome. For example, Peterson 2006 measured ‘stage of change’ only at baseline. Other relevant factors identified in the research literature include, among many others, goal setting and decision making (Lightfoot 2011). These appear to be target elements of interventions included in the review but were not treated as outcomes. Some interventions, such as CRA employed in Slesnick 2007/08, define concrete behavioural targets, such as an increase in positive (as opposed to risk-inducing) social activities and peer relationships, which were not translated into outcomes apart from the aggregate measure of 'social stability'. Few studies measured factors specific to street-connected populations and highlighted by both ethnographic (for example Karabanow 2008) and quantitative or mixed methods studies (for example Whitbeck 2009) as crucial for exit from street life.
We can infer that many of these potential ‘process factors’ were nevertheless relevant to most interventions, as well as SAU. Although they may be considered moderating or mediating factors in relation to concrete outcomes such as reduced substance use, they do not appear to differ in a fundamental sense from constructs such as self-esteem. Furthermore, in order to evaluate intervention effectiveness, it might be considered important to measure whether the intervention appeared valid as a method with a particular research population (for example whether a motivational intervention in fact increased motivation). One of the challenges of evaluating effectiveness of psycho-social interventions derives from the fact that they typically consist of multiple treatment components which are difficult to quantify. Meta-analyses of common psychological therapies have shown that common process factors, especially therapeutic relationship variables, may account for 30% of the variance in treatment outcomes for adults, above and beyond the 15% of variance which is accounted for by therapeutic techniques (Lambert and Barley 2002, cited in Karver 2006). None of the included studies controlled for treatment variables such as the quality of the therapeutic relationship or group cohesion.
Viewing outcomes in a narrow context or in isolation from each other, and without locating them in the real, everyday experiences of study participants, may lead to misleading conclusions. For example, a study by Ferguson 2008 found an increased number of sexual partners among youth taking part in a social enterprise intervention, in marked contrast with a (non-randomised) control group from the same drop-in centre who significantly reduced their number of sexual partners over the same period of time. Some of their qualitative data suggested that this fact could be explained by increased self-confidence among intervention participants, which by itself may be considered a desired outcome. Similarly, a cross-sectional study by Booth 1999 could not confirm an expected relationship between increased knowledge about HIV/AIDS or perceived likelihood of infection and sexual risk behaviours; on the contrary, youth with higher levels of knowledge engaged in more risk behaviours, possibly reflecting "a realistic appraisal of their increased risk" (p1302). Of the included studies, those by Slesnick (Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT) enable the most comprehensive comparisons across a relatively broad range of outcome categories. However, contradictory outcome findings were explored to a limited extent.
In summary, while the studies covered important outcomes, predefined outcomes often directly transposed from research with very different study populations, and usually with limited relevance to a particular intervention, may not adequately reflect the full range of risks that street-connected children and youth are likely to encounter. Conversely, important intervention benefits may go undocumented. More work is required to develop appropriate research tools in this area of research, ideally drawing both on bottom-up participatory methods (as exemplified, for example, in Ferguson 2008b) and broader theories of change.
As depicted above, despite overall homogeneity of outcomes measured, the measurement tools and, consequently, outcome components reported in the studies were very heterogeneous. The majority of the measures used were validated and data on their reliability was made available in the study publications. However, measures were not commonly validated in the context of studying homeless or street-connected young people. The measurement tools employed were typically self-report, due to practical and ethical reasons. The potential biases inherent in self-report measures are well-known and were highlighted by several authors. For example, under- and over-reporting may occur due to social desirability or trust issues. This is compounded by potential problems of recall. For instance, it could have been challenging for some youth to calculate and report the numbers of times they 'used alcohol' or 'had sex' in the past three months (Milburn 2012 [pers comm]) partly due to chaotic lives marked by high rates of substance abuse, which may impact negatively on accurate recall (for example Rew 2007).
The quality of the reporting of outcomes for review purposes was varied. Mean and standard deviation data for some outcomes was included in six out of 12 study publications (Baer 2007; Cauce 1994; Hyun 2005; Peterson 2006; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT). However, some of these only included raw data for outcomes favouring the intervention (other outcomes were reported narratively). The unpublished data were available on request in all instances. In some instances different studies using the same measures reported different outcome items, which raises the possibility of selective reporting (it should be noted that the number of outcome items measured was very large in some studies). Three publications presented data in graph or composite form (Milburn 2012; Rotheram-Borus 2003; Slesnick 2005), and we were able to obtain relevant raw data for two of these (Milburn 2012; Slesnick 2005). No other unpublished data from past or ongoing studies were made available to us.
It has been argued that the analysis of data from complex social interventions calls for sufficiently sophisticated statistical methods in order to produce meaningful evidence of "how programs affect individuals, who is most affected, and under what circumstances" (Lipsey 2000, p362). While statistical methods to capture this level of complexity have been evolving in recent decades, research practice is lagging behind methodological advances (see Lipsey 2000). The included studies were considerably varied in their choice of statistical methods. Below we highlight some examples.
Population heterogeneity may significantly contribute to the variance in outcomes, and street-connected children and youth typically represent a diverse population with multiple needs and relatively high levels of comorbidity (Slesnick 2006). Most included studies provided some analysis of baseline differences. One way of accounting for variance in the study population is to use propensity scores, as exemplified in Rotheram-Borus 2003 (non-randomised study sample). The authors calculated propensity scores for each participant, based on 45 baseline characteristics, which were used to classify participants into five subgroups. Since significant differences emerged between the control and intervention groups in terms of risk profile, those with the least and those with the most sexual and substance use risk acts were excluded from the analyses. The remaining three groups were pooled for data analysis purposes.
Grouping individual participants according to their change profile (for example positive, negative, no change) and performing analyses on predictors of a particular direction of change, as exemplified in Slesnick 2007/08 described below, would seem to be a particularly useful form of analysis. The value of such analyses is evident particularly in the context of psychosocial interventions with heterogeneous, non-clinical populations (Lipsey 2000), and can usefully complement interpretations based on group level mean scores and standard deviations.
For example, the only differential (though statistically non-significant) trend found for mean scores related to sexual health risk outcomes, in Slesnick 2007/08, was a slightly greater (though statistically non-significant) improvement in the frequency of self-reported condom usage in the intervention group, with age as a mediating variable. In practical terms, the change appears marginal. The mean score for overall HIV risk progressively decreased for both groups.
In contrast, a discriminant function analysis by the authors revealed that in terms of overall HIV risk, 26.5% of participants (in either group) experienced a statistically significant reduction, 53.6% no statistically significant change, and 20% a statistically significant increase. Further, in a comparison of nine associated attributes, including demographic factors and treatment condition, baseline HIV risk behaviour emerged as the strongest predictor of HIV risk behaviour; those who had the highest HIV risk behaviour score at baseline significantly reduced their HIV risk behaviour over time. (A similar analysis is not offered in the context of other outcome categories.)
While not offering support for the intervention, the result is encouraging since it suggests that high-risk individuals benefited the most from any form of structured support (although the study design does not allow control for a general maturational effect). On the other hand, they were also more likely to drop-out from the study (see section on attrition analyses). In the absence of comparable analyses, the finding cannot be generalised.
Among the included studies, Slesnick 2005 and Slesnick 2009 EBFT; Slesnick 2009 FFT stand out positively in terms of their comparative research design (replicating the same intervention with two different populations, comparing two different interventions) as well as comprehensive and longitudinal outcome evaluation, combined with statistical analyses of potentially moderating factors (including both demographic variables and process factors such as treatment attendance). However, the studies did not report on qualitative process evaluation, and the analyses as well as outcome measures used or reported were not entirely consistent across the different studies. For example, different parts of the data (sometimes combining data from two studies) are subject to very varied types of analyses which are reported across several publications (for example Slesnick 2006; Slesnick 2006c) and not always cross-referenced. Moreover, discrepancies between individual and composite analyses are not discussed. Prof Slesnick has directed several large projects measuring varied outcomes, reportedly including process factors not included in analyses published so far (Slesnick 2012); and it is possible that future research publications will address current gaps in the data. However, we were unable to confirm if any of the studies were ongoing.
Interpretation of results
Analyses were usually based on mean scores and standard deviations of participant scores on a particular scale. Most studies reported findings in terms of statistical significance or non-significance. Despite utilising several clinical scales, there was little discussion around the clinical significance of particular scores. Furthermore, there were no attempts to evaluate outcomes within the real-life contexts or subjective perspective of study participants in any of the studies. For some measurement tools (for example delinquency scales) little information was available, making interpretation of results difficult. Outcome scores were also not compared to not street-connected populations, although some studies (for example Milburn 2012) did offer such comparisons for baseline scores. Finally, as recognised by a number of authors, ambiguity of findings with this study population highlights the need for more extensive qualitative and quantitative process evaluation to help explain and interpret results. Evaluations need to go beyond merely assessing service-user satisfaction.
Potential biases in the review process
Agreements and disagreements with other studies or reviews
Our literature search identified two relevant reviews with inclusion criteria sufficiently similar to the current review (Altena 2010; Slesnick 2009). However, these reviews also included non-randomised studies and studies without a control group. Eight of the 11 studies included in the current review were included in Slesnick 2009 (Baer 2007; Cauce 1994; Hyun 2005; Peterson 2006; Rew 2007; Rotheram-Borus 2003; Slesnick 2005; Slesnick 2007/08; Slesnick 2009 EBFT; Slesnick 2009 FFT), and five in Altena 2010 (Baer 2007; Cauce 1994; Hyun 2005; Peterson 2006; Slesnick 2007/08). Studies included in the current review but not in any of the two other reviews were Tischler 2002 and Milburn 2012. Similar to Altena 2010, we did not identify relevant studies from LMICs. The broad conclusions of the current review are in agreement with those in Altena 2010 and Slesnick 2009.
We also agree with the overall conclusions in Naranbhai 2011, who included three studies overlapping with this review (Rotheram-Borus 2003; Slesnick 2005; Slesnick 2007/08). Ross 2006 identified only two studies with street-connected children and youth in LMICs. These studies, as did the Ross 2006 review, had a primarily HIV/AIDS prevention focus and therefore were not considered for inclusion in this review.
Implications for practice
In most studies, outcomes were similar for both intervention and control groups. Thus, decisions on preferred mode of practice must rest on other considerations, such as feasibility, economic effectiveness, service user preference, long term sustainability, etc. However, control conditions in the included studies were often of high quality. Not surprisingly, positive effects were more pronounced in interventions targeting needs not covered by service as usual (for example involvement of families for young people residing in a runaway shelter, or provision of therapy for young children in a shelter for homeless families).
It is unclear to what extent the types of interventions such as those included in this review are generally available to street-connected populations in the relevant countries or localities, and how representative they are of the most common types of interventions offered by service providers. Since most were delivered by relatively highly qualified professionals (for example counsellors or therapists), we may assume they are unlikely to be integrated into typical service provision. Although family-oriented therapy appeared partially effective with certain newly homeless or runaway populations (Milburn 2012; Slesnick 2005; Slesnick 2009 EBFT; Slesnick 2009 FFT), referral to mainstream services may not be as effective as delivering the intervention in collaboration with a service setting such as a shelter or drop-in service. Cost and feasibility evaluations must take this into account.
In many contexts, the finding that in most cases the therapeutic intervention did not produce better results than service as usual might assist planning and development of policy and service delivery.
Implications for research
Although most studies included in the review were grounded in a well-defined theoretical framework, the studies were commonly the first of their kind to test a particular intervention or an outcome measure in the context of street-connected children and youth. In this respect, all the studies reviewed provide valuable indicators for future research, and demonstrate that some specialised interventions are both viable and, in some respects, effective in application to certain subpopulations of street-connected children and youth (especially runaways with connections to their families). However, many of the study designs appeared to be determined, above all, by theoretical literature on the particular type of intervention employed in response to a set of narrowly defined problems (for example substance abuse). In contrast, the findings of our review suggest that characteristics of the study population and other process factors may be more relevant to achieving positive outcomes than the technical or theoretical underpinnings of an intervention.
All of the included studies were conducted in high income countries. Across all socio-economic and cultural contexts, there is a need for more research which includes control groups not in receipt of services, as well as research focusing on street-recruited as opposed to agency-recruited populations. Further, we found no evidence that service as usual conditions had been robustly evaluated and, as such, a key recommendation for further research is that such services in all geographical locations are evaluated in comparison with no active intervention. Further process evaluation data, in particular as regards the nature of engagement or motivational strategies, would also add considerably to understanding within the field.
The nature of control conditions in future research needs to be adequately captured and reported. In addition, it may be useful to employ a research instrument which will provide adequate comparative data on participants' experiences of the intervention and control conditions.
Overall, on the basis of our findings, we encourage research which is more directly guided by the characteristics and concerns of the research population in question, and builds on the findings from previous and ongoing research involving participation of street-connected children and youth including qualitative research literature. For example, there is scope for thinking more creatively around the conceptualisation and measurement of relevant outcomes for interventions with this study population. Researchers should also attempt to provide a clear theoretical and methodological rationale for the outcomes selected for measurement. If measuring standard outcomes, the use of standardised tools comparable to other studies would positively contribute to the accumulation of research evidence.
With this heterogeneous study population, calculating the percentage of participants who improved on a particular outcome, as opposed to the percentage of participants who deteriorated or remained unchanged, would seem a potentially useful way of analysing findings. Finally, more attention should be paid to analyses of potential demographic confounders and process factors, considering the complex nature of psycho-social interventions in varied contexts. Considerable gaps remain in our understanding of the relationship between contextual factors, interventions and outcomes. Logic models such as developed in this review (Figure 1 and Figure 2) and existing qualitative and quantitative research on street-connected children and youth (for example Ferguson 2007; Karabanow 2008) could aid researchers in clarifying their conceptual frameworks in this regard.
Thanks are due to the advisory group members who have contributed their wisdom to the development of the protocol: Sarah Thomas de Benitez, Harriot Beazley, Nicole Howard, Christopher Hands, Petra Englebrecht, Anna-mai Estrella, Angeles Fiallo Montero, Claudia Stoicescu, Joe Walker, Louise Meincke, Damon Barrett and Kirstin Mitchell.
Thanks also to Tamara Rader from the Campbell Collaboration International Development Co-ordinating Group and the Cochrane Musculoskeletal Group for assistance in developing the search strategy, and to Jodie Doyle of the Cochrane Public Health Group for assistance with the process.
Thanks are also due to Jean Paul Boddu of Madras Christian College for assistance with screening, and to Jenny Witherden of Canterbury Christ Church University for assistance with institutional project management.
Data and analyses
- Top of page
- Summary of findings [Explanations]
- Authors' conclusions
- Data and analyses
- What's new
- Contributions of authors
- Declarations of interest
- Sources of support
- Differences between protocol and review
- Index terms
Appendix 1. Descriptive map of studies
Interventions for promoting reintegration and reducing harmful behaviour and lifestyles in street-connected children and young people
Report on internal descriptive map.
The original search for the systematic review was broad based and inclusive and retrieved 15,995 unique references. These were screened by two people into the following categories: Effectiveness study: probability of inclusion, Evaluation study with other study designs, Ethnography or other qualitative studies, Excluded: related to street children but not evaluating effectiveness, Narrative Review, Excluded: not related with street children, Non-English language studies. Fifty-seven studies were coded by one or both reviewers as ‘Effectiveness study: probability of inclusion’. For the mapping exercise, full text was obtained for all of these, where available. In addition, non-English language studies and forty references identified through narrative reviews were evaluated according to the same categories. As a result, six more studies (seven references) were added to the mapping.
The research team developed a pilot coding scheme with 50 coding categories which was implemented using SPSS to describe and categorise the studies. This framework was based on the conceptual thinking underlying the street children review and explicated in the review protocol. Thus codes included study location, income status of country, age, gender, religion, ethnicity, disability, sexual orientation status, numbers of participants, study design, objectives, and intervention details. The criteria for in the mapping exercise were otherwise the same as in the main review, but a broader range of study designs (e.g. cross-sectional, 1-group pretest-posttest) were included.
Sixty references were included in the mapping exercise and full text was sought for all of them. The majority were published journal articles. Out of the sixty publications reviewed by full text, 23 were excluded on various grounds, e.g. for not focusing on street children or not evaluating outcomes (full text was unavailable for three of these references). Five studies were associated with two references, bringing the total number of studies considered for mapping to fifty-five. Eleven of these studies (12 references) were included in the review.
Two of the mapped studies (three references) did not report on outcomes, but were deemed relevant for the mapping since they pertain to LMI countries and included useful process evaluation (these studies are excluded from the frequency analysis). One of these reports on a qualitative cross-regional comparison study, and the other provides an overview of an HIV/AIDS prevention programme for street youth in Uganda.
The following overview draws on a statistical frequency analysis of the remaining thirty studies (34 references). More detailed data, including missing values where relevant, are available in tables from the authors. Numbers in brackets refer to the number of studies.
The included publications were published between 1985 and 2012, with the majority being published from 2001 onwards, and the highest number of studies (6) published in 2007, including two out of the three MIC studies included in the map. The third MIC study was published in 2010.
Countries and regions
Out of the total of thirty studies included in the map, twenty-seven were conducted in high income (HI) countries and three in middle income (MI) countries (two in Africa and one in Latin America). The only relevant study from a low income (LI) country is the study from Uganda mentioned above. The overwhelming majority of the studies (22) were conducted in the USA. Other study countries were Canada (2), Australia, Brazil, Egypt, Korea, Nigeria and UK (one study in each).
Approximately half of the studies (16) examined children and young people in the 11-24 age-group (of these, two studies only described the average age of participants). Seven studies looked at the age-group between 11 and 18, two studies at participants aged 15-18 and one study at participants aged 11-14. Only four studies included children under the age of 10, two of these being studies of homeless families with children in HI countries. Two out of the three MIC studies recruited participants in the 11-24 age-group, while one Egyptian intervention was aimed at boys aged 7-15.
Demographic data and equities
The majority of the studies (27) reported a mixed sample of males and females. One US study had an all-female sample, and two studies (one Korean and one Egyptian) only included street-connected boys.
Seven studies did not indicate ethnic minority status for participants. The majority of studies conducted in HI countries, especially in the USA, reported on demographic data and included participants of various ethnic backgrounds.
Data on sexual orientation of participants were reported in five studies. Data on disability status was not reported in any study.
The majority (27) of the studies included in this map (aside from the two excluded from the current analysis) were quantitative, while three employed mixed methods. We classified eight studies as randomised controlled trials, two as controlled before and after studies, and one as a quasi-randomised trial. All of these studies were conducted in HICs (see Table 1 below) and were included in the review following further screening against criteria specified in the review protocol.
Types of control groups ranged from those receiving treatment/ services as usual (SAU) (11), to groups receiving no treatment/ unspecified SAU (4), or a comparison or control group receiving an alternate form of treatment (3). Four studies included a comparison group from a different setting.
Reasons for excluding studies from the review are summarised in the Characteristics of excluded studies table in the review. A common reason was the lack of a comparable control group, e.g. due to selection bias. Nine studies did not include any control or comparison group  and were thus excluded from the review.
Longest follow-up points ranged from less than three months (7) to 3 months (4), 5 months (1), 6 months (4), 9 months (1) one year (5) and over one year (4). Relevant data was unavailable for four studies.
The most typical type of intervention was multi-component (15), e.g. consisting of an educational, health and counselling or other type of intervention. The next most common type of intervention was focused on HIV/ AIDS education (4). Other interventions were therapeutic (5), focusing on sexual health (3), drug and alcohol abuse (2), or educational (1). Process factors regarding aspects of delivery of the interventions, were highlighted in approximately half of the studies.
A range of outcomes were measured in the studies. For MI countries, the most commonly measured were education and empowerment. Also measured in these studies were family reintegration, reduced risk behaviours, mental health outcomes and employment, all of which are relevant to the review.
The map highlights a paucity of robust evaluations conducted in low and middle income countries. Even within HIC evaluations, some do not utilise robust methods. Where evaluations are conducted in LMICs the study designs tend towards the more observational. Intervention types evaluated vary, and many are multi-component, although there is consistent focus on addressing reintegration and welfare promotion related factors. However, it should be noted that due to imbalance in the evidence base regarding studies from these countries, the sample we are describing is small. Future research needs to address this gap in knowledge with more evaluated intervention studies that acknowledge the specific socio-economic conditions of HI, MI and LI countries.
Bragge P ; Clavisi O ; Turner T; Tavender E ; Collie A ; Gruen R (2011) The ‘Global Evidence Mapping Initiative: Scoping research in broad topic areas’ BMC Medical Research Methodology 11, 1 rec.no 92 Issn:14712288
 As defined by the World Bank
 Baer 2007
 Booth 1999, Olley 2007, Peterson 2006
 Beharie 2011, Davey 2001, Slesnick 2009 EBFT; Slesnick 2009 FFT
 Gleghorn 1997, Sears 2001, Thompson 2002, Upshur 1985
 Auerswald 2006, Barber 2005, Dalton 2002, Haley 1998, Hosny 2007, Schram 1991, Scivoletto 2010, Steele 2001/03, Stewart 2009
Appendix 2. Search result summary & strategies
1. Search date: March 6, 2012
Detailed Search strategies
Cochrane Library Issue 7, 2011
Search Name: Street kids Cochrane Library
Save Date: 2011-11-03 12:51:56.94
#3street near kids
#4street near youth
#6MeSH descriptor Child, Abandoned, this term only
#7MeSH descriptor Homeless Youth, this term only
#8MeSH descriptor Runaway Behavior, this term only
#9MeSH descriptor Child, Unwanted, this term only
#10street near child
#11Criancas de rua
#12Meninos de rua
#13street near urchins
#14Pavement near dweller
#15railway near children
#16unaccompanied near migrant
#17unaccompanied near minor
Database: Ovid MEDLINE(R) In-Process and Other Non-Indexed Citations and Ovid MEDLINE(R) <1948 to Present>
1 "Homeless Youth"/ or (homeless$ adj2 (child$ or youth$ or young or teen$ or adolescen$)).tw
2 "Runaway Behavior"/
4 (street adj4 kids).tw.
5 (street adj4 youth).tw.
6 Child, Abandoned/
7 abandoned child$.tw.
8 Child, Orphaned/
9 (orphan$ adj3 child$).tw.
10 Child, Unwanted/
11 (unwanted adj4 child$).tw.
12 (street adj4 child$).tw.
13 Criancas de rua.tw.
14 Meninos de rua.tw.
15 (street adj3 urchins).tw.
16 (Pavement adj3 dweller$).tw.
17 (railway adj2 children).tw.
18 (unaccompanied adj4 (refugee$ or migrant$)).tw.
19 (unaccompanied adj4 minor$).tw.
21 (niños adj3 calle).tw
22 (ninos adj3 calle).tw
23 (enfants adj3 rue).tw
24 (jeunes adj3 rues).tw
39 young people.mp.
48 Infant, Newborn/
63 exp Vulnerable Populations/
64 human trafficking.mp.
67 Homeless Persons/
69 (human adj4 traffick$).tw.
70 (sex adj4 trade).tw.
71 (sex adj4 work$).tw.
73 (62 and 72)
74 73 or 25
75 exp animals/ not humans.sh.
76 74 not 75
Database: OVID Embase <1947 to Present>
1. (homeless adj3 youth).tw.
2. runaway behavior/
4. (street adj3 child$).tw.
5. (street adj3 kid$).tw.
6. abandoned child$.tw.
7. (orphan adj3 child$).tw.
8. (unwanted adj3 child$).tw.
9. Criancas de rua.tw.
10. Meninos de rua.tw.
11. (street adj3 urchins).tw.
12. (Pavement adj3 dweller$).tw.
13. (railway adj2 children).tw.
14. (unaccompanied adj4 migrant$).tw.
15. (unaccompanied adj4 minor$).tw.
17. (ninos adj3 calle).tw.
18. (Enfants adj3 rue).tw.
19. (enfants adj3 rues).tw.
33. (young$ adj3 person$).tw.
40. vulnerable population/
41. human trafficking.tw.
46. (human adj4 traffick$).tw.
47. (sex adj4 trade).tw.
48. (sex adj4 work$).tw.
50. 38 and 49
51. 20 or 50
CINAHL via Ebscohost
S24 S8 or S23
S23 (S9 or S10 or S11 or S12 or S13 or S14 or S15) and (S16 or S17 or S18 or S19 or S20 or S21)
S22 S9 or S10 or S11 or S12 or S13 or S14 or S15
S21 "teen*" S20 "youth"
S19 (MH "Young Adult")
S18 (MH "Adolescence")
S17 (MH "Latchkey Children")
S16 (MH "Child") S15 "sex work"
S14 sex work
S13 "human traffic*"
S12 TI beg* OR AB beg*
S11 (MH "Prostitution")
S10 (MH "Vulnerability")
S9 (MH "Homelessness") OR (MH "Homeless Persons")
S8 S1 or S2 or S3 or S4 or S5 or S6 or S7
S7 "street youth"
S6 "unwanted children"
S5 (MH "Orphans and Orphanages")
S4 (MH "Child, Abandoned")
S3 (MH "Runaways")
S2 "street kids"
S1 "street children"
1. Runaway Behavior/
2. (homeless adj3 youth).tw.
4. (street adj3 child$).tw.
7. (abandoned adj3 child$).tw.
8. (unwanted adj3 child$).tw.
9. Criancas de rua.tw.
10. Meninos de rua.tw.
11. (street adj3 urchins).tw.
12. (Pavement adj3 dweller$).tw.
13. (railway adj2 children).tw.
14. (unaccompanied adj4 migrant$).tw.
15. unaccompanied minor.tw.
17. (ninos adj3 calle).tw.
18. (Enfants adj3 rue).tw.
19. (enfants adj3 rues).tw.
23. (young$ adj3 person$).tw.
26. Juvenile Gangs/
31. (sex adj4 trade).tw.
32. (sex adj3 work$).tw.
35. (pan adj3 handle$).tw.
39. 29 and 38
40. 20 and 39
41. 20 or 39
42. 20 or 39
ERIC via Ebscohost
S24 or S26
S24 or S26
(S11 or S12 or S13 or S14 or S15) and (S23 and S25)
S11 or S12 or S13 or S14 or S15
S2 or S3 or S4 or S20
S21 or S22
AB children or child
(MH "Child, Abandoned") OR (MH "Latchkey Children")
""sex N3 work""
"sex N3 trade"
"sex N3 trade"
"human N4 traffick"
(MH "Special Populations")
"street N3 urchin"
"niños N3 calle"
"Enfants N3 rues"
"Enfants N3 rue"
“"ninos N3 calle""
"niños N3 calle"
(MH "Orphans and Orphanages")
AB street children OR AB street children OR AB Homeless children OR AB homeless youth OR AB Runaway teenagers OR AU Runaway children
(MH "Homeless Persons") OR (MH "Homelessness")
Sociological Abstracts via Proquest
(((KW=((vulnerable population*) or homeless* or squatter*) or KW=((sex work*) or (sex trad*) or (human traffic*))) or(DE=("homelessness" or "child poverty" or "deinstitutionalization" or "skid row" or "squatters" or "underclass" or "urban poverty" or "prostitution")) or(DE=("vulnerability" or "prostitution"))) and((KW=((street kid*) or (street child*) or (homeless youth)) or KW=((child abandon*) or (abandon* child*) or (missing child*)) or KW=((homeless child*) or runaway or throwaway) or KW=((Criancas de rua) or (Meninos de rua) or Tikyan) or KW=((unaccompanied minor$) or (child migrant) or (migrant child$)) and KW=(unwanted child)) or(DE=("adolescents" or "children" or "infants")) or(KW=(youth or juvenile or kid)))) or((DE="runaways") or(DE="orphans"))
Social Work Abstracts via Ebscohost
S21 (S1 or S2 or S3 or S4 or S5 or S6 or S7 or S8 or S9 or S10 or S11 or S12 or S13 or S14 or S15 or S16 or S17 or S18 or S19 or S20)
S20 unaccompanied minor S19 railway children S18 railway children S17 Meninos de rua S16 Criancas de rua S15 tikyan S14 tikyan S13 street urchin S12 homeless and youth S11 street youth S10 street youth S9 child prostitute S8 orphans S7 abandoned children S6 unwanted children S5 runaway S4 runaway S3 street children S2 street children S1 homeless youth
Database: Ovid HealthStar
1. "Homeless Youth"/
2. "Runaway Behavior"/
4. (street adj4 kids).tw.
5. (street adj4 youth).tw.
6. Child, Abandoned/
7. abandoned child$.tw.
8. Child, Orphaned/
9. (orphan$ adj3 child$).tw.
10. Child, Unwanted/
11. (unwanted adj4 child$).tw.
12. (street adj4 child$).tw.
13. Criancas de rua.tw.
14. Meninos de rua.tw.
15. (street adj3 urchins).tw.
16. (Pavement adj3 dwellers).tw.
17. (railway adj2 children).tw.
18. (unaccompanied adj4 migrant$).tw.
19. (unaccompanied adj4 minor$).tw.
35. young people.mp.
44. Infant, Newborn/
59. exp Vulnerable Populations/
60. human trafficking.mp.
63. Homeless Persons/
65. (human adj4 traffick$).tw.
66. (sex adj4 trade).tw.
67. (sex adj4 work$).tw.
70. 58 and 69
Appendix 3. Change scores
We calculated the change scores by subtracting mean scores at the relevant time point from mean scores at baseline. Directions of the change are explained in the text. They were calculated for all outcome items and time-points included in the meta-analysis. Further, ‘longer term’ change scores were calculated for these items at the longest follow-up point not included in the meta-analysis.
For outcome items not included in the meta-analysis (see Appendix 4 for a list of these outcome items by study), change scores were similarly calculated using relevant time points (where raw data was available) or the longest follow-up point (where only percentage data were available). These scores are presented separately under relevant subsections below.
P-value and F-values for some of these trends can be found in the relevant studies. However, it was beyond the scope of this review to confirm or calculate these values for each of the outcome items discussed here. For the sake of consistency, we have chosen not to refer to the values reported by study authors.
The numbering of the outcome categories follows the numbering in Effects of interventions. The sub-outcomes are numbered in the order they appear here.
1. Safer or reduced sexual activity
1.1 Number of times had sex in last 90 days (3 & 6 months)
Change scores: 3 months: Milburn 2012: 0.56 (intervention), 1.38 (control). Slesnick 2007/08: 0.02 (intervention), -0.05 (control). 6 months: Milburn 2012: 3.26 (intervention), 0.23 (control). Slesnick 2009 EBFT; Slesnick 2009 FFT: 0.08 (intervention), -0.24 (control). The changes show mixed patterns but overall are marginal to small.
Longer term change scores. Number of times had sex in last 90 days: Milburn 2012: 12 months: 4.39 (intervention), -0.53 (control). The figures appear to be in benefit of the control group, although the validity of this measure in indicating high-risk behaviour is unclear.
1.2 Number of sexual partners in last 90 days (3 & 6 months)
Change scores: 3 months: Milburn 2012: 0.15 (intervention), 0.22 (control). Slesnick 2007/08: -0.05 (intervention), -0.09 (control). 6 months: Milburn 2012: -0.25 (intervention), -0.06 (control). Slesnick: -0.11 (intervention), -0.19 (control). The changes appear marginal in all groups.
Longer term change scores. Number of sexual partners in last 90 days: Milburn 2012: 12 months: -0.36 (intervention), 1.02 (control). The figures appear to be in benefit of the control group. Among significant intervention effects in this study, this trend shows the most divergent long-term pattern between intervention and control groups.
In Milburn 2012, an unexpected result (Milburn 2012 [pers comm]) was that while the (already low) number of partners differentially reduced in the intervention group, for number of times had sex there was a reversal in trends at 6 months: compared to the 3 month scores, the mean increased 5.8 times in the intervention group, and reduced 6 times in the control group. These data are difficult to interpret but may mean that intervention participants had fewer partners but had sex more often. In terms of this review, fewer partners may entail less risk so this may be seen as a positive result.
1.3 Unprotected sex/ condom use
In Slesnick 2007/08, change scores for condom use frequency (self/ partner) were: 3 months: 0.16 (intervention), 0.24 (control). 6 months: 0.15 (intervention), 0.05 (control). Total scores on the condom attitude scale (Slesnick 2007/08) were not available.
Longer term change scores: Milburn 2012: 12 months: Percentage of participants who had had unprotected sex in last 3 months: 2.3 (intervention), 0.9 (control).
1.4 Other sexual risk behaviour
In Slesnick 2007/08: Raw numbers were not available for these outcome items. So we calculated change scores at 6 months for percentage of participants who had had sex with more than one partner within 24h: 1.11 (intervention), -5.75 (control). Percentage of participants had had sex with high-risk sex partners in last 3 months: -2.6 (intervention), -4.25 (control). Percentage of participants had engaged in anal sex in last 3 months: -2.36 (intervention), -5.14 (control). Percentage of participants had engaged in casual sex in last 3 months: -10.19 (intervention), -12.54 (control). Percentage of participants had engaged in survival sex in last 3 months: 0.6 (intervention), -8.3 (control).
Notably the above figures suggest that control group participants had reduced risky behaviours considerably more than intervention participants on several outcome items. In particular, for percentage of participants who had had sex with more than one partner within 24h, the figure increased in the intervention group but reduced in the control group (see also the last outcome item, although total numbers of participants for this were small). These figures suggest that the comparison intervention may have been more efficient for this outcome category.
Longer term change scores: Milburn 2012: 12 months: Percentage of participants who had had sex in last 3 months: -4.9 (intervention), 1.9 (control). As the authors maintain, fewer participants in the intervention group appear to have initiated sexual relations over the duration of the study. In the control group there was a marginal increase. According to the figures, on average less than half of participants in either group had had sex in the last 3 months (no SD available).
2. Safer or reduced substance use (e.g. reduced sharing of injecting equipment).
2.1 Number of days of alcohol use in last 30 days (1 & 3 months)
Change scores: 1 month: Baer 2007: -1.9 (intervention), -1.3 (control), Peterson 2006: -1.16 (intervention), -1.2 (control). 3 months: Baer 2007: -1.1 (intervention), -1.9 (control); Peterson 2006: -1.47 (intervention), -3.09 (control). The changes were similar in all groups, showing only a marginal reduction in number of days of alcohol use. The largest reduction was in the control group in Peterson 2006.
2.2 Percentage days of alcohol use in last 90 days (3 months)
Change scores: Slesnick 2005: 3 months: -2.04 (intervention), -1.22 (control). Slesnick 2009 EBFT; Slesnick 2009 FFT: 3 months: -15.0 (EBFT intervention), -18.0 (FFT intervention), -8.0 (control). Percentage days of alcohol use was reduced across all groups, but the largest reduction was in both intervention groups in Slesnick 2009 EBFT; Slesnick 2009 FFT.
Longer term change scores: Slesnick 2005: 12 months: -0.95 (intervention), -1.59 (control). Slesnick 2009 EBFT; Slesnick 2009 FFT: 15 months: -26 (EBFT intervention), -20 (FFT intervention), -10 (control). While the figures for Slesnick 2005 appear to favour the control group at 12 months, overall the trends are similar to above, with reductions across the group.
Longer term change scores. Milburn 2012 (12 months): Times used alcohol: -6.16 (intervention), -0.82 (control). This appears to represent a benefit in favour of the intervention group. Accurate data for percentage of participants who used alcohol (intervention) was not available.
In Baer 2007, change scores for number of days of abstinence in last 30 days were: 1 month: 3.7 (intervention), 6.4 (control). 3 months: 2.7 (intervention), 6.0 (control). These figures appear to favour the control group.
2.3 Number of standard drinks in last 90 days (3 months)
Change scores: Slesnick 2009 EBFT; Slesnick 2009 FFT: 5.31 (EBFT intervention), 5.16 (FFT intervention), 0.18 (control). The number of standard drinks reduced for both intervention groups, but only marginally for the control group.
In Peterson 2006, change scores at 1 month for number standard drink units in last 30 days were -5.51 (intervention), -7.89 (control). The trends appear similar, with a slightly larger reduction in the control group. 3-month data were not available. Data for number of days of binge drinking in last 30 days were also not available.
2.4 Adolescent Drinking Index score (3 months)
The differences between the groups appear small but all show an overall reduction in the ADI score.
2.5 Percentage days of alcohol/ drug use in last 90 days (3 months)
Change scores: Slesnick 2009 EBFT; Slesnick 2009 FFT: -10 (EBFT intervention), -28 (FFT intervention), -13 (control). The percentage of days of alcohol or drug use reduced in all groups. The largest reduction was in the FFT group, whereas the change in the EBFT group was slightly smaller than in the control group.
These appear to be in benefit of the intervention groups, but reductions are indicated across all groups.
In Cauce 1994, change scores at 3 months for substance abuse were: -3.4 (intervention). -5.3 (control), indicating slightly more change in the control group.
2.6 Percentage days of alcohol/ drug use
Changes scores for Slesnick 2007/08 were: -24 (intervention), -10 (control). There were reductions in alcohol and drug use in both groups, seemingly larger in the intervention group.
2.7 Percentage days of only drug use in last 90 days (3 months)
Longer term change scores: Slesnick 2009 EBFT; Slesnick 2009 FFT: 15 months: -23 (EBFT intervention), -23 (FFT intervention), 2 (control). These data appear impressive and the authors claim benefit, however, due to the lack of information on error or deviation from the mean, they should be treated with caution, and as indicators only.
In Slesnick 2005, change scores for percentage days of any drug use (not alcohol or tobacco) in last 90 days were: 3 months: -20.31 (intervention), -15.1 (control). 6 months: -18.94 (intervention), -17.04 (control). 12 months: -28.94 (intervention), -20.26 (control). Percentage days used tobacco in last 90 days: 3 months: -8.63 (intervention), -5.55 (control). 6 months: -1.45 (intervention), -3.44 (control). 12 months: 0.39 (intervention), -7.27 (control). Percentage days used cocaine in last 90 days: 3 months: -1.63 (intervention), -4.82 (control). 6 months: -2.89 (intervention), -5.01 (control). 12 months: -2.8 (intervention), -5.94 (control). Percentage days used opiates in last 90 days: 3 months: -0.9 (intervention), 0.43 (control). 6 months: -1.74 (intervention), -0.64 (control). 12 months: -2.08 (intervention), -0.87 (control).
These findings are very mixed, which may partially reflect the fact that use of different substances is being assessed in each category, and thus potentially different participants. For some of these, there appeared to be significant baseline differences. The authors claim some benefits which are partially supported by these data, but without great certainty, in view of the missing standard deviation/error data. The data also suggest some benefits for control groups, especially for tobacco and cocaine use.
2.8 Number of categories of drug use in last 90 days (6 months)
Longer term change scores: Slesnick 2005: 12 months: -0.88 (intervention), -1.41 (control)
A slightly larger reduction is suggested in the control group.
2.9 Number of days of marijuana use in last 30 days (1 & 3 months)
Change scores: 1 month: Baer 2007: -3.7 (intervention), -6.1 (control), Peterson 2006: -2.16 (intervention), -1.77 (control). 3 months: Baer 2007: -2.6 (intervention), -5.9 (control); Peterson 2006:- 3.94 (intervention), -4.44 (control). The number of days of marijuana use reduced across all groups. The largest reductions were for the control groups at both time-points in Baer 2007 and at 3 months in Peterson 2006.
In Slesnick 2005, change scores for percentage days used marijuana in last 90 days were: 3 months: -17.92 (intervention), -12.58 (control). 6 months: -15.39 (intervention), -14.59 (control). 12 months: -25.65 (intervention), -16.96 (control). There were reductions in both groups at both time points but changes were slightly larger in the intervention group.
Longer term change scores: Milburn 2012: 12 months: Percentage of participants who used marijuana: -10.8 (intervention), -22.8 (control). Times used marijuana: 0.88 (intervention), -5.19 (control). Both figures appear to be in favour of the control group.
2.10 Number of days of illicit drug use other than marijuana in last 30 days (1 & 3 months)
Change scores: 1 month: Baer 2007: -2.3 (intervention), -3.0 (control), Peterson 2006: -1.42 (intervention), -0.71 (control). 3 months: Baer 2007: -2.8 (intervention), -2.3 (control); Peterson 2006: -1.37 (intervention), -0.29 (control). The number of days of other drug use reduced across all groups. In Baer 2007, the changes for the two groups were similar. In Peterson 2006, the reduction was larger in the intervention group.
In Peterson 2006, change scores for 'summed drug use other than marijuana' were: 1 month: -2.94 (intervention, 0.34 (control). 3 months: -4.53 (intervention), -1 (control). These appear to favour the intervention group, though at 3 months there was a reduction in both groups.
Longer term change scores: Milburn 2012: 12 months: Percentage of participants who used hard drugs: -8.8 (intervention), -9.8 (control). Times used hard drugs: -2.3 (intervention), -1.34 (control). The first figures indicate a similar change in both groups, whilst the second indicates a larger reduction in the intervention group.
2.11 Number of problem consequences (3 & 6 months)
Change scores: 3 months: Slesnick 2005: -1.14 (intervention), -1.78 (control); Slesnick 2009: -1.44 (EBFT intervention), -2.08 (FFT intervention), -3.66 (control). 6 months: Slesnick 2005: -2.15 (intervention), -2.25 (control); Slesnick 2007/08: -2.33 (intervention), -1.74 (control). The number of problem consequences reduced across all groups. The reduction was relatively larger in the control group at 3 months in Slesnick 2009.
The figures indicate similar reductions across all groups.
2.12 Number of substance use diagnoses (3 months)
2.13 Use of injection drugs
In Slesnick 2007/08, change scores for number of people shared needles to inject drugs in last 3 months were: 3 months: -0.01 (intervention), -0.01 (control). 6 months: 0.0 (intervention), 0.0 (control). Percentage of participants who had shared needles to inject any drugs in last 3 months: 6 months: -0.5 (intervention), -0.9 (control). Percentage of participants who had injected drug use in last 3 months: 6 months: -3.86 (intervention), -3.05 (control).
The trends are similar for both groups, ranging from no change to a very small reduction in risk behaviours. The baseline levels for these outcome items were very low.
3. Increased use of hostel/shelter type services
In Baer 2007, the change scores for 'number of visits at drop-in centre in last 30 days' (agency reported) were: 1 month: 0.9 (intervention), -0.2 (control). 3 months: -1.1 (intervention), -1.0 (control). 'Number of visits to additional services in last 30 days' (agency reported): 1 month: 0.5 (intervention), 0.0 (control). 3 months: 0.1 (intervention), -0.1 (control). 'Number of visits to other services in last 30 days' (youth reported) were: 1 month: -2.4 (intervention), -7 (control). 3 months: -3.4 (intervention), -8.2 (control).
The figures indicate little differences between the groups, apart from number of visit to drop-in centre at 1 month, which may be a contingency effect (youth using drop-in services while attending the intervention). Also, self-reported number of visits to additional services appear to have reduced more in the control group. Overall, the intervention did not appear to increase service use.
Change scores: Cauce 1994: 0.2 (intervention), 0.5 (control); Hyun 2005 2.29 (intervention), 3.07 (control). The trends are similar for all groups, but largest increase in mean score for self-esteem was in the control group in Hyun 2005.
Change scores: 3 months (/8 weeks, Hyun 2005): Cauce 1994: -5.1 (intervention), -3.7 (control); Hyun 2005: -5.79 (intervention), 2.38 (control); Slesnick 2005: -4.69 (intervention), -4.65 (control); Slesnick 2009: -4.09 (EBFT intervention), -5.88 (FFT intervention), -3.24 (control). 6 months: Slesnick 2005: -5.13 (intervention), -6.05 (control); Slesnick 2007/08: -8.25 (intervention), -3.8 (control).
Again, the trends are similar across the studies, indicating reductions in depression scores, apart from a contrasting trend in Hyun 2005, favouring the intervention. The figures for Slesnick 2007/08 also seem to favour the intervention.
In Cauce 1994, the change scores for anxious/ depressed (as measured on the YSR) were: 3 months: -1.7 (intervention), -0.9 (control), indicating no clinically significant difference between the groups and no significant change from baseline on this scale, as compared to the RADS cited above.
The figures show reductions in all groups but slightly different patterns in terms of the scale of change between the two studies.
10. Reduced use of violence
10.1 Verbal aggression (youth) (3 months)
Longer term change scores. Slesnick 2005: 12 months: -0.25 (intervention), -0.25 (control). Slesnick 2009 EBFT; Slesnick 2009 FFT: 15 months: -0.23 (EBFT intervention), -0.36 (FFT intervention), -0.18 (control).
There appear to be no differences between the groups, with reduced aggression reported in all. The mean scores on this measure appear to be consistently low.
10.12 Family violence (youth) (3 months)
Longer term change scores. Slesnick 2005: 12 months: -0.06 (intervention), -0.05 (control). Slesnick 2009 EBFT; Slesnick 2009 FFT: 15 months: -0.07 (EBFT intervention), -0.06 (FFT intervention), -0.03 (control).
Again, there appear to be no differences between the groups, with reduced aggression reported in all. The mean scores on this measure appear very low overall.
11. Increased contact with family
Change scores: Slesnick 2009 EBFT; Slesnick 2009 FFT: -11.0 (EBFT intervention), -19 (FFT intervention), 3 (control). The control group increased their number of days living at home by three days on average, in contrast to both intervention groups who reduced it by more than one week (EBFT) and two weeks (FFT) on average.
At 15 months, all groups had increased the amount of time spent at home. However, there appears to have been a considerably larger increase in the average percentage of days living at home in the control group, compared to both intervention groups.
13. Social functioning
13.1 Delinquent behaviours (at 3, 6 & 12 months)
Change scores: 3 months: Cauce 1994: -1.4 (intervention), -0.7 (control); Milburn 2012: -1.45 (intervention), -1.07 (control); Slesnick 2005: -87.63 (intervention), -192.61 (control); Slesnick 2009: -154 (EBFT intervention), -151 (FFT intervention), -842 (control). 6 months: Milburn 2012: -1.55 (intervention), -1.09 (control); Slesnick 2005: -71.20 (intervention), -274.29 (control); Slesnick 2007/08: 10.56 (intervention), -29.97 (control). 12 months: Milburn 2012: -1.74 (intervention), -1.12 (control); Slesnick 2005: -84.43 (intervention), -307.8 (control).
According to these figures, there was a reduction in delinquent behaviours across all groups, with the single exception of the intervention group in Slesnick 2007/08, for whom the number of delinquent behaviours had increased at 6 months, as opposed to a reduction in the control group. Further, in the Slesnick studies, the reductions appear considerably larger in the control group, while the opposite was true in Milburn 2012 and Cauce 1994.
Little information was available for any of the scales to aid interpretation of the scores. However, in the Slesnick studies, the data were skewed and in two Slesnick studies there was considerable baseline imbalance for this outcome, with the control groups in Slesnick 2005 and Slesnick 2009 EBFT; Slesnick 2009 FFT having considerably higher mean scores at baseline.
Longer term change scores. Delinquent behaviours: Slesnick 2005: 12 months: -84.43 (intervention), -307.17 (control). Slesnick 2009 EBFT; Slesnick 2009 FFT: 15 months: -154 (EBFT intervention), -159 (FFT intervention), -912 (control). Milburn 2012: 12 months: -1.74 (intervention), -1.12 (control).
The figures indicate reductions in all groups. The figures for Slesnick appear to favour the control group, while the figures for Milburn appear similar in both groups.
13.2 Other social functioning measures
In Cauce 1994, the change scores for 'social problems' were: -0.4 (intervention), -0.1 (control), indicating marginal change in both groups.
In Slesnick 2007/08, change scores at 6 months for 'social stability', were: 28 (intervention), 7 (control). This figure appears to significantly favour the intervention. Social stability was measured in this study only on Form 90 by the percentage days in work, education, being housed, and seen for medical care.
In Tischler 2002, the percentage of participants (children rather than their parents) who improved on peer relationship scores of the SDQ at 6 months from baseline were: 44% (intervention), 20% (control). The figure appears to favour the intervention.
14. Psychological functioning
14.1 Internalising problems (at 3 & 6 months)
Change scores: 3 months: Cauce 1994 2.7 (intervention), 1.8 (control); Slesnick 2005: 1.33 (intervention), 4.91 (control); Slesnick 2009: 3.44 (EBFT intervention), 3.44 (FFT intervention), 4.46 (control). 6 months: Slesnick 2005: 3.76 (intervention), 4.64 (control); Slesnick 2007/08: 7.04 (intervention), 3.31 (control). The figures indicate a reduction in internalising problems in all intervention and control groups. As for depression, the largest reduction appeared to be in the intervention group in Slesnick 2007/08.
In Cauce 1994, the change scores for 'Withdrawn' were: 0.3 (intervention), 0.0 (control). For 'Somatic complaints', the change scores were: -0.8 (intervention), -1 (control). The changes appear marginal.
Longer term change scores. Internalising problems: Slesnick 2005: 12 months: -4.64 (intervention), -8.43 (control). Slesnick 2009: 15 months: -6.09 (EBFT intervention), -5.96 (FFT intervention), -5.5 (control). There appear to be reductions in all groups, with the largest reduction appearing to be in the control group in Slesnick 2005.
14.2 Externalising problems (at 3 & 6 months)
Change scores: 3 months: Cauce 1994 2.9 (intervention), 0.6 (control); Slesnick 2005: 2.31 (intervention), 6.11 (control); Slesnick 2009: 4.78 (EBFT intervention), 6.38 (FFT intervention), 6.43 (control). 6 months: Slesnick 2005: 4.35 (intervention), 6.62 (control); Slesnick 2007/08: 5.09 (intervention), 4.26 (control). The figures indicate a reduction in externalising problems in all intervention and control groups. In Slesnick 2005 and Slesnick 2009 EBFT, the reduction in externalising problems was larger in the control group.
Longer term change scores. Externalising problems: Slesnick 2005: 12 months: -7.6 (intervention), -9.23 (control). Slesnick 2009 EBFT; Slesnick 2009 FFT: 15 months: -10.52 (EBFT intervention), -11.11 (FFT intervention), -6.23 (control).
In Cauce 1994, the change scores for attention problems were: -0.7 (intervention), 0.1 (control). For aggressivity, the change scores were: -1.5 (intervention), 0,2 (control). For problem behaviour, the change scores were: -0.2 (intervention), -0.3 (control).
In Tischler 2002, the following percentage of participants improved on conduct scores at 6 months from baseline: 42% (intervention), 32% (control) and on hyperactivity scores: 44% (intervention), 28% (control).
14.3 Number of psychiatric diagnoses (3 months)
Change scores: Slesnick 2005: -0.17 (intervention), -0.33 (control);Slesnick 2009 EBFT; Slesnick 2009 FFT: -0.52 (EBFT intervention), -0.39 (FFT intervention), 0.46 (control). The changes for this outcome measure were marginal, indicating that some psychiatric diagnoses may be stable over time and not responsive to interventions not specifically targeted at a clinically mentally ill population.
Longer term change scores. Slesnick 2005: 12 months: -0.63 (intervention), -1.04 (control). Slesnick 2009 EBFT; Slesnick 2009 FFT: 15 months: -1.0 (EBFT intervention), -0.85 (FFT intervention), -0.85 (control).
14.4 Other psychological functioning measures
In Cauce 1994, change scores at 3 months for 'thought problems' were: -0.3 (intervention), 0.0 (control). Change scores for 'total problems' were: -3.2 (intervention), -1.2 (control). Change scores for 'quality of life' were: 0.2 (intervention), 0.0 (control).
Of these, the most change appears to be manifest in reductions in the scores in the 'total problems' category.
In Slesnick 2007/08, change scores at 6 months for CISS task scale ('task-oriented coping') were: 2.41 (intervention), 1.57 (control); change scores for CISS emotion scale ('emotion-oriented coping') were: -7.52 (intervention), -3.96 (control), and change scores for CISS avoidance scale ('avoidance-oriented coping') were: -1.55 (intervention), -2.26 (control).
The scores reported here appear to go in different directions, but the interpretation of the results is unclear since we have been unable to find further detail of the meaning of scores on this particular scale.
In Tischler 2002, Children's total SDQ score at 6 months (Change from baseline (mean/ SD)) was: -2.64 (7.26) (intervention), 1.88 (4.30) (control).
While the intervention achieved a positive reduction in terms of the overall score, we could not compare it with any other study. The following percentages of participants had improved on the following outcome item (sub scale of the SDQ): Emotional scores: 56% (intervention), 44% (control). The total parental mental health score at 6 months (Change from baseline (mean/ SD)) was: -6.05 (7.23) (intervention), -6.10 (8.85) (control).
15. Family functioning
It should be noted that it is only one group of studies (Slesnick 2005 and Slesnick 2009 EBFT; Slesnick 2009 FFT) that included this category of outcomes. They are all measured on a self-report measure for which limited information was available. The data for these outcomes is mixed, generally indicating improvements in all groups in both studies. We are unable to comment on the clinical significance of the changes.
15.1 Family cohesion (3 months)
Longer term change scores. Slesnick 2005: 12 months: 1.88 (intervention), 2.10 (control). Slesnick 2009 EBFT; Slesnick 2009 FFT: 15 months: 1.68 (EBFT intervention), 1.28 (FFT intervention), 1.65 (control).
At 3 months, family cohesion appears to have improved the most in the intervention groups in both studies, particularly in Slesnick 2009 EBFT. At 12 months, family cohesion appeared to have improved similarly in all groups, again in both studies.
15.2 Family conflict (3 months)
Longer term change scores. Slesnick 2005: 12 months: -1.78 (intervention), -1.88 (control). Slesnick 2009 EBFT; Slesnick 2009 FFT: 15 months: -1.69 (EBFT intervention), -2.52 (FFT intervention), -1.3 (control).
At 3 months, family conflict appears to have reduced the most in the control group in Slesnick 2005, and the intervention group in Slesnick 2009 FFT. At 12 months, change scores appear similar for all groups, but with a greater reduction in Slesnick 2009 FFT.
15.3 Parental care (3 months)
Longer term change scores. Slesnick 2005: 12 months: 4.88 (intervention), 5.62 (control). Slesnick 2009 EBFT; Slesnick 2009 FFT: 15 months: 2.48 (EBFT intervention), 4.66 (FFT intervention), 2 (control).
15.4 Parental overprotectiveness (3 months)
Longer term change scores. Slesnick 2005: 12 months: -4.37 (intervention), -4.91 (control). Slesnick 2009 EBFT; Slesnick 2009 FFT: 15 months: -5.26 (EBFT intervention), -2.5 (FFT intervention), -2.5 (control).
At 3 months, the figures suggest a similar trend in all groups, apart from the control group in Slesnick 2005. This was the only instance where parental overprotectiveness appears to have slightly increased. At 12 months, there was a reduction in all groups, especially both groups in Slesnick 2005, and in Slesnick 2009 EBFT.
Appendix 4. Outcome items not included in meta-analyses
Outcome items not included in meta-analysis were (for each study):
Baer 2007 (3 months): Number of days of abstinence (in last 30 days); number of visits at drop-in centre (last 30 days); number of visits to additional services (last 30 days, agency reported), and number of visits to other services (last 30 days, youth reported).
Cauce 1994 (3 months): Withdrawn; Somatic complaints; Anxious/ depressed; Social problems; Thought problems; Attention problems; Aggressive; Total problems; Problem behaviour, and Quality of life.
Hyun 2005 (8 weeks): Self-efficacy.
Milburn 2012(3, 6 & 12 months): Had sex (past 3 months); Had unprotected sex (past 3 months); Used alcohol (past 3 months); Used marijuana; Used hard drugs (past 3 months), and Number of times used hard drugs (past 3 months).
Peterson 2006 (1 & 3 months): Number of days of binge drinking; Number of standard drink units (last 30 days), and Drug use consequences.
Rew 2007 (T1, T2, T3; up to 6 weeks): AIDS/STD knowledge; Future time perspective; Condom self-efficacy; Self-efficacy to perform breast/ testicular self-examination; Assertive communication; Help-seeking for STDs; Safe sex practices, and Risky sexual behaviour.
For these, we report p-values as calculated by the authors. The data refer to T1-T2, T2-T3, and/ or T1-T3.
Rotheram-Borus 2003 (3, 6, 12, 18, 24 months): Number of sexual partners; Number of unprotected sex acts; Abstinence from vaginal/ anal sexual acts; Used alcohol; Used marijuana, and Number of drugs used.
For these, we report odds ratios, p-values and confidence intervals, as calculated by the authors. The data refer to 3, 6, 12, 18 and 24-month data.
Slesnick 2005 (3, 6 & 12 months): Percentage days used tobacco; Percentage days used marijuana; Percentage days used cocaine; Percentage days used opiates; Percentage days any drug use (not alcohol or tobacco); HIV knowledge, and High-risk behaviours.
Slesnick 2007/08 (3 and/ or 6 months): Percentage days of drug/ alcohol use; High-risk behaviours, CISS task scale; CISS emotion scale; CISS avoidance scale; Social stability; HIV risk behaviour (total); Number of people shared needles to inject drugs; Number of people having sexual intercourse with; Condom use frequency (self/ partner); HIV knowledge; Injected drug use; Shared needles to inject any drugs; Engaged in casual sex; Had sex with more than one partner within 24h; Had sex with high-risk sex partners; Engaged in anal sex; Engaged in survival sex; and Condom attitude scale total score
Tischler 2002 (6 months): Children's SDQ score (total); Parental mental health score.
Appendix 5. Table of scales information
Last assessed as up-to-date: 8 March 2012.
Contributions of authors
Esther Coren (EC), Rosa Hossain (RH), and Manuela Thomae (MT) drafted the protocol with input from other authors and from the advisory group. Jordi Pardo Pardo (JPP) developed the search strategy with input as above and consulted on the development of the logic models. He was involved in many strategic discussions with EC on the overall progress of the review. Mirella MS Veras (MV) contributed to refining the search and Portuguese language terms, was closely involved in detailed screening of Spanish and Portuguese language texts retrieved in the search, and contributed to data extraction. Kabita Chakraborty (KC) contacted organisations and NGOs in the field for unpublished data, and made a contribution to screening, as well as to extrapolation and discussion. RH also contributed to development and pilot of the data extraction and mapping tools, to data extraction and mapping, analysis and synthesis and to writing the review including the discussion section. Holly Harris contributed to screening, data extraction and mapping, as well as to extrapolation and the writing of the discussion section. Anne Martin (AM) contributed to data extraction and to final production of the completed review. EC directed all stages of the review, coordinated the team and was involved directly in all stages and and in all aspects of problem solving through each stage of the review.
Declarations of interest
Research discussed in this publication has been funded by the International Initiative for Impact Evaluation, Inc (3ie) through the Global Development Network (GDN). The views expressed in this article are not necessarily those of 3ie or its members, or of GDN.
Sources of support
- No sources of support supplied
- 3ie, Not specified.Funding to support the project
Differences between protocol and review
The review followed processes planned in the protocol, except where circumstances did not arise, for example cluster trials.
Medical Subject Headings (MeSH)
MeSH check words
Adolescent; Child; Child, Preschool; Female; Humans; Male; Young Adult