PROTOCOL: Studies of the effectiveness of interventions to improve the welfare of those affected by, and at risk of, homelessness in high‐income countries: An evidence and gap map

Abstract Homelessness – people living on the street, in temporary accommodation, or at risk of losing their homes – is a persistent problem across the developed world. Policies and programmes to tackle homelessness should be informed by evidence of effectiveness. This is the protocol for an evidence and gap map for studies of the effectiveness of interventions to improve the welfare of those experiencing homelessness or at risk of homelessness. We proposed a comprehensive search for studies, with systematic screening, coding and reporting. The available studies will be presented in an online interactive map together with a supporting report.

Effective interventions are, therefore, required to place and keep people in stable housing, and address the health and wider support needs of all people experiencing or at risk of homelessness. There is a range of interventions to try to prevent homelessness and to increase housing stability. However, the evidence base of studies of the effectiveness of these interventions is thought to be uneven by both study design and geography, with most studies being conducted in North America.
Development of the map will support efforts to tackle socioeconomic exclusion, and sustained deprivation and inequality. It will support related research initiatives such as Inclusion Health (Luchenski et al., 2018). And importantly the maps will support a suite of evidence tools produced by the Centre for Homelessness Impact (CHI; http://homelessnessimpact.org). Homelessness is broadly defined to include not only those sleeping rough. Those experiencing homelessness are those who have no accommodation and so sleep on the street (sleeping rough) or are in temporary (i.e., transitional), insecure or poor-quality housing (European Commission, no date). People in temporary shelters or other transitional accommodation are still considered homeless. Those at risk of homelessness may currently be in satisfactory accommodation but at risk of losing it-for example, because of loss of employment or other income sources.

| Scope of the EGM
The interventions, which are listed below, are interventions whose main purpose is to improve the welfare of those experiencing or are at risk of homelessness. Figure 1 shows the logic model for the interventions and how they link to the major outcomes. This does not provide a detailed theory of change of how specific interventions are meant to work but rather provides an overview of the policy space covered by the evidence map and how those parts fit together.

| Conceptual framework of the EGM
Key features are as follows.
1. Legislation sets the context affecting services, care and accommodation.
2. Public opinion, which is affected by advocacy and communication, affects legislation and provision of interventions such as accommodation.
3. Services and outreach and social care can improve health and reduce substance abuse, thus allowing access to education and skills training and so employment. 4. These can lead to increased income and so stable housing and improved wellbeing.
5. Providing accommodation can support stable housing which in turn supports health and employment prospects (the reverse causation shown by the dotted line). 6. Prevention enters into this causal chain at several points. 7. Interventions interact reflecting that clients often need multiple services.

| Why it is important to develop the EGM
Currently, there is no single resource that allows policy makers, practitioners and researchers working to improve the welfare of those experiencing homelessness to access the available relevant evidence on which programmes work. The review team is working with the UK Centre for Homelessness Impact to develop the evidence architecture for the sector.
The CHI plans to become a "one stop shop" for evidence for policy makers and practitioners in the sector. As a first step, working with the Campbell Collaboration, the Centre is producing to two evidence maps of evidence on homelessness. This protocol is for the map of effectiveness studies of What Works to improve the Welfare of those Experiencing Homelessness. A second map will show implementation issues for such interventions as identified in process evaluations. The two maps together will comprise the largest single source globally of evidence on interventions for those experiencing and at risk of homelessness.
CHI aims to improve the welfare of people affected by homelessness by providing evidence-based resources for policy makers and practitioners. The EGMs are the first part of that F I G U R E 1 Overview of logic model for homelessness interventions evidence architecture, and a building block for what will come next.
The maps will identify the evidence to be used in the Centre's online evidence resources. And the maps will inform the future-policyoriented research programme of the Centre.
In the coming years, CHI will be commissioning new studies to assess the effectiveness of programmes for those affected by homelessness. The map will inform the identification of priority areas where evidence is currently lacking, such as rigorous studies of the effectiveness of reconnection programmes or those being discharged from mental health or penal institutions.

| Existing EGMs and relevant systematic reviews
We are aware of two other maps related to homelessness. One is being prepared by the Canadian Homelessness Health Network. That map has a narrower focus than ours, but we are sharing resources with the team to ensure consistency in coverage. A second, unpublished map, was produced by the Sax Institute for the New South Wales state government. That map, which includes only 16 studies, is narrower in scope than the proposed sector wide map we will produce.
There are a number of systematic reviews, all of which are narrower than the proposed map.
Most recent is a rapid evidence review undertaken by CRISIS. The review has a broad scope but limits the evidence being reviewed: 120 studies were identified as high quality of which 35 were analysed (SCIE,  (2013) report studies relate to homelessness amongst female veterans. There are three on-going reviews registered with Campbell which have been identified on the basis of earlier editions of this map.
There are also prevalence reviews related to homelessness, especially related to mental health (e.g., Folsom and Jeste (2002)

| OBJECTIVES
The proposed EGM will present studies of the effectiveness of these interventions across a range of outcome domains. Specifically, the objectives of the map are to: 1. develop a clear taxonomy of interventions and outcomes related to homelessness in high-income countries; 2. map available systematic reviews and primary studies of the effectiveness of interventions for those experiencing homelessness and those at risk of homelessness, with an overview provided in a summary report; 3. provide database entries of included studies that summarise the intervention, context, study design and main findings.

| Defining EGMs
This evidence and gap map (EGM) is an effectiveness map in which the primary dimensions are the rows and columns of the map which are, respectively, intervention categories (and subcategories) and indicator domains (and subdomains). Secondary dimensions, such as country and target group will be included as filters.

| EGM framework
The EGM framework will inform the inclusion and exclusion criteria of the EGM. Here, we describe the population, intervention, comparison, outcomes (indicators) and study designs (PICOS) for the map.

| Population
The population is individuals and families who are homeless or at risk of becoming homeless.
Population subgroups of interest are listed under filters. Table 1 lists the intervention categories. Examples of programme names are given in brackets. These are listed to aid with search and coding. They will not appear in the subcategory label in the map.

| Intervention
Some programmes are either multicomponent or straddle intervention subcategories. Examples are Housing First (congregate/scatter site; ACT/ICT) and Homeless Veterans' Reintegration Program.
Studies of these interventions can appear in more than one category.
The map will have a searchable field (filter) for programme name where these programmes are not included in the intervention subcategories (see below on filters).

| Indicators (outcomes)
The indicator domains are shown in Table 2

| Types of study designs
This is a map of the effectiveness of interventions to improve the welfare of those experiencing, or at risk of, homelessness.
Effectiveness studies are those using large and statistical designs to measure the impact of an intervention, or systematic reviews of such studies.
Given this purpose, the map will include experimental and nonexperimental impact evaluations with a design which controls for selection bias. The following designs will be included: RCTs, natural experiments, regression discontinuity, propensity score matching, difference in difference, instrumental variables, and other matching designs. Before versus after designs with no control group will not be included.
The map will also include systematic reviews of effects that include studies from high-income countries.
Comparison: Studies with both active and passive controls will be included.

| Treatment of qualitative research
We do not plan to include qualitative research in this map. A separate map is being prepared that will include process evaluations.

| Types of settings
Studies will be from high-income countries.
T A B L E 1 Intervention categories and subcategories Accommodation Accommodation (excluding emergency accommodation) with minimal or no support services. This includes: community-led housing; modular homes; private-rented sector; social housing; temporary accommodation.
Accommodation with support services Accommodation (excluding emergency accommodation) combined with some form of support services. This includes: housing first; rapid rehousing; supported accommodation and supported lodging; hostels; women's refuges.
Accommodation-based support services Accommodation (excluding emergency accommodation) based support services. This includes continuum of care/staircase; coordinated assessment; floating support; housing advice; landlord/ tenant mediation; tenancy training.

Armed forces
Interventions targeted at people in the armed forces. This includes: induction and initial training; ongoing development and support; discharge from armed services.
Arts, sports and culture Mainstream and specialist arts, sports and cultural activities.

| Status of Studies
On-going studies will be included. Status of studies will be a filter.

| Search strategy and status of studies
The search strategy comprises both, where to look and how to look.
This map is being produced in stages. The approach for both elements of the search strategy is described below.
On account of the need for early results for CHI the map is being produced in the following stages. 2. Stage 2 will map the results from the full database search, including both primary studies and systematic reviews.

This search is described below. This map was published in
March 2019.

Stage 3 will be the version of the map published in the Campbell
Library. In addition to the above, we will (a) search additional websites for grey literature, (b) screen all included studies in included systematic reviews, (c) consult experts, and (d) screen submissions received in response to dissemination of the Stages 1 and 2 maps.

| Database search for Stages 2 and 3
The databases to be searched are as follows.

Academic databases
• Econlit • The National Bureau of Economic Research (NBER) • Social Science Research Network (SSRN) • International Bibliography of Social Sciences (IBSS) • Applied Social Sciences Index and Abstracts (ASSIA) • Social Service Abstract All titles and abstracts, and then full text, will be double screened, with a third-party arbitrator in the event of disagreement.

| Grey literature and websites
In addition to electronic studies, we shall search and screen publications from the following websites.

| Contacting researchers
We will send copies of the preliminary map to authors of included studies, which serves both a dissemination purpose and to invite submission of additional studies.

| Data extraction, coding and management
Coding will be done independently by two coders, with a third-party arbitrator in the event of disagreement.

Coding of bibliographic information and intervention and study design and characteristics
Full bibliographic information will be captured, along with the information necessary to construct the map (interventions, outcomes and filters). The coding form is given in Appendix B.

| Critical appraisal
Coding will also capture the data needed for critical appraisal of all included studies. Critical appraisal of primary studies shall be conducted using the tool contained in Appendix C. The quality of the included systematic reviews will be assessed using AMSTAR 2.

| Unit of Analyses
The unit of analysis is each paper. Each entry in the map is a report or paper.
It is possible (indeed likely for public health) that there are multiple papers for a single study. If this occurs as there are different versions of the same paper, then only the latest or most complete version will be used in the map. However, if different papers report different analyses-for example, on different outcomes or for different population subgroups-then each such paper is included in the map. Hence, in principle, there may be multiple entries from a single study. If any study accounts for more than 10 papers or reports that study shall be included as a filter. The accompanying EGM report will identify the number of studies covered by the map and list those studies with multiple papers in an annex.

| Presentation
The intervention and outcomes, described above, are the primary dimensions of the map.
In addition to intervention and outcomes, the following filters will be coded for primary studies (and reviews where appropriate).

| Planned analyses
The EGM report shall provide tabulations or graphs of the number of studies, with accompanying narrative description, by the following: • intervention category and subcategory; • outcome domain and sub-domain; • table of "aggregate map" of interventions and outcomes; • region and country; • year; • study type.
The report will contain a network analysis of authors of included papers (see Rousseau, Egghe and Guns (2018): Chapter 10). In the network figure, each author will be represented by a circle, with size proportional to a number of studies authored, and lines connecting coauthors. The network will allow the identification of prominent authors and clusters of authors. (It will also help identify papers that are from a single study that may have been missed during coding.)

| STAKEHOLDER ENGAGEMENT
The framework was developed through a consultative process.
Stage 1: Two existing frameworks were considered as a basis for the framework to be used for this map: The map will be discussed with the Advisory Group for the Centre for Homeless Impact and presented at consultations organised by the Centre. Audrey Portes will be part of the screening and coding team, and undertake searches for studies in French and Norwegian.

ROLES AND RESPONSIBILITIES
Project management: Audrey Portes will manage the project to ensure timely delivery.

SOURCES OF SUPPORT
Production of the map has been supported by the UK Centre for Homelessness Impact with in-kind support from the Campbell Collaboration Secretariat. WHITE ET AL.

| 7 of 11
FHI is providing in-kind support through the generous provision of all studies subject to full-text screening for the review by Munthe-Kaas et al. (2016).

DECLARATIONS OF INTERESTS
Ligia Teixeira is the Director of the Centre for Homelessness Impact. This role should not provide any conflict as CHI's mission is to make evidence available. Suzanne Fitzpatrick is a leading researcher in the area so her some studies may be eligible for inclusion in the map.

PRELIMINARY TIMEFRAME
Approximate date for submission of the EGM: November 8, 2018.

PLANS FOR UPDATING THE EGM
The Centre for Homelessness Impact has agreed to provide resources to update the map every two years. The EGM team is in discussions with the EPPI Centre, who is responsible for the mapping software, about possible real-time updating through (a) automated searches with machine-learning powered screening and (b) moderated submissions of suggested papers.