Foster parents' experiences of using child mental health and welfare services in Norway: Associations with youth, placement, and service characteristics

Funding information Norges Forskningsråd, Grant/Award Number: 256598; EkstraStiftelsen Helse og Rehabilitering, Grant/Award Number: 149980 Abstract The measurement of user experience has gained international attention as a tool for improving quality of care. Because foster families have a high need for service support, we examined quality of care from the foster parent's perspective and associated characteristics. We collected information about type and frequency of service use in the last 2 years and standardized measures of user-reported experiences and outcomes from foster parents of youths aged 11–18 years in Norway (N = 290). We analysed the data using descriptive statistics, independent samples t-tests and multiple regressions. Overall, foster parents had positive service experiences, and around half reported improvement in youth condition and function compared with before the service contact. The foster parents gave similar evaluations of child welfare services and specialized mental health services but indicated different strengths and weaknesses of the providers. Younger age, more frequent service contact and less waiting time were associated with positive service experiences, while less mental health problems and fewer years in current foster home were related to positive perceptions of outcomes. Our results indicate focus areas for increasing quality of care from the user perspective, for example, sharing information, cooperation between services, having frequent enough service contact and reducing waiting time.

described three commonly accepted constructs for measuring the quality of health care: how patients experience treatment, clinical improvement, and patient safety. Patient-reported experience measures evaluate whether the care is patient-centred and thus reveal a dimension of quality of care that would otherwise not be captured (Anhang Price et al., 2014). A systematic review indicated positive associations between patient experiences and other quality measures across disease areas, settings, outcome measures and study designs (Doyle, Lennox, & Bell, 2013), which indicate utilitarian grounds for measuring patient experiences as well. Furthermore, Norwegian legislation states that users have a right to influence health and welfare services (Patient and User Rights Act, 2017) and that user inputs shall influence treatment at the individual, service and system level (The Norwegian Health library, 2009).
Receiving support through services of high quality is important for foster families as many children in foster care have complex needs (Luke, Sinclair, Woolgar, & Sebba, 2014), including a high prevalence of mental disorders (Lehmann, Havik, Havik, & Heiervang, 2013), medical and dental treatment needs (Kling, Vinnerljung, & Hjern, 2016) and school difficulties (Scherr, 2007). Furthermore, foster families often have extensive contact with health and welfare services (Larsen, Baste, Bjørknes, Myrvold, & Lehmann, 2018;Minnis, Everett, Pelosi, Dunn, & Knapp, 2006). Moreover, a recent meta-analysis indicated no overall changes in foster children's adaptive functioning or mental health problems during their time in foster care (Goemans, van Geel, & Vedder, 2015). Given the high prevalence of mental health problems in this group, the lack of overall improvement over time is troublesome and may indicate that many foster families need better service support to enable positive youth development. As foster parents are crucial agents for enabling positive change in foster children (Fernandez, 2007;Minnis & Del Priori, 2001), it is important to gain knowledge about their service experiences. Moreover, their ability to provide supportive care is affected by their interactions with service providers (Benesh & Cui, 2017;Geiger, Piel, & Julien-Chinn, 2017), and a lack of support from child welfare services (CWS) and health services seems to increase the risk of placement breakdowns (Khoo & Skoog, 2014;Rhodes, Orme, & Buehler, 2001;Tonheim & Iversen, 2018).
Studies have found that most foster parents were satisfied with health (Hayes, Geiger, & Lietz, 2015) and welfare services received (Geiger et al., 2017;López López & Del Valle, 2016). However, many foster parents felt insufficiently involved in decisions regarding the child (Geiger et al., 2017;Hayes et al., 2015;López López & Del Valle, 2016) and reported a lack of continuity of care within services (Pasztor, Hollinger, Inkelas, & Halfon, 2006), problems with availability and timeliness of mental health services (Hayes et al., 2015;López López & Del Valle, 2016) and issues with receiving sufficient information about the child from CWS workers (Geiger et al., 2017;López López & Del Valle, 2016;Pasztor et al., 2006). Moreover, foster parents expressed a need for more support and sensitivity from caseworkers (López López & Del Valle, 2016 Currently, there is a lack of research on characteristics related to foster parents' experiences of service quality following contact for the youths in their care. Studies of the general population have indicated that younger age (Bjørngaard, Wessel Andersson, Osborg Ose, & Hanssen-Bauer, 2008;Holmboe, Iversen, & Hanssen-Bauer, 2011;Turchik, Karpenko, Ogles, Demireva, & Probst, 2010), shorter waiting time (Bjørngaard et al., 2008;Holmboe et al., 2011), more treatment sessions and longer treatment episodes (Bjørngaard et al., 2008;Garland, Haine, & Lewczyk Boxmeyer, 2007;Holmboe et al., 2011) were associated with positive parent experiences with child and adolescent mental health services (CAMHS). However, these results were equivocal, and many of the effects identified were weak. Holmboe et al. (2011) found that youth characteristics explained a small part of the variance in service experiences, while service characteristics accounted for more variance.
In Norway, children are generally older when they are placed in foster care than in the United States, and adoption is rare (Pösö, Skivenes, & Hestbaek, 2014). Because of such differences, it is not given that existing knowledge of foster parents' experiences, predominantly from the United States, is applicable in a European and Norwegian context. Furthermore, there is a dearth of studies investigating foster parents' service experiences using standardized and psychometrically sound instruments. Such knowledge can improve quality of care by informing service development at the system level, on how to support foster parents in enabling positive youth development. The aim of this study was to examine quality of care from foster parents' perspective. Quality indicators were foster parent reports of service experiences and perceived outcomes of services received. In addition, we compared CWS and specialized mental health services on foster parent-reported quality of care and examined whether quality of care was associated with youth (gender, age and mental health problems), placement (years in current foster home) and service characteristics (frequency of contact, type of service, number of services and waiting time).

| Procedure and study sample
This study was a part of the larger study, 'Young in Foster Care' (Lehmann, 2016). The data collection took place between October 1. 2016 and March 31. 2017. Eligible participants were foster parents of youths born between 1999 and 2005 (youths aged 11-18 years) with whom the youths had lived for at least 6 months following legally mandated placement. We included foster parents with placements from municipalities in five Norwegian counties encompassed by The Office for Children, Youth and Family Affairs (Bufetat)-South (43 municipal CWS offices). Foster parents were assessed for eligibility from regional records from Bufetat South (n = 573) and from the municipal CWS offices (n = 279) in the region. We identified the foster parents of 736 youths as eligible (see Figure 1 for a detailed flow chart of the data collection).
We sent informational letters to foster parents through the mail describing the study and how to participate. Foster parents completed the questionnaires either online on a secure webpage or by telephone interview. We asked foster mothers and foster fathers to respond separately. We provided reminders by mail and subsequent telephone contact.
In total, one or both foster parents of 330 youths completed the survey (a 44.8% response rate). Of these, 290 reported service contact during the last 2 years and were included in this study. We collapsed foster mothers (n = 244) and fathers (n = 102) into one group of informants. We used responses from foster fathers when the foster mother was a nonresponder, otherwise we used information from foster mothers. There were no statistically significant differences between foster mothers and foster fathers responding for the same youth on any of the quality indicator items, with the exception that foster fathers reported more improvement in the youths' conditions compared with foster mothers (mean difference = 0.26, p = 0.031).

| Youth and placement characteristics
Youth gender, age and years lived in the current foster home were derived from CWS regional records and checked with the municipal CWS through telephone interviews.

| Service use
We assessed foster parents' contact with a broad range of services through a custom-made questionnaire asking foster parents how often the youth (or themselves, on behalf of the youth) had had contact with the following services during the last 2 years: CAMHS, school health services, educational psychology services, general practitioners, adolescent health clinic, CWS and special education. We also asked respondents if they had had contact with any other services and, if any, to name the service in an open text field. For each type of service, the following contact frequency alternatives were listed: every week (=4), every month (=3), every 3 months (=2), every 6 months (=1), or less often/none at all (=0). We computed the variable 'number of services' by adding up the services with which foster parents reported contact every 6 months or more often.

| Service experiences
We used a generic short questionnaire about parent experiences with CAMHS, derived from a more comprehensive and validated service questionnaire (Sjetne, Bjertnaes, Iversen, & Olsen, 2009). The generic short questionnaire consisted of 11 items that were rated on a fivepoint Likert scale, ranging from not at all (=1) to to a very large extent (=5), in addition to a sixth category of not applicable. We made some minor language changes to the questionnaire to make it fit contact with a broader range of service providers, for example, by defining 'clinicians' more widely than in the original description. The questionnaire had the following introduction: 'The following questions refer to your experiences with help services. When you answer, think of the service you've had the most contact with. By the term "clinicians" we mean those who have had the main responsibility for assessments and counseling. This may be doctors, psychologists, social workers, or other health and social personnel'. For this study, we included an item asking if the clinician talked to the youth in a way that she/he could understand (see Table 2 for an overview of F I G U R E 1 Flow chart of data collection the 12 items in the service experiences questionnaire). Principal component analysis was conducted in R (R Core Team, 2019) using the Psych package (Revelle, 2018) and through parallel analysis and investigation of the scree plot showed support for one dominant component accounting for 54% of the variance in overall service experience. We therefore treated these 12 items as indicators of overall service experience. We made a 'service experiences index' by reversing the one negative item and calculating the mean score of responders who had answered nine or more of the items and multiplied this score by 12 (index range 12-60). We treated 'Not applicable' responses as missing. This procedure gave an n of 238 on the service experiences index.
The generic short questionnaire also included an item asking if the foster parents had to wait for the service. This item was rated on a four-point scale with the response alternatives: 'no', 'yes, but not long' , 'yes, quite long' and 'yes, too long'. We refer to this variable as 'waiting time' and treated it as a categorical variable where 'no' was used as the reference group.

| Perceived outcomes
We assessed foster parents' perceived outcomes of services received with a section from a longer questionnaire regarding parent experiences with CAMHS (Holmboe & Garratt, 2007), which was one of the precursors of the generic short questionnaire on service experiences. The section consisted of three items measuring changes in the youths' condition and functioning compared with before the service contact; for details, see Table 3. The items were rated on a five-point Likert scale, ranging from much worse (=1) to much better (=5). A principal component analysis conducted in the same way as for service experience supported one dominant component explaining 85% of the variance in perceived outcome. Therefore, we treated these items as indicators for overall perceived outcome. We created a 'perceived outcomes index' variable by calculating the mean score for the three items for responders who had completed all three items and multiplying this by three (index range 3-15), resulting in an n of 259 on this variable.

| Type of service provider
The foster parents filled in an open text box describing which service provider they had in mind when answering the service experience and perceived outcome questions. We coded their answers into four categories: (a) CWS (including municipal, private and regional CWS services The frequency alternatives were every week (=4), every month (=3), every 3 months (=2), every 6 months (=1) or less often/none at all (=0). d Mean sum score of the 12 service experience items. e Mean sum score of the three perceived outcome items.
(including all municipal health providers, that is, educational psychological services, municipal psychologist and general practitioner).
(d) Other services (when none of the categories were applicable).
Where several providers were mentioned in the text box (n = 54), we applied the service with the highest reported contact frequency in further analyses. We computed a 'frequency of service contact' variable by matching the service category with the contact frequency reported for the relevant service, which was possible in 226 of the cases. There were 237 foster parents who described a service provider, and 188 of these noted their contact as being with either CWS or specialized mental health services. We created a 'service provider' variable where CWS contact was coded 1, specialized mental health was coded 0 and the other service types were set to missing.

| Mental health
We assessed youth mental health using the parent version of The Strengths and Difficulties Questionnaire (SDQ; Goodman, 1999 (Goodman, 1999(Goodman, , 2001, and the predictive value of a caregiver's completed total difficulties score was supported for foster children (Lehmann, Heiervang, Havik, & Havik, 2014).

| Data analysis
Youth, placement and service characteristics; the service experiences index; and the perceived outcomes index are presented as percent, means (M), standard deviations (SD), minimum and maximum values. For each service experience and perceived outcome item, we calculated the distribution of responses, means, and SDs. We used independent samples t-tests to compare responders with nonresponders on youth gender, age and years in current foster home. We compared foster parents evaluating CWS with foster parents evaluating specialized mental health services on the service experience and perceived outcome items and waiting time, using independent samples t-tests and a chi-square test.
We conducted linear regression analyses where the service experience index and perceived outcome index were regressed on the independent variables (IVs): gender, age, total difficulties score, years in current foster home, service provider, frequency of service contact, number of services and waiting time. We first tested all the IVs individually (unadjusted), then simultaneously (adjusted for all the other IVs) within a multiple regression model. The IVs were not highly correlated (≤0.49) and did not indicate problems with multicollinearity.
We conducted descriptive statistics, independent samples t-tests and the chi-square test using IBM SPSS Statistics for Windows, version 24 (IBM Corp., 2016). We conducted linear regression analyses in R (R Core Team, 2019) using the Lavaan package (Rosseel, 2012), with full information maximum likelihood estimation to handle missing data. This method assumes a missing-at-random mechanism. The significance level was set to 0.05.

| Sample characteristics
Sample characteristics are presented in Table 1. Of the 237 responders that provided information about which service they evaluated, 100 (42.2%) responded CWS and 88 (37.1%) responded specialized mental health services. Foster parents were in contact with three services on average (SD = 1.6). Most reported that they either did not wait for the service (41.0%) or did not have to wait long (41.0%), while fewer reported that they had to wait quite long (11.2%) or too long (6.8%). We found no significant differences between responders and nonresponders on youth gender, age or years in current foster home.

| Service experiences
For the 11 positively loaded service experience items, the amount of foster parent who agreed 'To a large extent' or 'To a very large extent' varied from 50.9% up to 92.4% (

| Perceived outcomes
Around half of the foster parents reported improvement of the youths' condition (59.4%) and that the youths functioned better in the family (55.2%) and among friends and at school (51.7%) than before the service contact (Table 3

| DISCUSSION
To the best of our knowledge, this is the first study to assess quality of care from foster parents' perspective and associations with youth, placement and service characteristics. Overall, the foster parents reported positive service experiences, which is in line with findings in the general population that most parents were satisfied with health services provided for their children (Aarons et al., 2010;Bjertnaes et al., 2008;Garland et al., 2007;Turchik et al., 2010). A large majority of foster parents reported that clinicians communicated in a way that was easy to understand and that they had confidence in the clinicians' professional skills. However, a substantial portion of foster parents evaluated some parts of the service contact as low to medium, including receiving information about the child's condition, cooperation with other services and organization of the work. This is consistent with studies from the United States, which indicated that foster parents experienced problems receiving relevant information from CWS workers, a lack of continuity of services and difficulties navigating different services (Bass, Shields, & Behrman, 2004;Geiger et al., 2017;Pasztor et al., 2006). As research has shown that a large proportion of foster families were in contact with several service providers (Larsen et al., 2018;Minnis et al., 2006), it is especially important for this group that different services work well together.
Around half of the foster parents reported improved youth condition and better functioning within the family and with friends and at school compared with before the service contact. The foster parents' reports of perceived outcomes were similar to how parents in the general Norwegian population rated outcomes of contact with CAMHS (i.e., overlapping confidence intervals; n = 7,906, child ages 0-16 years; Bjertnaes et al., 2008). This indicates that foster parents' experiences of outcomes of services received are comparable with those of parents of youths in a clinical sample. Despite an overall positive perception of outcomes of services for the foster youth, approximately 40% of the foster parents reported no change in youth condition or function, and approximately 5% reported a decline. One possible explanation is that many foster parents received support and guidance instead of therapeutic interventions directed at the youths' functioning per se, as much of the services provided by CWS in Norway are supervision and counselling of foster parents (Christiansen et al., 2015; The Office for Children, Youth and Family Affairs, 2019).
Alternatively, many foster parents did not experience positive changes compared with before the service contact, which might be a consequence of issues with information flow or lack of coherent treatment across services.
There were no overall differences in reported quality of care between responders evaluating CWS and specialized mental health services in the adjusted analyses. However, there were differences on single items in the quality indicators. Foster parents evaluating CWS were less satisfied with information given about the youths' mental health, more often reported no waiting time, and reported more improvement in youth functioning compared with responders evaluating specialized mental health services. The difference in youth functioning in favour of the CWS contact was surprising, given that specialized mental health services formally have the highest therapeutic competence. We need future studies including more information on the form and content of the services provided to examine whether this difference is found in other samples and can be explained by features of the service content. Our findings regarding information and waiting time align with findings from the United States and Spain where foster parents reported difficulties with the availability and timeliness of mental health services (Hayes et al., 2015;López López & Del Valle, 2016) and had problems receiving information about the child in their care from CWS (Geiger et al., 2017;López López & Del Valle, 2016;Pasztor et al., 2006). However, our results showed that responders evaluating the CWS more often reported no waiting time, while responders evaluating specialized mental health services more often reported that they had to wait, but not long. This difference seems reasonable, as referrals are needed to receive specialized mental health services.
Foster parents of older youths had less positive experiences with services, which is in line with findings from the general population (Bjørngaard et al., 2008;Turchik et al., 2010). This might indicate that services are better adapted to children than to adolescents. Alternatively, foster parents may be more involved in services for younger youths and therefore had more positive service experiences. Furthermore, foster parents of youths with more mental health problems reported less improvement in youth condition and functioning. One interpretation of this finding is that families that experienced less positive outcomes of services received consequently had youths with poorer mental health. Another possibility is that foster parents of youths with more initial mental health problems experienced less youth improvement. It would be troubling if the last explanation were true, and therefore, future research should examine this relationship further.
Foster parents of youths who had lived with them for longer periods experienced less positive outcomes of services received compared with foster parents of youths with shorter stays. This finding highlights the need to evaluate services provided to youths in foster care regularly to ensure that foster families receive appropriate service support over time. Finally, higher frequency of service contact and reporting no waiting time were related to more positive service experiences. This is consistent with findings from the general population, which showed that more treatment sessions, longer treatment episodes and shorter waiting times were associated with service satisfaction (Bjørngaard et al., 2008;Garland et al., 2007;Holmboe et al., 2011). This indicates that available services with sufficient resources to enable frequent contact are important to provide highquality care from the user perspective, also for foster families.
The full models of associated characteristics explained 12.9% of the variance in service experiences and 8.6% of the variance in perceived outcomes. This is in line with findings that youth and service characteristics explained a small fraction of the variance in parents' experiences with CAMHS (Bjørngaard et al., 2008;Garland et al., 2007).
Consequently, future studies should include characteristics such as alliance with the helper and other organizational and treatment-specific factors as possible predictors of quality of care.

| LIMITATIONS
One limitation in this study is that we only had foster parent reports of quality of care. Studies have shown that there are weak-tomoderate correlations between service satisfaction of parents and youth (Biering, 2010;Garland et al., 2007;Turchik et al., 2010). Therefore, we cannot infer youths' perceptions from their foster parents' responses. However, foster parents' experiences are important in their own right as foster parents are also users of youth and family services and depend on sufficient service support to provide nurturing and stable homes for youths in their care (Benesh & Cui, 2017;Tonheim & Iversen, 2018 Moreover, as foster parents rated the items in the service experiences questionnaire very positively, there was a possibility of a ceiling effect in the measure, which could have inhibited our ability to detect associated characteristics and identify differences between service providers. Previous studies has also found high levels of service satisfaction, and authors discussed possible ceiling effects (Bjertnaes et al., 2008;Turchik et al., 2010). Although foster parents rated the single items positively, only 5.5% of responders had the highest possible score in the service experiences index. As ceiling effects are considered to be present if more than 15% of respondents achieve the highest score (Terwee et al., 2007), a ceiling effect in the overall service experience scores was not indicated. Furthermore, as we measured service use over the last 2 years, the timeframe for user-rated quality of care was wide. This is positive for capturing experiences with service support given over time but reduces the specificity in the evaluations.

| CONCLUSIONS
This paper presents foster parents' experiences with services for youths in their care and indicates that foster parents experienced the services as useful and about half reported positive changes in youth condition and functioning following the service contact. Foster parents evaluated clinicians' communication and professional skills highly, while they gave poorer evaluations of information about the child's condition, cooperation between services and coordination of the work. Interagency collaboration is especially important for high-risk groups like foster youths that often depend on simultaneous services from different providers, and may be improved by using screening procedures and having routines for sharing information and cooperation. Furthermore, we found that CWS provided less information about youths' mental health but had shorter waiting times and more improvement in youth function compared with specialized mental health services. These results indicate focus areas for the specific providers to increase quality of care and highlight the importance of screening procedures in the CWS specifically, as it may improve the information provided to both foster parents and other services, while specialized mental health services can benefit from assessing whether services lead to improvements in youth condition and functioning.
Our findings also indicate areas for services in general to focus on to improve quality of care for foster families. These include involving foster parents of older youths in treatment, having appointments at the needed frequency, reducing waiting times, and regularly evaluating services provided for youths in foster care. The characteristics studied in this article explained only parts of why some foster parents experience higher quality of care than others, and future studies could expand this knowledge by using longitudinal designs and including additional organizational and treatment-specific variables as possible predictors of service quality.

ETHICS
The study was approved by the Regional Committee for Medical and

DATA AVAILABILITY STATEMENT
Access to data is restricted by Norwegian law on medical and health related research. Information about the data and analysis is available from corresponding author Marit Larsen on request.