Increasing concern about adolescent mental health problems has resulted in schools being identified as central sites for their prevention (PsycInfo; ERIC 2000–2012). In this context, SEAL for secondary schools was launched in 2007.
Increasing concern about adolescent mental health problems has resulted in schools being identified as central sites for their prevention (PsycInfo; ERIC 2000–2012). In this context, SEAL for secondary schools was launched in 2007.
The implementation strand of the national evaluation of secondary SEAL comprised longitudinal case studies of nine schools and used multiple methods and data sources, including interviews, observations and document review.
Implementation was variable, both within and between schools; barriers were identified at programme, teacher and school levels.
Implementation issues identified have implications for SEAL and school-based mental health prevention and promotion interventions more generally.
There has been increasing international concern in recent years about the mental health of children and young people (World Health Organisation, 2005). An estimated 10% of children experience clinically significant difficulties (Ford, Goodman & Meltzer, 2003), with many cases remaining undiagnosed and/or untreated (Macdonald & Bower, 2000; Meltzer, Gatward, Goodman & Ford, 2000). Mental health difficulties at all levels impact negatively on children's quality of life (Rothi & Leavey, 2006), school attendance (Meltzer et al., 2000) and educational performance (Richards et al., 2009) and, if untreated, are likely to persist and develop into psychological difficulties in adulthood (Stallard, 2011).
As concern has grown about the increasing prevalence of mental health difficulties (Collishaw, Maughan, Natarajan & Pickles, 2010), schools have increasingly been viewed as ideal settings for reaching vulnerable and undiagnosed children and adolescents. Although some school-based interventions ‘target’ identified pupils who require specific support (see, for example, Kendal, Callery & Keeley, 2011), most are ‘universal’, that is for all pupils regardless of perceived need. These interventions typically take a ‘prevention and promotion’ approach designed to prevent mental health problems through the promotion and development of skills and strategies to maintain or achieve emotional health and well-being (EHWB). This approach may effectively ‘immunise’ vulnerable children against later difficulties (Merrell & Gueldner, 2010) and avoid the stigmatisation that may result from targeted interventions (Greenberg, 2010). The universal prevention and promotion approach, supported by targeted and indicated intervention for those children who need it, is increasingly found in school-based mental health initiatives across Europe (Department for Education and Skills, 2005, 2007; Fundacion Marcelino Botin, 2008), in Australia (e.g. Graetz et al., 2008) and the United States (e.g. Barrett, Eber & Weist, 2012).
Social and emotional aspects of learning for secondary schools (11–16 years) is a whole-school approach designed to improve learning and attainment, positive behaviour, regular attendance, staff effectiveness and the emotional health and well-being of all staff and pupils (Department for Education & Skills, 2007). These outcomes are expected to be achieved through the development of key social and emotional skills classified under the five domains proposed in Goleman's (1995) model of emotional intelligence, that is self-awareness, self-regulation (managing feelings), motivation, empathy and social skills. SEAL was designed to be flexible, to allow schools to tailor it to their own contexts and needs (Weare, 2010) and this is reflected in a lack of prescribed instructions, specific lessons or a fixed model for implementation. The Department for Education and Skills (2007) advises, however, that the skills may be developed and outcomes achieved through the implementation of four key interrelated components and processes. These are the explicit teaching of the key social and emotional competences, both as specific learning opportunities and within the wider curriculum; a whole-school approach to the creation of classroom and school climates that promote the development of these skills and provide opportunities for them to be practised and reinforced; continuous staff development to enhance skills and understanding of the benefits of social and emotional learning; and the use of teaching and learning approaches that create a positive learning and social environment. SEAL is primarily a universal intervention; it is expected, however, that pupils who may benefit from additional support are offered more targeted, ‘focus group’ work.
The national evaluation of SEAL in secondary schools was commissioned by the then Department for Children, Schools and Families (DCSF) and conducted in 2007–10 (Humphrey, Lendrum & Wigelsworth, 2010). The two interrelated aims of the evaluation were to (a) assess the impact of secondary SEAL on a variety of outcomes for pupils, staff and schools, and (b) to examine how schools implemented SEAL. The relationship between levels of implementation and outcomes was also assessed. This article focuses on the second of the two main strands, but a full discussion of the impact of secondary SEAL can be found in Humphrey et al. (2010) and Wigelsworth, Humphrey and Lendrum (2012). To set the appropriate context for this article, however, we note that the quasi-experimental impact evaluation failed to yield positive outcomes [Wigelsworth, Humphrey and Lendrum (2012)].
The examination of the processes of implementation as part of a programme evaluation is essential for a number of reasons (for a full discussion see Lendrum & Humphrey, in press). Research across multiple disciplines has consistently shown that organisations, including schools, typically fail to implement programmes with fidelity (that is, as intended by the developers), and that this is likely to negatively impact upon the achievement of the expected outcomes (Durlak & DuPre, 2008). Lack of fidelity may be due to intentional adaptations as implementers seek to improve the ‘goodness-of-fit’ between the programme and their own needs and contexts, or may result from barriers to implementation at programme, organisation or implementer levels. The assessment of the implementation of SEAL was therefore intended to examine the extent to which schools implemented aspects of the programme as suggested in the guidance materials, explore local adaptations and the reasons for them, and to identify barriers to and facilitators of implementation. As a flexible framework for promotion and prevention, the study of the implementation of secondary SEAL has the potential to yield important insights that have implications for the design and execution of universal school-based interventions more generally. This article therefore focuses on the implementation variability within and between schools and the barriers to effective implementation that may have influenced this variability (for a broader analysis of the implementation of secondary SEAL see Lendrum, 2010 and Humphrey et al., 2010).
Our evaluation of the implementation of secondary SEAL employed qualitative methods, using a multiple-case study design. This approach was intended to provide in-depth accounts of how schools implemented the four key components of SEAL, the extent to which they took a whole-school approach and the operation of barriers and facilitators to implementation. The flexibility of SEAL and the lack of prescribed activities and processes meant that the more typically employed quantitative measurement of aspects of implementation fidelity – dosage (frequency and duration of delivery) and adherence (to specified processes) – was neither feasible nor potentially informative. The study was approved by the University of Manchester Research Ethics Committee.
Schools were visited approximately once per term over five terms to enable in vivo examination of the processes of implementation. Data were collected using multiple methods and sources, summarised in Table 1.
|Case Studies (CS)||CS2||CS3||CS4||CS5||CS6||CS7||CS8||CS9||CS10|
|Outside classroom (e.g. lunch)||2||2||2||2||2||2||2||3||2|
|Interviews||Pupil focus group||2||4||2||4||4||4||4||4||4|
|Head teacher or deputy||2|
Interviews and observations were semistructured. This supported the collection of equivalent data for comparison across sites and between respondents, allowed for the emergence of unanticipated themes and more detailed exploration or clarification as necessary. The development of schedules was initially informed by the literature, the research aims and the characteristics of SEAL. The analysis of data after each visit allowed the modification of schedules to allow progressive focusing and case-specific questions were added for the collection of contextual data.
All secondary schools (c300) identified by the DCSF as implementing SEAL were invited by letter to participate in either or both strands of the study. Forty-eight schools volunteered as case studies and 10 of these were selected on the basis of geographical convenience. One school dropped out in the initial stages of the project. Schools were selected from seven different local authorities (LAs) in the North-West and South-East of England, to allow a greater comparison of LA involvement and support and its impact. Although the sampling strategy was inevitably purposeful and constrained by travel and budgetary considerations, schools were selected as far as possible to provide maximum variation in size, school type (comprehensive/selective), single- or mixed-sex, urbanicity, and the proportion of pupils speaking English as an Additional Language. Information on these characteristics was taken from the DCSF Performance Tables (2006) and schools' most recent Office for Standards in Education (OFSTED) Reports, accessed via the OFSTED website. Each school had some unique and some shared characteristics (see Table 2, below) to maximise the potential for the application of findings to multiple contexts.
Within-case sampling decisions were purposeful and informed by the type of data required to answer the research questions, expectations about who could provide this and the need to triangulate methods and data sources to enhance the credibility and validity of the research. In some instances, such as classroom observations and interviews with staff and pupils, participants were selected by the SEAL lead (SL), the main point of contact in each school. This was necessary due to the availability of participants and limited involvement and/or awareness of SEAL among school staff – hence the variability in the breadth and depth of data across schools (Table 1). Plans to interview parents and members of the wider community were abandoned due to their lack of involvement by schools. Informed consent was obtained from all participants.
Recorded interviews were anonymised, professionally transcribed and then imported with field notes of observations and reviews of documents into NVivo 7 (updated to 8) for data management. Data were thematically analysed following the standard procedure proposed by Braun and Clarke (2006). A framework for the generation and initial analysis of data was derived from Greenberg, Domitrovich, Graczyk and Zins's (2005) model of barriers to implementation, Durlak and DuPre's (2008) ecological systems framework, the SEAL guidance materials (Department for Education & Skills, 2007) and the researchers' initial impressions following the first visit to schools. Amendments were made and additional codes added during the initial analysis of data from the first school visits to better reflect the data and represent unanticipated findings and emergent themes specific to SEAL and the English educational context. These codes were applied to all the data generated during the first four visits, with both positive and negative instances included, for example lack of training as well as training delivered. Data were analysed following each visit and arranged into a series of within and between-school data displays (Miles & Huberman, 1994) to summarise all the available data, explore convergences and divergences, reveal gaps in the data and identify potential themes and relationships to be explored.
Prior to the final visit to the case-study schools, the coding scheme was reorganised and earlier data revisited to identify any gaps in the data. Following the final analysis, factors were regrouped into themes at programme, classroom and school levels for ease of discussion.
Validity and reliability were supported by the triangulation of data generated through repeated visits over an extended period of time and the use of multiple methods and data sources. The semistructured nature of interview and observation schedules allowed progressive focusing and the emergence of unanticipated data rather than reliance on preordained categories. Intercoder reliability was established. Iterative analysis and the creation of displays to explore the data from multiple perspectives allowed the generation and assessment of rival conclusions and alternative interpretations. Inferences were supported through regular respondent checks.
A thematic framework derived from our analysis is presented in Figure 1, and provides the main structure for this section. The focus of this article is on those factors that may potentially threaten the implementation of mental health prevention and promotion programmes in secondary school settings; detailed discussion of all the factors affecting implementation may be found in the main evaluation report (Humphrey et al., 2010.
The implementation of SEAL was variable both across and within the case-study schools. No school fully implemented all four recommended components of SEAL, although some made more progress than others. This was possibly due to a greater ‘goodness-of-fit’ between the existing school culture and the SEAL approach, although this also resulted in a perception among some staff members that it was ‘not needed’. Fuller implementation may possibly have been seen over a longer period of evaluation (e.g. >2 years); however, the evidence indicated that levels of activity declined in the 2nd year as SEAL became less of a priority.
Consistent with the findings of previous research (e.g. Durlak & DuPre, 2008; Greenberg et al., 2005), our analysis revealed a range of barriers to and facilitators of implementation at programme, classroom/implementer and school levels. These are presented in Figure 1 as distinct ‘themes’ in the interests of clarity; however, they clearly interrelated with one another (for instance, a lack of staff buy-in was affected by multiple competing initiatives, time and priorities and contextual characteristics of secondary schools).
There was ambivalence in schools towards the complexity and flexibility of SEAL and whether the multiple possible models of implementation operated as a barrier, facilitator or potentially even both. Some welcomed the autonomy – it ‘credit[s] teachers with intelligence’ (SL, CS10) – but most also found it confusing and wanted more guidance on implementation:
When I first set off doing it thinking ‘oh no, I'm going here, I'm going there…’ it balloons ‘cause it involves so many things… and it's not… people have got in their mind all the time, saying, ‘well is this SEAL really?… and it's just… it encompasses so many things.’ (Teacher CS10)
At the implementer level, attitudes to SEAL were complex and varied. The majority of staff commented on the general benefits of the underlying principles of SEAL. This was not unexpected, as most staff involved in SEAL had volunteered to participate. However, not all believed it was necessary in their schools. Some staff members objected to an ‘imposition of values’ and the ‘homogenisation of pupils’ while others were dissatisfied with the superficiality of the SEAL approach, the language used and the encouragement of ‘over-familiar’ behaviour from pupils.
Some teachers believed SEAL was ‘irrelevant’ and there was variability between and within schools in the extent to which teachers integrated SEAL across the curriculum; this was more likely in subjects such as English, Art and Drama, than in more ‘rationalist’ subjects such as the Sciences and Maths:
The… youngsters and the staff universally value least those subjects as they get older they don't see any relevance to their immediate priority, which is getting GCSEs (Headteacher, CS9)
I've got fifty minutes and my priority is that they leave the room… knowing about particle theory, you know, the fact that they're emotionally illiterate, well really… it's not your problem is it? (Teacher, CS4)
Although some staff expressed support for SEAL they were less willing to implement it:
What I could produce and show you would be… the whole of the Year 7 schemes of work, areas of study for all the subjects and how they've fitted in and jigged things around to meet the themes that we're teaching in SEAL. The reality of that – I am honestly not sure if it is happening in reality (SL, CS8)
Some were reluctant to invest time and effort when they feared that SEAL would be abandoned or replaced before it had time to achieve any impact:
You get the, ‘isn't it just another one of these ideas from the government that will fade out? We'll do it for a couple of years and then it'll be… we've got another idea now’. There is a little cynicism from people [who are] a bit weary of initiative after initiative (SL, CS10)
A second key factor at implementer level was the capacity or ability of teachers to deliver SEAL due to limited knowledge, skills and the unfamiliarity of the underlying concepts. Many staff members, including those who believed in the need for and benefits of SEAL, expressed confusion or a lack of confidence, and although initial training was available for all staff, the quality and extent of this was variable.
Sometimes I struggle thinking ‘oh how can I…what can I do with this? (Teacher, CS7)
A major factor at the level of the implementation environment was ‘time constraints’, which interacted with various factors within and across levels. A perceived lack of time prevented some supporters of SEAL from delivering it in the classroom:
It is all to do with time really ‘cause lots of people are interested and have got lots of ideas, but then it's about when do you do it? (Teacher, CS6).
Multiple initiatives in school resulted in competing, and at times conflicting, priorities:
There is so much else coming into school and you can only ask people to do so many things. People are pulled in different directions and dedicated staff are pulled in different directions and that's hard (SL, CS6).
Given the overarching governmental emphasis on academic outcomes, SEAL was at times forced to take a back seat:
I know that maths, English and science will take priority and I know SEAL…is going to be the bottom of the pile (SL, CS7)
Time constraints and multiple initiatives combined to restrict opportunities for staff professional development and training, which in turn impacted upon implementer skill. Hence, most schools delivered initial SEAL briefing sessions to both teaching and nonteaching staff, but ongoing training was a priority in only a few schools:
Its very intensive in terms of staff training needs and then of course you're battling to find training time because of course there are all sorts of other things that schools also want to do (LA co-ordinator, CS9)
The national evaluation of SEAL in secondary schools found that there was no significant impact on pupil outcomes in terms of social and emotional skills, mental health difficulties and behaviour (Humphrey et al., 2010). This was the case in all schools, regardless of apparent progress made in the delivery of SEAL, and there was no evidence of a relationship between level of implementation and the achievement of expected outcomes (ibid). Although this has serious implications for the future of SEAL specifically, there are also important lessons to be learnt about potential barriers to the implementation of other universal approaches to the promotion of mental health in secondary school settings.
At the programme level, the flexibility of SEAL was designed to enable the integration of existing initiatives, support ‘goodness-of-fit’ with a school's needs and thus encourage ownership and commitment to implementation and sustainability. This flexibility, however, resulted in vague guidelines and a lack of clear and specific instructions for how SEAL should be implemented and delivered, leaving schools confused and unsure of how to progress. This suggests that although schools appear to welcome flexibility and autonomy (Smith, O'Donnell, Easton & Rudd, 2007), this should be within a more prescribed framework of implementation. This is supported in the broader implementation literature, which recommends that implementers should be made aware of how a programme works, including which components are essential for the operation of the mechanisms of change and which may be adapted to improve compatibility with an organisation's needs and context (that is, the ‘must dos’ vs. the ‘should dos’ – Greenberg et al., 2005). Future programmes should not only aim for a balance between adaptability and prescription but also ensure that implementers are fully aware of how the programme is intended to work, so that expected outcomes may be achieved.
The barriers and implementation difficulties at teacher level – primarily staff ‘will and skill’ – appear to have been compounded by the limited guidance for and flexibility of SEAL. The low staff involvement in SEAL and the reliance on ‘volunteers’ who already appreciated the benefits of the promotion of EHWB implies that there was insufficient information to improve staff understanding and awareness or to overcome reluctance at a more general level. Even among those staff who showed the ‘will’ to implement SEAL, the lack of guidance appeared to result in uncertainty about how to proceed, suggesting that there should be greater emphasis on developing skills, confidence and competence. This is not an isolated finding; several studies have shown that teachers often feel inadequately prepared to manage the needs of pupils with mental health difficulties and would benefit from more training (Child & Adolescent Mental Health Services, 2008; Kidger, Gunnell, Biddle, Campbell & Donovan, 2010; Loades & Mastroyannopoulou, 2010; Rose, Howley, Fergusson & Jament, 2009; Rothi, Leavey & Best, 2008). The failure to develop staff understanding and skills may present one of the biggest barriers to the successful implementation of school-based MH prevention and promotion programmes.
It is important to note, however, that the SEAL guidance included ‘continuing staff development’ as one of its key recommended processes and schools' general neglect of this should be more closely examined. It may, perhaps, have been that the significance of this element was not sufficiently emphasised; however, the failure to deliver more than an initial staff briefing in most of the case-study schools may also reflect barriers at school level, particularly the shared contextual factors inherent in secondary schools.
The influence of characteristics of secondary schools on the implementation of EHWB programmes has not been widely studied. Much of the research around the promotion of mental health and social and emotional skills in schools has focused on interventions at primary/elementary school level (Durlak, Weissberg, Dymnicki, Taylor & Schellinger, 2011) and indeed secondary SEAL itself was informed by this research (Department for Education & Skills, 2007). There are, however, major differences between primary and secondary school settings and these present particular challenges, which are rarely discussed in the literature. This perhaps explains why the SEAL guidance materials (ibid) acknowledged these potential barriers, but did not provide advice on how to overcome them. Institutional size presents a telling example; secondary schools are typically much larger than primary schools and the multiple subject departments and greater numbers of staff inevitably present greater organisational and management challenges when attempting to implement change (Berman & McLaughlin, 1976). Unless there is good communication between departments, a coordinated whole-school approach may not be feasible. This may be particularly problematic for programmes like SEAL, which emphasise the application and reinforcement of skills and thus depend on a consistent approach from the multiple members of staff with whom students interact on a daily basis.
The size and structure of secondary schools also means that the child-centred approach of primary education is difficult to replicate, particularly as pupils are typically taught less frequently by more teachers, restricting the opportunities for supportive relationships to develop. The greater emphasis on formal curriculum subjects and academic attainment and the judging of teachers and schools on examination results may support a subject-focused, rather than child-centred ethos in secondary settings. Previous research has suggested that secondary school teachers may be particularly resistant or indifferent to nonacademic interventions (Berman & McLaughlin, 1976; Keating, Kerr, Lopes, Featherstone & Benton, 2009; McLarty, Highley & Alderson, 2010), perhaps explaining the low levels of teacher involvement seen in the current study. This, arguably, should have been anticipated for SEAL, as Weare and Gray's (2003) review that supported the development of SEAL reported greater staff resistance towards the promotion of pupil EHWB in secondary schools. This was believed to be partly due to a lack of understanding of the concept and insufficient training on how it should be taught, factors which have since emerged in other research, including the current study.
However, opportunities for training to improve understanding and skills are limited by multiple competing priorities – another predominant feature of secondary schools – which may potentially restrict the adoption of new, nonacademic interventions. ‘Time’ is a valuable resource and curriculum preparation and training time are only available for the highest priority initiatives. This may present a particular problem for EHWB promotion in secondary schools. Although improved understanding of the contribution of EHWB to broader outcomes may raise the status of related interventions, there has to be sufficient existing awareness of its significance for it to be afforded a high enough priority to warrant the input required to develop this understanding. This is perhaps the key problem that needs to be addressed before universal mental health interventions are afforded the status that is required for their successful implementation in secondary settings.
It may be argued that if reluctant staff members are not persuaded of the importance of the universal promotion of EHWB (Stallard, 2011), then perhaps a more targeted approach should be considered, in which only pupils with an identified need are subject to intervention. The advantages of this would be that fewer staff would need to be involved and, potentially, more effective use made of limited resources. However, this argument neglects the numerous, important benefits that may result from universal prevention and promotion programmes, not least an improved awareness and understanding of mental health issues, which may result in earlier recognition of difficulties or encourage self-referral or help-seeking behaviour (Kendal et al., 2011).
The use of qualitative methods necessitated a limited number of case studies in two geographical regions and this may imply that findings are context-specific. The potential for the application of findings to multiple contexts is improved by a maximum variation sampling strategy, however, and the broader applicability of the results is supported by the convergence of findings with the emerging literature in this area. Schools volunteered to participate in this study, and consequently it must be assumed that they were at least partially receptive to SEAL. The anticipation of regular visits from the researchers is also likely to have affected the implementation process in some of the schools, particularly in motivating progress and the addressing, or even denial, of barriers. There may also have been bias towards support for SEAL among interviewees as only those staff members involved in delivery were able to comment and the majority of these had volunteered to deliver it or join the working party. Despite the potentially more favourable contexts for implementation, however, this was variable and partial in all schools. Although this might have been seen to improve over a longer period of evaluation (e.g. >2 years), the evidence indicated that levels of activity declined in the 2nd year as SEAL became less of a priority.
This article has highlighted several key lessons learnt from the examination of difficulties in implementing secondary SEAL. Some of these have also been seen in other EHWB programmes across the world, which strengthens their generalisability, but also leads the current authors to wonder why these were not addressed sufficiently in a programme that was ostensibly designed to build upon the substantial research base in this area.
In terms of implications for the future of school-based promotion of EHWB, programme complexity and flexibility is a key consideration. This study demonstrated that simplicity and parsimony is perhaps advantageous, and suggests that we may need to think more carefully about the balance between adaptability and prescription (see Lendrum & Humphrey, in press). Our analysis also demonstrated the need to develop greater awareness and understanding of the significance of EHWB in schools (although recent changes to the school inspection framework, which removed the strand on personal development and well-being, suggest that the coalition government may be ‘throwing out the baby with the bathwater’). Staff members need to feel better supported and prepared, so that they are confident and have the necessary knowledge, skills and self-efficacy to play their part in this important aspect of schooling. Many of the barriers reported in this article appear to have been compounded by the secondary school context. We argue, therefore, that there needs to be a greater awareness of the complexities of secondary school settings when considering EHWB promotion, and, crucially, that further research on this important issue should be undertaken. Finally, the failure of SEAL – in terms of implementation and outcomes – should not be seen as the death knell for the promotion of EHWB in secondary settings. Rather, the lessons learnt from its evaluation should prompt a renewed interest into how we can best create educational environments that encourage positive mental health among all students.
The national evaluation of the secondary SEAL programme was funded by the Department for Education (formerly the Department for Children, Schools and Families). The authors have declared that they have no competing or potential conflicts of interest.