By continuing to browse this site you agree to us using cookies as described in About Cookies
Notice: Wiley Online Library will be unavailable on Saturday 7th Oct from 03.00 EDT / 08:00 BST / 12:30 IST / 15.00 SGT to 08.00 EDT / 13.00 BST / 17:30 IST / 20.00 SGT and Sunday 8th Oct from 03.00 EDT / 08:00 BST / 12:30 IST / 15.00 SGT to 06.00 EDT / 11.00 BST / 15:30 IST / 18.00 SGT for essential maintenance. Apologies for the inconvenience.
There are arguments that a specialist service for adults with intellectual disabilities is needed to address the health inequalities that this group experiences. The boundary of such a specialist service however is unclear, and definition is difficult, given the varying experiences of the multiple stakeholder groups.
The study reported here quantitatively investigates divergence in stakeholders’ views of what constitutes a good specialist service for people with intellectual disabilities. It is the first step of a larger project that aims to investigate the purpose, function and design of such a specialist service. The results are intended to support policy and service development.
A Delphi study was carried out to elicit the requirements of this new specialist service from stakeholder groups. It consisted of three panels (carers, frontline health professionals, researchers and policymakers) and had three rounds. The quantification of stakeholder participation covers the number of unique ideas per panel, the value of these ideas as determined by the other panels and the level of agreement within and between panels.
There is some overlap of ideas about of what should constitute this specialist service, but both carers and frontline health professionals contributed unique ideas. Many of these were valued by the researchers and policymakers. Interestingly, carers generated more ideas regarding how to deliver services than what services to deliver. Regarding whether ideas are considered appropriate, the variation both within and between groups is small. On the other hand, the feasibility of solutions is much more contested, with large variations among carers.
This study provides a quantified representation of the diversity of ideas among stakeholder groups regarding where the boundary of a specialist service for adults with learning disabilities should sit. The results can be used as a starting point for the design process. The study also offers one way to measure the impact of participation for those interested in participation as a mechanism for service improvement.
People with intellectual disabilities (ID) are a heterogeneous group that have in common functional and intellectual impairments that begin in childhood and remain throughout life. People with ID often have multiple and complex health-care needs that are not always readily met by generic health services. As such, there is substantial evidence that this group of people experience high levels of health inequalities. Yet, government policy in the UK rightfully aims to promote a rights-based approach to enabling people with ID to be included as full citizens in society. To achieve this objective, additional support is required to overcome barriers to inclusion, usually delivered by independent support providers. In addition, specialist health support is required to address the range of health problems that are particularly common and may impede social inclusion and seriously affect well-being. This is the rationale for the provision of a ‘specialist service for adults with ID.’
In order for those articulating policy or developing services to provide the appropriate resources and expertise for such a service, it is important to clear as to what is required. The boundaries of such a specialist service, however, are often unclear. For example, a person with ID who has broken his leg should see an orthopaedic surgeon; however, a person with ID who has behavioural and/or mental health problems is likely best served by a specialist service rather than the generic mental health service. Between the two extremes, there is a grey area rather than a clear boundary. This paper explores this boundary in the context of the larger project of service design.
Defining this boundary is complicated by the large number of stakeholders involved in providing care. These include different agencies and professionals, such as local authorities, health-care trusts, general practice doctors (GPs), therapists and social workers, as well as people with ID, their families and paid support workers. Each of the stakeholders is likely to have had a different experience, and they are therefore likely to have their own view, thereby making it difficult to reach consensus on the role and responsibilities of such a service. The potential ‘conflicting expectations and objectives of multiple stakeholders’ pose a challenge in service design. Understanding the variation in stakeholder views then is an essential first step in the service design process.
The study reported here quantitatively investigates divergence in different stakeholders’ views of what constitutes a good specialist service for people with ID. The study was planned to realize the first step in Pahl and Beitz's design process, design requirements. In keeping with this design approach, we focus on ideas generated from those views. Specifically, we consider the number, value and agreement of ideas provided by participating stakeholder groups in a Delphi study intended to elicit the requirements, or boundary, of a specialist service for adults with ID.
We used a Delphi study, a consensus-building method for expert groups,[6, 7] to efficiently sample stakeholder ideas across a distributed geographical area with different services. It is a popular study design in health care to elicit opinions and achieve consensus on priorities for research[8, 9] or service development as a prelude to more conventional research studies driven by clinical questions. In contrast to the usage of a Delphi study to obtain consensus, this study was designed to looked for differences in the ideas of stakeholder groups.
The method allows for comparisons within and between stakeholder groups and structures the re-examination of different themes. A further advantage of a Delphi study is that it reduces the power differentials between conventional experts and experts by experience. As all ideas are fed back at the same time, social desirability effects, common in a face-to-face meeting of different stakeholders, are avoided. We recognize that despite the advantages of this study in addressing the research question posed, it is not suitable for sampling the ideas of people with ID and rarely has been used with them.
Walmsley and Johnson discuss with nuance the difficult trade-offs to be made between carrying out inclusive research with people with ID and how to address particular research questions that do not lend themselves to inclusive approaches. They argue that both approaches are valid if there is appropriate rationale. The larger project encompassing this study included a group of individuals with ID to whom proposed and finished research was presented in an appropriate manner for feedback. For the study reported in this paper, we chose to focus on gaining the ideas of the other stakeholder groups. We have put particular emphasis on carers and frontline health professionals as people directly affected by the design of services, but whose voices are rarely heard in their creation. We recognize, however, that the research cannot stop here and the use of other qualitative or participatory design methods[13-15], are needed to gain the views of those with ID.
The Delphi study consisted of three panels and three rounds, as depicted diagrammatically in Fig. 1. In the first round, all panels received a questionnaire concerning the role of an ID service and what constitutes a good service. The questionnaire presented two personas, or hypothetical cases stories, of individuals with ID. Personas are a common design tool to focus the elicitation of requirements. These fictional service users encourage participants to articulate their experience at the correct level of detail for design. They support the avoidance of abstract requirements, while helping people generalize their experience to a larger group of service users.
Participants were asked what care these hypothetical individuals would need and by whom it should be provided. The participants were also instructed to list other roles a specialist ID service should perform in cases beyond the presented examples. Finally, they were asked to name characteristics of a specialist ID service and criteria that would make such a service a good service. The focus on characteristics and criteria is a common design approach to encourage thinking in more general terms following the specific focuses of the personas at the beginning of the questionnaire. A copy of the questionnaire is available as a Data S1. All questions were free response.
The answers were grouped for each panel into three categories relevant to the design process: ideal attributes of a service, needs which people with ID have and possible service solutions which address these needs.[17, 18] To create these lists, the data were coded individually by both researchers using a template of finer categories derived from a preliminary qualitative study. Differences in the codes for a given answer were discussed, and a joint decision reached on the final theme. See Table 1 for coding categories and example themes. The full list of themes is available as a Table S1.
Table 1. Coding categories and examples of themes
Integration of different disciplines and services
Holistic view of needs
Being able to take sufficient time for clients
Problems with mainstream services
People with learning disability and the community/society
Address medical needs which are independent of ID
Balance care by family and role of society
Providing training for residential, supported living and carer staff
Rapid intervention team to address behavioural support and breakdown of placements
All themes raised by the three stakeholder groups were combined into a single list for each of the above three categories. The resulting three lists, attributes, needs and solutions, were fed back to each of the three panels as a starting point for in Round 2. Participants were asked to state if they thought the proposed attributes and solutions were appropriate and/or feasible. In addition, participants were able to indicate if they thought an item was unclear. The items on the list of proposed needs were ranked on a five-point Likert scale according to their perceived priority.
A third round was carried out to allow participants the opportunity to revise their decision in the light of the opinions of other stakeholders in their group. A list of all attributes, solutions, and needs was prepared for each group. Solutions and attributes were ordered by the rating they had received in Round 2 and cut above a set threshold; the average Likert scale score for each need was provided. Each group was fed back this list from Round 2 with the results from their own panel. Approval ratings in Round 2 were already very high, and the Likert scores of Round 3 did not change much. Therefore, this paper will focus on the results of Round 2, which represent the opinion of stakeholders without the influence of their peers.
The Delphi study had two unusual features put in place to enable the analysis carried out in this paper: multiple panels and panel crossover. Splitting participants into multiple panels was a study design decision to ensure that the voice of each panel was heard. Separate coding also enabled comparison of ideas across stakeholder groups. Crossover was achieved by starting with a common questionnaire (Round 1) and feeding back the combined theme lists (Round 2) before keeping the panels separate in the third round. This study feature allowed subsequent quantitative analysis of agreement and disagreements between stakeholder groups.
Three groups of stakeholders, each comprising one panel, were included in the Delphi study: researchers and policymakers, carers and frontline health professionals working in current services for people with ID. Researchers and policymakers were grouped together as conventional experts who provide abstracted, or generalized, knowledge. As the community of conventional experts in ID is small, they are usually well positioned to influence service provision. In contrast, carers and frontline health professionals offer two types of experiential knowledge as participants in services. These two groups are often not included in the design of services as they are neither conventional experts nor service users. While carers and frontline health professional are stakeholders in their own right, they may also offer a lens onto services as experienced by those they care for, albeit with differing priorities.
Recruitment strategies, in line with Powell, focused on representational quality in the panels, such that a range of views would be expressed within panels. Specifically, we aimed to recruit researchers with varying research foci, carers from different geographical locations and health professionals from multiple professional backgrounds. The aim was to recruit 20 participants per group, which is similar to advisory panels or expert commissions.1
The researchers and policymakers panel was comprised of 20 key figures in UK ID research and policymaking. Researchers were identified through analysis of two major publication database, PubMed and Web of Knowledge. Those with four or more publications in the last 5 years were selected. Policymakers were identified through the examination of those involved in the production of 26 of the most influential policy reports in the past 12 years. This first step produced a ‘long list’ of 110 names. This list was shorted to 37 with support from two prominent ID researchers by excluding those who were retired, provided only statistical advice, or were from the same research group. Those on the shortened list received a personal invitation to participate in the study.
The carers panel, consisting of 17 individuals, was recruited through approaching special interest groups, and representative organizations, both online and offline. These included the local authority's group of volunteer carer representatives, local carer support groups and ID mailing lists and forums. These groups then generated further interest in the study by word of mouth.
The third panel, frontline health professionals, was originally intended to be comprised of GPs. According to the policy of normalization, GPs are the default providers of health care for people with ID, but have not been part of the decision-making process in service development. We invited 65 GPs across one English county following a sampling protocol to ensure areas with different levels of prosperity were equally represented. After repeated efforts, only five GPs were recruited. As numerous frontline staff from ID services had contacted the lead researcher in order participate, and frontline staff are often a neglected stakeholder group in the design of services, as the preliminary qualitative study had shown, they replaced GP participation as the third panel. Nine self-selected individuals were recruited, with a bias towards health, as opposed to social, care professionals.
The data analysis reported in this paper focuses on the variation, particularly divergence, of ideas within and between stakeholder groups as captured quantitatively. The ideas themselves will be reported separately. We quantify the divergence of ideas by counting the number of distinct ideas in each group, considering the value placed on those ideas by all three stakeholder groups and then looking at agreement of ideas and their evaluation within and between panels.
To measure comparative idea generation, the overlap of ideas produced by each group was analysed. This paper specifically focuses on the reported attributes and solutions as these are of direct relevance to the design of a service. We derive the measure of value, from a combined rating, low/medium/high, for each idea by panel. This reveals both how a panel judged ideas that their panel members had proposed as well as how each panel values the ideas of the other panels. The combined rating amalgamated the appropriateness and feasibility measure into a single value to give an overall assessment of priority.
Finally, we consider how the assessment of the feasibility and appropriateness varied. To quantify intragroup agreement, we calculated the total number of items each participant agreed to and then computed the median and interquartile range (IQR) of this number across all participants in a panel. The smaller this range the more the panel agrees. To measure agreement between panels, the initial summation was done per attribute, calculating the percentage of panel members who agreed with it and then compared across panels. These analyses look at feasibility and appropriateness separately because the combined rating measure is discontinuous and hence does not capture how much spread there is in the assessment of an item. Using data with a finer granularity also allows more nuanced insights which otherwise would be masked.
Unique ideas contributed
Seventy-six unique ideas were generated about the attributes of a good specialist ID service. The researchers and policymakers panel raised 58 (76%) ideas in total, the carers panel 46 (61%) and the frontline health professionals panel 36 (47%). Figure 2 illustrates with a Venn diagram2 how these ideas overlap. Eighteen attributes were raised only by one or both of the carers and frontline health professionals panels. Notably, 11 attributes were raised solely by the carers panel. While the researchers and policymakers made the largest contribution, nearly a third of ideas came from carers and frontline health professionals.
A similar analysis was performed for ideas of solutions of how to meet the needs of people with ID. Sixty-two unique solutions were generated from 54 (87%) proposed by researchers and policymakers, 49 (79%) by carers and 42 (68%) by frontline health professionals. The overlap of solutions proposed by the stakeholder groups is presented in Fig. 3, which demonstrates that it is much greater than for attributes. Carers and frontline health professionals only added about 15% more ideas than the researchers and policymakers panel.
Value of ideas
Each panel valued their own ideas most, as shown in Table 2, although the affect is smallest for the researchers and policymakers. Researchers and policymakers rated 37 ideas as high priority. The origin of these ideas is illustrated in a Venn diagram in Fig. 4. It shows the number of initial ideas about the attributes of an ID service that were ranked as ‘high’ in the combined measure by the researchers and policymakers in the 2nd round. Seven (20%) ideas put forward solely by carers (and in some cases also by frontline health professionals) were considered valuable by researchers and policymakers. Carrying out the same analysis for the solutions, 25 solutions were valued highly. Ninety-six per cent of those were provided by the researchers and policymakers. The overlap between the solutions proposed by different groups is substantial, as shown in Fig. 5.
Table 2. First column: percentage of the combined codeset contributed by a particular group in Round 1. Following columns: percentage of the total amount of attributes rated as high priority by a particular group that had been raised by the group in the respective row in Round 1
Round 1: Combined codeset (%)
Round 2: Percentage of attributes rated ‘high’ by
Frontline Health Professionals (%)
Researchers and policymakers (%)
Bold values indicate each panel valued their own ideas most.
Raised in Round 1 by
Frontline Health Professionals
Researchers and Policymakers
The same analyses were performed for frontline health professionals (Figs 6 and 7) and carers (Figs 8 and 9). The frontline health professionals were very critical and only seven attributes and five solutions were high priority, providing too few items to draw meaningful conclusions. The carers found a total of 22 attributes high priority. Of these, more were initially raised by their own panel (16 attributes) than by the researchers and policymakers panel (15 attributes), although the researchers and policymakers panel had contributed more initial ideas (Fig. 2). Figure 9 shows that the same number of solutions that were ranked as high priority originated in the carers panel as in the researchers and policymakers panel. Again, the researchers and policymakers panel made a larger contribution of ideas, but the difference (Δ = 3) is smaller compared to the attributes analysis (Δ = 12).
Variation of idea assessment
Figure 10 represents the aggregated levels of agreement that an idea is appropriate. The IQRs are small, meaning that most of these items received comparable levels of approval regarding their appropriateness. As the medians of the three groups are close together, the approval level is similar across the groups. In contrast, Fig. 11 shows the median and the IQR per panel of the feasibility of an attribute. The medians of the three panels vary much more and the IQR of researchers and policymakers panel is much larger than the other two panels. Hence, researchers and policymakers are more optimistic (higher median) overall, but also differentiate more between items (larger IQR). Carers, on the other hand, are generally more pessimistic about feasibility (lower median), and this assessment is consistent across the panel (smaller IQR).
We also consider the agreement within a group. Figures 12 and 13 show the median number of ideas (out of 76) panellists judged to be appropriate and feasible, respectively. Figure 12 looks similar to Fig. 10; in both cases, the medians are close together and the IQRs are small. This means that within a panel, most participants ranked most ideas as appropriate. Figure 13, however, is strikingly different from Fig. 11. The most prominent feature is the very large IQR of the carers panel, which ranges from about 3 to over 57.
The participating carers had widely different opinions. Fifty per cent of the panel rated between 3 and 57 items as feasible. However, this means that a quarter of the panel thought <3 ideas feasible and the other quarter, more than 57 ideas. The IQRs for frontline health professionals and researchers and policymakers panels are both above 20 items, compared to 5–10 in Fig. 12. Thus, within these panels, there is also more disagreement about feasibility than appropriateness. The researchers and policymakers panel is the most homogeneous and optimistic regarding feasibility, while the carers panel is the least.
Our results indicate that while there is some overlap of ideas about what should constitute a specialist service for adults with ID, there is substantial divergence regarding feasibility both within and between panels. This can pose a significant challenge to service design if not addressed. Further consideration of the data suggests that the variety of ideas may be an asset in widening the views of other stakeholders. Carers, for example, generated an additional 30% of ideas about how a service could be delivered, many of which were valued by the researchers and policymakers. Frontline health professionals, although small in number, also contributed unique ideas that were valued by the other stakeholder groups. We would suggest that the inclusion of carers and frontline health professionals is beneficial in increasing the variety of ideas if the divergence is appropriately managed in the design process.
Interestingly, the contribution of carers was greater in describing attributes of a service rather than in providing solutions. In other words, carers generated more ideas regarding how to deliver services than what services to deliver. This is in line with user experience research in the design literature, which suggests that consumers add important insights about the experience of a service, but the designers contribute more intensely to the generation of solutions. Solutions exceeding incremental improvements often require ‘challenging and reframing the design brief’, a key skill of designers. There are similar findings from qualitative studies in the participation literature, suggesting that participants are more likely, and prefer, to be involved with elements that they directly interact with, such as patient information or transport, rather than questions of budget allocation.[26, 27]
The results also illustrate differences within and between stakeholder groups. Regarding appropriateness, the variation both within and between groups is small. This high level of agreement suggests that the design process should go smoothly. On the other hand, the feasibility of solutions is much more contested, with large variations among carers. Indeed, the fragmentation of the carer group is profound and reflects fundamental disagreements within the group. This finding indicates the efforts will need to be made to overcome these or at least manage them, topics which are currently neglected in the participatory design literature.
This study explores the boundaries of a specialist service for adults with ID, providing a representation of the diversity of ideas among stakeholder groups regarding where this boundary should sit. Those ideas for which there is agreement are a starting point for the design of such a service. Those ideas around which there is divergence raise areas that need to be addressed as a prerequisite of service design.
This study has focused on the design of a specialist service for those with ID; however, the method may have wider relevance to the participation literature in health research and service design. The quantification of participation provides another way to consider participation, augmenting the more standard approach of case studies. It adds to the literature that describes how to enable participation, a way to inspect how participation has played out once it has been enabled. Such a method can reveal the contribution of each stakeholder group, or perhaps more importantly from a service design perspective, where there is divergence of views which need to be addressed.
This study presents only one possible way to quantify participation, adding to the few studies published in this area.[10, 29] Yet, it may be of particular interest to those researchers who have raised concerns about the utility of participation to produce a better service,[26, 30-32] given the costs and the persistent difficulty of convincing health service providers and researchers of embracing it. The study is limited, however, to a particular conceptualization of participation, namely as a mechanism for increasing the value, or quality, of services. Value is only one driver for participation, and there are alternative mechanisms to consider the role of participation as an ethical mandate.[34-36]
Taking this particular conceptualization of participation, further work is needed to refine notions of what constitutes an idea and value. Approval of panel of another is a crude proxy for value. Doel provides an alternative definition of value as dialogue, but also suggests and that there is currently a scarcity of robust measures. The concept of value as a dialogue resonates with the service dominant logic in the design literature which states that value is always created in the interaction between the service provider and the customer and hence fundamentally subjective.[38, 39] A prerequisite for quantifying these types of interaction, however, is the presence of mechanisms to encourage participation.
The analysis in this study is brief, but provides a good starting point for further development of the measurement of the impact of participation. Several authors have pointed out the dichotomy between measuring processes and measuring outcomes,[34, 37, 41] whereby the latter is more subjective and harder to measure. The work which we present here provides a middle ground. By quantifying the impact of participation on the idea finding process, it goes beyond counting the number of participation meetings that have taken place, an example for a process measure, but avoids the difficulties of measuring the improvements in consumers’ quality of lives.
The validity of Delphi studies rest on the size of the panels. In this case, the carers and frontline health professionals panels were smaller than desired. The small panel size did not affect the statistics. The use of the interquartile range (IQR) means the results focus only on the spread of the central 50% of the respondents. Unless our sample includes more than 50% of people with extreme opinions, the analysis should be unaffected by the different size samples. Larger panels, however, would increase the chance of including all relevant views and most likely increase the number of ideas generated. This is particularly true when comparing ratings of appropriateness and feasibility, as over- and underrepresentation can lead to a distorted picture.
More participants would not have changed the existence of divergent opinions and hence would not have altered the general finding of notable disagreement within and between panels. Larger carers and frontline professionals panels would most likely make the observed effect more pronounced. It is also important to note that both the carers and frontline health professionals panels were self-selected as they were recruited through networks and word of mouth. This can lead to a lack of diversity on the panels and a bias towards those who are optimistic about the possibility of change. However, as both panels were quite pessimistic, this does not seem to be the case in this study.
Specialist services for people with an intellectual disability differ from other specialist health services as they are directed towards the needs of a group of people rather than a particular illness. For this reason, the boundaries of the service are far from clear. This is the first study to use the Delphi method to explore the extent to which the views of different stakeholders about such a service diverge. In extending this work, the next stage would be to identify those areas that there was agreement on and to use this to move to the next stage of service design – that is, from identifying the purpose of such a service to developing service structures.
We would like to acknowledge all of those people who participated in the study as well as the statistical advice of Adam Wagner. We further want to acknowledge Isabel Clare for her general advice as well as support in recruiting participants, and Elizabeth Jones, Alison Lillywhite and Sallyanne Broughton for their help in creating the personas. Further advice was provided by John Clarkson and Terry Dickerson. Thank you to our colleagues in the CLAHRC who provided comments on earlier drafts.
Conflict of interest
Source of funding
This article presents independent research commissioned by the National Institute for Health Research's (NIHR) Collaborations for Leadership in Applied Health Research and Care (CLAHRC) programme for Cambridgeshire and Peterborough. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. EMH's PhD funding was provided by the Gates Cambridge Trust and AH's work is funded by the Health Foundation.
E.g. Survey Adults with Learning Difficulties in England 2003/04 (panel size 23), Learning Disability Advisory Group (panel size 26), Health care for all (panel size 12).
Due to mathematical reasons, 3 circle Venn diagrams are never fully accurate, but they still provide a useful and intuitive illustration of the amount of overlap between the responses of the three groups. To complement this, each area is labelled with its exact size.