Supporting public involvement in interview and other panels: a systematic review

Abstract Background Members of the public are increasingly being invited to become members of a variety of different panels and boards. Objective This study aimed to systematically search the literature to identify studies relating to support or training provided to members of the public who are asked to be members of an interview panel. Search strategy A systematic search for published and unpublished studies was carried out from June to September 2015. The search methods included electronic database searching, reference list screening, citation searching and scrutinizing online sources. Inclusion criteria We included studies of any design including published and unpublished documents which outlined preparation or guidance relating to public participants who were members of interview panels or representatives on other types of panels or committees. Data synthesis Results were synthesised via narrative methods. Main results Thirty‐six documents were included in the review. Scrutiny of this literature highlighted ten areas which require consideration when including members of the public on interview panels: financial resources; clarity of role; role in the interview process; role in evaluation; training; orientation/induction; information needs; terminology; support; and other public representative needs such as timing, accessibility and support with information technology. Discussion and conclusions The results of the review emphasize a range of elements that need to be fully considered when planning the involvement of public participants on interview panels. It highlights potential issues relating to the degree of involvement of public representatives in evaluating/grading decisions and the need for preparation and on‐going support.

in providing feedback on health-care delivery or interventions, but in processes whereby decisions are made. 2 In healthcare, public involvement at all levels including Governing Boards has been advocated. 3 Organizations such as INVOLVE have developed to support public participation in NHS, public health and social care research. 4 Public representation on the majority of committees and panels is increasing, and the public are also often included on interview panels for health-care staff and during the recruitment of prospective healthcare students. 5 It is argued that lay people are in a good position to contribute to interview panels, in particular to evaluate communication skills, attitudes and values. 5 In response to this growing involvement, there is a need for guidance to inform organizations planning involvement, or members of the public who have been asked to contribute.
The aim of this study was to search for and identify any research or guidance relating to members of the public taking part in interview panels. The work intended to use existing literature to underpin development of future guidance, to support members of the public who are asked to take part in interviewing.

| METHODS
We carried out a rigorous and systematic review of published and unpublished literature relating to the involvement of members of the public in interview panels. An initial search was developed, comprising of terms relating to patient representation on interview panels. This initial search retrieved only a limited number of papers which specifically related to out topic area; patients on interview panels. When terms were added which related to training/guidance for patients to take part in interviews, very few citations remained. A second phase of searching was carried out in a wider set of databases, and using a broader range of terms. It took several iterations to get the search to a stage where it found a manageable number of citations, while still including studies that had been identified as relevant in the first search. Further details of the search strategy are provided as additional online material.

| Search strategy
In addition to standard electronic database searching, citation searching was undertaken later in the project (September 2015) and reference lists of relevant papers were screened. Searches for UK grey literature were undertaken in order to identify any reports or evaluations of "grass roots" projects or other evidence not indexed in bibliographic databases, and also to minimize problems of publication bias. We searched websites of relevant patient and public involvement organizations such as INVOLVE and also extensively searched for online documents using Google, with identified sources and terms used in the titles of these documents used as terms for further searching.

| Sources searched
As it was anticipated that relevant literature would be dispersed across a number of disciplinary fields, a wide variety of data sources were

| Search restrictions
We included work published in developed countries ([members of the

Organisation for Economic Collaboration and Development [OECD])
and published in English or with an English abstract.

Population
We aimed to include studies that related to public participation in interviews. A variety of terms may be used to describe this population including: lay member; patient and public representative; patient representative; expert patients; service users; and community members, consumers and lay advisors. Rather than attempting to define who we would consider to be "the public," we defined by exclusion, so considered any studies that related to individuals who were not professionals or not staff members employed by the organization hosting the panel. We searched for public involvement taking place in any context, including business, industry, public and private organizations.
As the terms "participation" and "involvement" can encompass a wide variety of activities and levels of inclusion, we deliberately adopted a broad definition. We searched for literature which described public individuals having any role in interviews or membership of a committee or panel, in a health-care context.

Interventions
We searched for studies which outlined preparation or guidance relating to public participants who were members of interview panels.
Due to the limited body of literature we located, we widened our criteria from interview panels to include public representatives on other types of panels or committees. The review focused on the process whereby the public was involved in decision-making activities, rather than looking for documents where public input had been sought (such as reports or guideline development). We scrutinized any literature describing involvement in interviews/panels/committees, looking for elements relating to preparation and training of public representatives.

Outcomes
We searched for literature which reported any outcomes following public representatives taking part in interviews or other panels. These could include actual or perceived benefits of members of the public taking part. We also included literature which did not report outcomes, and instead described the experiences of members of the panel, or provided guidance or policy recommendations.

Study design
Any study designs were eligible for inclusion, with articles that were more descriptive, in addition to empirical studies which reported data/results.

| Process of selection of studies
Citations retrieved via the searching process were uploaded to an EndNote database. This database of study titles and abstracts was screened by SB and MC. Full documents of all citations coded as potentially relevant were then retrieved for systematic screening.

| Data extraction strategy
Studies which met the inclusion criteria following the selection process above were read in detail and data extracted. An extraction form was developed to ensure consistency in data retrieved from each study.

| Quality appraisal strategy
We carried out quality evaluation only for the journal articles. We used the Centre for Evidence-based Medicine tool for critical appraisal of surveys. 6 This instrument requires a yes/no/can't tell response to 12 questions. For cohort (before and after) and mixed method studies, we used items from the Critical Skills Appraisal Tools checklist 7 with a yes/no/can't tell response to eight questions. For the qualitative/ action research studies, we used the CASP qualitative checklist 10 questions. 8 The full tools used are available as additional online material.

| RESULTS
Our searches identified 36 documents/papers that contained information relating to support, training or guidance for public representative members of interview panels, other panels and committees, or boards that had public members. From this database of citations, 59 potentially relevant papers were retrieved for further scrutiny, and of these, five met the criteria for inclusion.

| Quantity of the literature available
The second electronic database search using broader terms in more databases identified an additional 2204 citations that had not already been retrieved. From these, 33 potentially relevant papers were retrieved for further scrutiny. Examination of these articles resulted in an additional 21 papers that met the inclusion criteria for the review. In addition to these papers which were identified from electronic database searching, ten further documents were included from additional searching strategies (citation searching, reference list scrutiny or web searching).

| Type of the literature available
Our target literature related to public representation on interview panels, and we identified eight documents that related to this (see Table 1). 5,[9][10][11][12][13][14][15] One additional paper 16 described involvement in preparing interview questions, although not in the interview sessions. A further six documents considered public membership of other panels, such as those dealing with clinical audit, complaints or accreditation. [17][18][19][20][21][22] We identified seven documents that related to public membership of committees, such as research ethics committees. [23][24][25][26][27][28][29] The other papers we included related to public membership of boards  Table 2 provides a summary of the types of documents in the included set. As can be seen, just over half (19) of the documents we included were papers published in peer-reviewed journals. We identified a small number of reports (four), and 13 online documents, which ranged from one-or two-page guidance/recommendations to more substantial training manuals or guidelines.
The literature was dominated by studies originating from the UK (24 documents). This may be in part attributable to the inclusion of UK unpublished literature. Six articles originated from the United States and two from Australia (see Table 3).

| Quality of the literature
The included studies generally used less robust designs (see Table 4), with only one paper 29 including any form of comparator group, and two studies including initial (baseline/before) data, which were then compared to data collected later (follow-up/after). 32,34 The most common approach used by researchers to evaluate public involvement was self-reported surveys (questionnaires) that were completed by those taking part in an event.
The quality of the survey studies was affected by few of them using tools which were described as being previously validated, tested or piloted. The achievement of a reasonable response rate to surveys is always challenging, and some of these studies reported <50% response. For some of the studies, the authors reported only the number of completed questionnaires, and not the figure for those distributed. The group of survey studies predominantly reported findings as percentages, with few using statistical tests. The papers of other study designs were frequently limited by unclear/lack of reporting of elements, and limited reporting of study findings. See Table 5 for completed quality appraisals.

| Narrative summary of the literature
Analysis of the included documents identified a range of elements that need to be fully considered when involving the public in interviewing and other types of panels. These elements will now be sum-

Financial resources
The first aspect requiring consideration that was described in the studies, related to resourcing public involvement in interviewing, remuneration and the employment status of public members.
The authors of a study evaluating the involvement of service users in a social work training programme 16 described how they had intended to have users as co-interviewers, but had lacked the financial resources to pay them. This had led to their involvement being limited to suggesting interview questions. In another included paper, Gilbert 26 emphasized that financial or other recognition must be agreed in advance of implementing public involvement in interviews, with arrangements made for reimbursing travel costs or covering care. Roberts et al. 5 and Richardson et al. 14 similarly highlighted that resources need to be allocated. In the first of these 5 which evaluated the inclusion of lay people in student nurse interviews, the "lay involvement assistants" were employed on casual contracts.
A report 17 providing guidance on developing patient panels describes the issuing of honorary contracts. An example of information provided to lay council members also included details of remuneration. 33 In another study, people with learning difficulties who were involved in recruiting staff received £50 vouchers to recognize their contribution. 12 In contrast, a document providing information for lay interviewers of prospective medical students 15 describes the work as voluntary and unpaid, with reimbursement only of travelling expenses.
In a Canadian survey of lay representatives on health research committees, 28 around half were reportedly reimbursed in the form of a payment, meal, travel, conference attendance or general expenses.
In another study 23 10 Department of Health (2004) 11 Department of Health (2006) 24 Gilbert (2012) 26 Greco (2004)   A study of public involvement in accreditation assessments 21 described how some service user members provided only a verbal input to the report, whereas others contributed a written input. All participants in the evaluation of the system reported that there was a role for user members in feeding back/debriefing. This had led to the process being changed, with public members provided with a template for written reports, or given the option of presenting feedback verbally.
Another important element to consider regarding role was highlighted in four documents. These discussed who the public member was considered to be representing. A policy document 27 outlined that lay members are not expected to represent the views of the wider community (unless recruited because they are a member of a specific group and are authorized to speak on behalf of that group).
O'Hara et al. 28  The interview process Five papers highlighted the need to consider how public participants would take part in the interview process. The service users in one study 16 suggested questions, but did not take part in the interviews.
Matka et al. 13 in contrast outlined that a service user and a clinician interviewed each applicant together. Each interviewer evaluated the candidates' response using predefined model answers, and interpersonal skills were also assessed by both. Richardson et al. 14 described panels which included lay representatives, with all members contributing to asking questions. An alternative model described was of lay assistants meeting the applicants separately for half an hour before the interview and not taking part in student interviews. 5 Lay members were given tasks to do with the candidates around personal qualities and specific questions. Another study 12  However, they found that there was usually agreement.
O'Connor et al. 21 described a number of issues in relation to the role of the service user assessors in the accreditation assessments of health-care organizations. Some were asked to contribute to the rating of the organization and others were not, depending on who the team leader was. The authors reported that following their evaluation, this has now been clarified and staff facilitate the contribution of users to rating. In support of users having full involvement in evaluation, a paper 10 reported that overall comparison of scores given to candidates applying for GP training revealed no significant difference between the marks awarded by medical and lay assessors (P=.79). In this study, all assessors had attended the same training day which included discussion of scoring using video examples.

Training
The included documents emphasize the importance of training for public representatives before taking part in panels. Roberts et al., 5 for example, asserted that lay people should be briefed fully and initial and on-going training should be provided. Another study 28 reported that education enables well-informed and productive lay representatives.
The researchers in this survey suggested that education needs to be on-going, with training coming from multiple sources-online, face-toface mentoring, workshops and other forms of instruction. On-going training was similarly recommended by Ukpong 29 with update training at least once a year.
Three documents described a lack of training for patient representatives. Dougherty and Easton 30 found that lay board members usually received no training or orientation. Variability in practice was described in a study of research review boards. 36 A few had a full-time staff member to orient and train lay individuals, for others public members were told "just do what everyone else is doing." Similarly, another study 2 found that training was sporadic and varied considerably.

| Elements of training for involvement
Two papers described the benefits of the same training being provided to both lay and professional panel members. 10,24 Training was recommended in the following areas: 1. Knowledge of equal opportunities/equality and diversity. 11,14,15 2. Recruitment processes. 11,14 3. Confidentiality and data protection. 9,11,14,26 Three organizations reported that they required a confidentiality agreement to be signed. 17,27,39 In addition, the need to have a Criminal Records Bureau check was commonly described. Terminology Language was often mentioned as a barrier to meaningful involvement in panels. 23,40 Specialist jargon could be alienating, but business was described by committee members as being impeded if everything was "translated" into non-specialist language. 22 While it was highlighted that unnecessary terminology should be avoided, 26 there was a focus on lay members needing to learn terminology in order to be included.
It was suggested that public participants should receive a glossary of terms or "jargon buster" 22,27,35,38 or be trained in terminology. 32 Similarly, another document 24 suggested pairing with a buddy and handover between lay members. The induction process for a Safeguarding Children Board included a buddy system with new members paired up with those more experienced. 38 Oliver 22 reported that learning from others was a recurring theme; new panel members learnt from more experienced colleagues. New members in this study appreciated an induction day, but found this only a beginning to a long apprenticeship, and described the value of on-going support via mentorship. Support suggested for new members of panels included: practice sessions; someone to phone when something was unclear; and opportunities to meet others.

Consideration of other public member needs
Other elements highlighted as contributing to successful public involvement included ensuring that the venue was accessible, 24,27 sufficiently heated 14 and also that the timing of sessions was convenient. 24,26 Two studies described the need to consider Internet access and information technology support. 17,26 Another aspect of timing related to the time constraints and commitment of public members. Documents detailed the need to send papers well ahead of a meeting. 26 Buckland et al. 23 reported that factors hindering involvement included the amount of paperwork involved, the amount that needed to be covered in meetings and the accessibility and timing of meetings and training.

| DISCUSSION
This study aimed to systematically identify and summarize research or guidance relating to members of the public taking part in interview panels. We found only seven documents which specifically related to interview panels. Of these, four described services user/lay people taking part in selection panels for prospective health-care professional trainees and three outlined service user/lay members on staff recruitment panels. Other documents which we included related to public membership of other forms of panels or committees that were relevant to interview panels. Scrutiny of these two different groups of documents (interview panels vs other panels and committees) indicated common issues across them and supported our decision to extend the review to a broader inclusion criteria.
The body of literature was supportive of including public participants on panels, and several studies described benefits such as add- In some of the studies, public participants did not have an equal value in evaluation/scoring or only rated particular areas such as interpersonal skills. This raised questions regarding the value that is being attached to the contribution of the public member. One of the includ- The body of literature that we found was of limited quality, with a predominance of descriptive studies or those using a self-evaluated survey design. We echo the findings of other reviews of public involvement 43 which called for more research in the area and for the robust examination of public involvement in order that strategies may be evidence-based.

| CONCLUSIONS
The review indicates that public membership on interview panels was considered valuable, although a range of factors need to be fully considered when planning involvement. In particular, the intended role of the public member on the panel requires attention, together with sufficient preparation prior to the session and tailored on-going support.

CONFLICTS OF INTERESTS
No conflict of interests are declared.

SOURCE OF FUNDING
The Research Design Service Yorkshire and Humber is funded by the National Institute for Health Research. The views and opinions expressed are those of the authors and do not necessarily reflect those of the NIHR, NHS or the Department of Health.