Assessment tools in pre‐licensure interprofessional education: A systematic review, quality appraisal and narrative synthesis

Interprofessional education (IPE) aims to provide students with the opportunity to develop and demonstrate the team working behaviours and skills that will lead to positive patient outcomes. This systematic review aims to identify and critically appraise the assessment tools used after a pre‐licensure IPE intervention and provide guidance on which tool to use according to the focus of the intervention.


| INTRODUC TI ON
A recent report by West and Coia, 1 commissioned by the UK General Medical Council (GMC), emphasised the significance of effective, supportive team working environments that benefits both patient outcomes and staff well-being. Research has shown that staff well-being significantly improves productivity, care quality, patient safety, patient satisfaction, financial performance and the sustainability of health services. 1 Effective team working, however, is not being achieved throughout the health care system for a variety of reasons, including, but not exclusively, overburdened services, lack of organisational support and leadership, and poor preparation for collaborative multiprofessional working in pre-licensure education and subsequent training. 2,3 Interprofessional education (IPE) that fosters the competencies underpinning effective team working has long been recognised as a strategy towards improving interprofessional working and collaboration. 4 It occurs when 'two or more professions learn with, from and about each other to improve collaboration and quality of healthcare'. 5 It is widely believed that IPE improves the health care system and reduces medical errors, thus improving patient safety. 6 Many professional regulators in the UK, such as the GMC, General Pharmaceutical Council and the Nursing and Midwifery Council, support IPE as an integral part of health professional programmes. [7][8][9] Teaching and learning environments are a complex system involving educators, teaching and learning context, environment, learning activities and the outcome, where all components interact with each other towards a state of equilibrium. 10 It therefore follows that many forms of conventional assessment will be inextricably linked to the specifics of that context. Keeping in mind the importance of constructive alignment, educators are challenged to devise teaching and learning IPE interventions with accompanying assessments that will reassure higher education institutions, health care organisations and the public that students are prepared for effective team working and collaborative clinical practice. 11 Clinical practice itself will also be highly context-specific (ie clinical environment, staff, patients) subject to unpredictable variability. As such, the design and delivery of effective IPE experiences with appropriate, fair and standardised assessments appears a formidable undertaking.
A critical review of the literature identified that most of the available studies about assessments for IPE focused on pre/post study designs and used self-assessment tools measuring student's perceptions of their own performance, rather than measures of interprofessional team working. [12][13][14] Tools that measure self-reported change have been reported as the most common assessment employed. However, self-assessment measuring a change in either student reaction or student perception is considered a weak evaluative approach and a more comprehensive assessment approach has been recommended. [13][14][15] Lapkin et al 16 critiqued studies for lack of strong methodological approaches; poor sampling techniques; a focus on self-report outcomes; and lack of blinding when applying the education intervention.
There is a myriad of IPE interventions and a congested landscape of published tools and scales and this means educators need to dedicate significant time both to identify the most effective IPE interventions and the best tools to measure the anticipated change.
This systematic review aims to answer the following questions: 1. What tools have been developed to measure student knowledge, skills, behaviour, competence or patient benefit after an IPE experience? 2. What are the strengths and limitations of the tools identified?
3. How can this evidence be used to inform the future design and delivery of IPE and its assessment?
To address these questions, we conducted this review to interrogate the constructive alignment of the curricular activities within a narrative synthesis, investigate and critically appraise existing assessment tools that have been employed to measure the impact of IPE across pre-licensure health care programmes, and develop a decision aid that can be used to streamline the efforts of an educator looking to measure change in their students after an IPE intervention.

| ME THOD
The work was conducted following the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) standards for quality of reporting systematic review. 17 The search strategy was refined in consultation with the research team and an expert librarian.

| Study identification
In July 2019, five electronic databases were systematically searched with no date limits applied: MEDLINE, ERIC, CINHAL, EMBASE and NEXUS website. Medical subject heading (MeSH) and keywords were used with 'AND' and 'OR' to narrow or broaden the search depending on the search strategy for each database. Details of search strategies based on each database are included in the Appendix S1.
In order to capture as many relevant studies, different keywords were used to ensure inclusivity of the range of health care students. Included keywords such as healthcare student*, healthcare education, undergraduate*, health education and education. Also,

| Study eligibility
The population, intervention, control and outcomes (PICO) framework 18 was used to specify the inclusion and exclusion criteria.
Studies were included if they investigated pre-licensure health care students (Population), such as: Medicine; Pharmacy; Nursing; Dentistry; Physical Therapy; Occupational Therapy; Speech and Hearing Therapy; Health Education, and Nutrition. The studies included were those where these students were assessed after an interprofessional activity. The Kirkpatrick model (Box 1) was used to classify the outcomes measured by the assessment tool employed. This model has been recommended by Reeves et al 11 as an evaluation tool for measuring outcomes of IPE interventions (see Box 1).
Patient benefit was defined according to the definition of the UK Department of Health, which states that 'The definition of quality in health care, enshrined in law, includes three key aspects: patient safety, clinical effectiveness and patient experience. A high quality health service exhibits all three' P.8. 19 Studies were included if it measured one or more of these three patient benefits.
To ensure capture of all available tools, studies reporting on the development and validation of a new assessment tool were also included even if they were not used with an IPE intervention. All studies written in English and available as a full paper were included. Authors of studies were contacted if full papers were unavailable online.

| Data extraction and synthesis
A data extraction sheet for included studies captured: study citation (authors, title, journal, year); country; study setting, study design and IPE design details including: professions involved; duration; number of students; aim and description of intervention and IPE assessment; outcomes measured; limitations of the study; findings reported and any suggestions for future work. The data extraction sheet is available in the Appendix S1.

| IPE outcome classification and constructive alignment
The outcomes measured after the IPE intervention were classified using the levels described in the Kirkpatrick/Barr model. 12 Constructive alignment was assessed through review of the reported IPE aims against the classification of the assessment outcomes.

| Quality appraisal of the IPE assessment tools
It was recognised at the outset that assessments of knowledge and patient benefit would be highly specific to the context of the IPE interventions, so recommendations to IPE educators relating to these forms of assessment would have to pertain to the complete intervention rather than just the isolated assessment. However, those assessments measuring behaviour and competency change (as there were none that assessed change in organisational practice) were not limited to a topic or clinical area, for example cardiology and diabetes.
These tools were critiqued using the two-part Quality Appraisal   Systematic narrative synthesis was adopted to present the results more widely due to the heterogeneous nature of the data included in the studies. Narrative synthesis is particularly salient to inform onward practice in the field being investigated, by providing information on effects but also implementation. 21 No statistical analysis was appropriate to apply to the heterogeneous quantitative data.

| Quality assessment
There were no ethical issues identified in conducting this review; therefore, no applications for approval were sought.

| RE SULTS
From 9502 returned studies, 39 studies met the inclusion criteria and were included for analysis as illustrated in the PRISMA flow diagram (see Figure 1).

| Study characteristics
Analysis of included articles demonstrated that the nursing profession was the most frequently represented (n = 27), 22

| Acquisition of knowledge
The most common research design to measure acquisition of knowledge was a pre/post knowledge test (n = 13), 27 Knowledge Assessment (UMKC-SBIRT KA) (n = 1). 24 Of the studies that adopted a pre/post testing approach, only two also had comparator groups. 27,38 However, in one such study, the comparator group existed as a control where no IPE intervention was introduced. 27 A good example was provided by Racic et al, who illustrated that students attending an interprofessional course on diabetes scored significantly higher in a multiple choice knowledge test than those attending a uniprofessional course covering the same material.

| Acquisition of skills
Two studies 30,59 aimed to measure the development of skills. One study compared student skills post-IPE and compared results to a previous cohort not exposed to the IPE intervention 59

| Patient benefit
Different tools were used to measure patient experience such as a post-appointment survey, 51  Identification of drug therapy problems (n = 1) 52 was used to assess patient safety and facilitated optimisation of medication use.
Referral due to a health issue identified (n = 1) 34 and managing chronic disease (n = 1) 48 were used to assess clinical effectiveness.
This led to patients receiving more specialist care thereby improving clinical outcomes.

| Behaviour and competency development
The terms 'behaviour' and 'competency' were used across the different studies interchangeably. In our analysis, there was not an attempt to interrogate this use of terminology and the defini-

| QuAILS results
After contacting the authors/developers of each tool (n = 11), 45.4% responded (n = 5) and 33.3% (n = 4) of the tools' critical appraisal were revised based on the supporting evidence provided.

| Qualitative features
In the 12 tools with validity evidence, the scales and subscales pertained to a range of behaviours and competencies that map to the WHO For more about qualitative features, see Appendix S1.

| Psychometric properties
Psychometric properties for each tool were appraised using the three standards: validity, reliability and scale and score 13 which are presented in Table 2.

| Individual performance tools
All tools partially met standards for validity. 22

| Team performance tools
PACT-Expert and PACT-Novice met most of the validity standards with the former performing more superiorly than the latter. 41 All team performance tools partially met the reliability standards. 25,37,41 Scale and score standard were met only in PACT tools. 41

| Individual and team performance tools
All tools met similar standards for validity. 30 For more about the psychometric properties, see Appendix S1.

| Decision aid
A flow chart has been developed to facilitate the decision making of an educator looking to select an appropriate IPE assessment tool for their IPE intervention (see Figure 2). It is recommended that the educator consider the following questions to aid the selection of the appropriate tool and note recommendations based on the results of this review: • What is the aim of the IPE intervention?
• Who is being assessed?
• What are the domains of interest?
• What tools are available that will measure the achievement of that aim in that context?
• What tools have the most evidence to support their reliability, validity and scales and scoring?  with cohort studies, was a failure to include any strategies to deal with incomplete follow-up. 34,50,59 Finally, the two studies that used a randomised control trial design, both lacked a clear follow-up strategy. RCTs are understandably challenging, for example blinding is not feasible given the nature of the IPE intervention. 44 Previous reviews reported that measuring change in attitudes using self-assessment tools was the most commonly employed assessment. 13,14 Authors have previously critiqued the quality of study designs and the low sensitivity of tools to measure change. 13 Blue et al 14 further highlight the diversity of interventions used and outcomes measured, thus reducing the potential for pooling of data and direct comparison and therefore contributing more meaningfully to the evidence base, concerns reiterated here. Also, most studies were single site studies, focusing on short-term outcomes with small sample sizes. These specific concerns are shared in this review.

| Quality assessment
Students' knowledge acquisition was most commonly assessed by pre/post knowledge tests. These were specifically contextualised to the IPE intervention and therefore are not widely applicable unless the same IPE intervention is also adopted. However, very few studies adopted a robust research design, that is having a non-intervention or control group to check that the reported change was due to the intervention being delivered using an interprofessional format.
Similarly, tools which measured patient benefit were tailored to the patient outcome rather than the IPE intervention, limiting a more general adoption. All but one study adopted an observational ap-

| CON CLUS ION
There is no single IPE tool suited to all IPE interventions. There are a wide range of possible approaches to assess IPE, with more tools being developed in recent years that measure change in behaviour and competency. The selection of an appropriate assessment tool is a decision to be taken alongside the conception and design of the IPE intervention.
This review provides educators with a decision aid about which tools might be most appropriate for the purpose of their planned IPE intervention.
Future work is required on using these tools with stronger research designs to discern if any change detected in students is as a result of the education being interprofessional in nature. Such an approach will provide evidence about the value of IPE within pre-licensure health care curricula.

ACK N OWLED G EM ENTS
Hailah Almoghirah acknowledges the funding that she received as a scholarship from King Saud University, Saudi Arabia.

CO N FLI C T O F I NTE R E S T
None.