The state of mixed methods research in nursing: A focused mapping review and synthesis

Aims: To consider the scope and quality of mixed methods research in nursing. Design: Focused mapping review and synthesis (FMRS). Data sources: Five purposively selected journals: International Journal of Nursing Studies , Journal of Nursing Scholarship , Journal of Advanced Nursing , Worldviews on Evidence-Based Nursing , and Journal of Mixed Methods Research . Review methods: In the target journals, titles and abstracts from papers published between 2015–2018 were searched


| INTRODUC TI ON
Mixed methods research (MMR) is a well-established research approach that integrates qualitative and quantitative methods to give a breadth and depth of understanding about the phenomenon of interest.MMR is said to combine the strengths of qualitative and quantitative research and to compensate for any limitations of the individual approaches (Pluye & Hong, 2014) thereby offering 'multiple ways of seeing' (Greene, 2007).
Since the development of MMR in the 1980s, it has become an important research approach in the social sciences (Creswell & Plano Clark, 2017).However, in nursing its development has been slow.
According to Flemming (2007), the methodological divide of qualitative and quantitative research is more entrenched in the nursing discipline, reflecting the medical hegemony in healthcare research, where randomized controlled trials dominate.That said, its use has intensified to the point where a scoping exercise, searching titles in CINAHL, showed that from January 2017-May 2018, 748 MMR studies were published in journals relevant to nursing, covering subjects as diverse as assessing students in practice (Burden, Topping, & O'Halloran, 2018) to managing deteriorating health in nursing homes (O'Neill, Dwyer, Reid-Searl, & Parkinson, 2018).Over the years many authors have justified the use of the approach as one that has the power to uncover important evidence that may otherwise be overlooked.Given the professed value of MMR, we set out to explore the current state of MMR in nursing through a focused mapping review and synthesis (FMRS).FMRS is a new approach to literature reviewing that has been described recently by Bradbury-Jones et al. (2019).
It is an approach to literature reviewing that differs from a 'traditional' systematic review where the aim is to synthesize evidence to discover 'what works' in a particular area.Instead, the FMRS seeks to: (a) explore a body of research in a pre-defined field; (b) understand the main theoretical, methodological, and epistemological assumptions that underpin the work; and (c) give a critical report on these assumptions and their application.In doing this, the strengths and limitations of the approach are brought to light.

| BACKG ROU N D
Mixed methods research combines the techniques of qualitative and quantitative research to address a range of complex research questions.Fielding (2012) states that MMR brings the findings from different methods together into a dialogue that gives a balanced view of a phenomenon.Numerous definitions of MMR have been tendered and Johnson, Onwuegbuzie, and Turner (2007) suggest that in MMR, researchers need to: Combine elements of qualitative and quantitative research approaches (e.g.use of qualitative and quantitative viewpoints, data collection, analysis, inference techniques) for the purposes of breadth and depth of understanding and corroboration (p. 123) This definition has been embraced by several authorities on MMR including Creswell and Plano Clark (2017) and Pluye and Hong (2014), all of whom stress that integration is at the heart of MMR and should feature through the design, methods, interpretation, and reporting stages of the research process (Fetters, Curry, & Creswell, 2013).Fàbregues and Paré (2018) promote the ability of MMR to consider multiple world views and argue that the advantage of integration is its ability to produce knowledge that transcends what could be generated from separate qualitative and quantitative studies.Similarly, as Fielding (2012) suggests, MMR allows for greater 'analytic density' that is achieved through data integration.

| Mixed methods research in nursing
Capturing the essence of nursing is challenging, not least because as Bender (2018) asserts, nursing is not clearly demarcated but rather involves: Interdependent relations that constitute people, including nurses, in their health/environment circumstance, which comprises nursing's unique, fundamental point of access in the world (p.6) As such, nurses need to combine diverse ways of knowing and apply this to care delivery in different contexts (Reed & Shearer, 2011).
MMR may offer a means of capturing the complex nature of nursing because the combination and integration of qualitative and quantitative approaches corresponds with the multidimensional practice that symbolizes nursing (Fàbregues & Paré, 2018).Crucially, Kim (2015) contends that nurses need to integrate these sources of knowledge to be able to articulate what comprises nursing and to practise nursing effectively.

What problem did the review address?
• What is the scope and quality of mixed methods research in nursing?
What were the main findings?
• Many mixed methods articles lacked detail regarding the underpinning methodological approach and theoretical explanation • While it is evident that the quality of reporting of mixed methods studies has improved over the past decade, overall lack of transparency still compromises quality • More than one third of articles showed evidence of at least partial integration of findings • High quality mixed methods articles provided details of integration and included a flow chart showing how the different aspects of the mixed methods design were integrated Of course, MMR offers more scope than capturing the nature of nursing; it is also a design that can be used to evaluate nursing practice (Bressan et al., 2016) and to give evidence about nursing interventions (Fàbregues & Paré, 2018).For example, Chiang and Chan (2014) used an MMR approach to evaluate advanced simulation in nursing, and Söderhamn, Kjostvedt, and Slettebo (2015) used MMR to evaluate ethical reflections in community health care.These authors valued the latitude of the MMR approach and claimed that the integration of qualitative and quantitative methods strengthened the results, giving a broader and more comprehensive perspective to their evaluation.Thus, overall, advocates of MMR claim that it offers the scope to explore both the experiential and intuitive facets of nursing knowledge and integrates these to give a detailed representation of phenomena.
However, recent evidence points to the limitations of a great deal of MMR in nursing.Younas, Pedersen, and Tayaben (2019) undertook a review of nursing journals to determine the 5-year prevalence of mixed methods research and to determine the extent of integration of qualitative and quantitative findings.They found that there was inadequate justification for using mixed methods in many research reports.
They concluded that data integration remains a challenge for nurse researchers (Younas et al., 2019).

| Aims
We are a group of researchers, lecturers, and postgraduate students who came together either to deliver or participate in teaching and learning about MMR.As part of the program of teaching, we looked in some depth at definitions and typologies of MMR.Since some of us have a nursing disciplinary background, we were struck by the variation in type and quality of MMR either undertaken by nurse researchers or by those investigating nursing issues.This led us to undertake the project reported in this article.The research question was: What is happening methodologically and theoretically in the reporting of MMR in nursing?
As we were interested in a profile picture of the current status of a phenomenon, we set out to explore this using FMRS.

| Design
We followed the three-stage FMRS approach detailed by Bradbury-Jones et al. (2019), which as the name of the review suggests, involves: (a) Focus; (b) Mapping; and (c) Synthesis.

| Focus
A unique feature of the FMRS is the identification of journals at the outset of the review process.Because we were interested in producing a profile of MMR in nursing, we had a clear disciplinary focus.Thus, we intended to contain our search to nursing journals and elected to search the top four journals listed under 'Nursing' in Scimago Journal & Country Rank (a website that displays the rankings of journals; see https://www.scimagojr.com/;accessed 11 August 2020) for the most up-to-date profiles available at the time (2018).This strategy, we considered, was likely to elicit the best quality reported MMR studies in nursing.We included journals that dealt with a broad sphere of nursing and excluded those with a specialist focus, such as education, management, or clinical specialties.Our included journals therefore are the highest ranked 'generic' nursing journals, not the highest per se.Since nurses do not restrict their publishing activity solely to nursing journals, we decided to supplement the nursing journals with that of the most prominent MMR journal at the time (the Journal of Mixed Methods Research).As a result, we searched the journals identified in Table 1.Before beginning the search process, we undertook a scoping exercise, where two members of the review team searched the indexes (the table of contents) of each of the five journals independently, to establish whether MMR featured in the journals.Following this rapid feasibility exercise, we then came together to review our search strategy and agreed on the journals and the time frame.
As is common practice in other forms of literature review (Aveyard, 2018), we also imposed time parameters on the FMRS.
We wanted to ensure that we retrieved sufficient contemporary literature to address our research question and initially we restricted the search to a 1-year period.Searching and retrieval of articles followed a stepped process.We established four search teams of two or three reviewers and each team was allocated one or two journals to search.Each member of the team worked through the process separately and documented the process as the review progressed.
Each reviewer began by scrutinizing the index of every journal issue in reverse chronological order from May 2018-January 2017.
Titles were searched for the words or derivative words 'mixed methods' to identify articles that met the inclusion criteria.Articles that reported on multi methods were excluded, as were MMR review articles and articles reporting on an isolated part of an MMR study.
To ensure we retrieved all relevant articles, each team member The FMRS incorporates a calibration process at each stage to add rigour to the process (Bradbury-Jones et al., 2019; Figure 1).In the focused search, calibration was an iterative process.This involved frequent points of contact and deliberation among the entire review team, firstly to set search parameters and then to review and revise these as necessary.Additionally, having completed our searches separately, the team allocated to each journal came together to compare and agree on their retrieved articles.Finally, the whole team regrouped to agree on the final articles for inclusion.
This process revealed the need to revise the search parameters.
Our initial search was contained to article titles and this proved fruitful in three of the journals; however, no MMR studies were captured from the search of Worldviews on Evidence-Based Nursing or the Journal of Nursing Scholarship.As a team we needed to decide whether we worked with the retrieved articles from fewer journals than intended or extended the search in some way.Since there was a danger of missing a fuller profile of nursing if we restricted journals, we agreed to extend the search.One possibility was to lengthen the time frame of the search, but we agreed that this was likely to yield more articles from the journals that we had already successfully searched, rather than those that had returned no hits.We elected to extend the search to abstracts as it was possible that the methodology was not disclosed in article titles and this approach proved partially successful.However, we then still needed to extend the search period going back a second year and then a third year and by taking this approach we retrieved at least one further article from each of the target journals published between 2015-2018.This, we felt, gave the review greater meaning and as a result, we retrieved 34 articles across the five journals (Table 2).

| Mapping
In their typology of reviews, Grant and Booth (2009) described mapping as the process of producing systematic maps to characterize studies in ways other than eliciting their findings.This, they suggest could include charting the studies' theoretical perspective, their population group, or the setting where they were undertaken.Two reviewers completed this mapping process for each article independently.Articles that met the inclusion criteria were read in full and the assigned reviewer extracted the information according to a predetermined extraction pro forma.The development of this pro forma was also subject to calibration.Here, using the most recent MMR article published in each of the five journals, two reviewers assessed the articles against the research questions and built the extraction form accordingly.This was then refined by the whole team.After each reviewer completed the mapping process for one article, all team members commented on the suitability of the extraction pro forma and some minor changes were made at this point (Table 3).Since we were interested in the reporting of the MMR design rather than the findings of the study, we did not extract data from the findings but rather, we drew out processes followed in an MMR study, as laid out by Pluye and Hong (2014).Put simply, we assessed what the authors said they did rather than what they reported as their findings.On this basis, we made a judgement about the quality of reporting in the articles using the six elements that formed the extraction process.We rated papers as 'good' if the elements were clear and explicitly described in the paper, 'poor' if most were unclear or missing and 'satisfactory' where some elements were reported and some were omitted or unclear in their description.We acknowledge that this judgement was necessarily subjective, but peer reviewed across the team to ensure agreement.
The review team then met as a group to undertake a calibration using a sample of the completed extraction documents.Here the two reviewers of each of the selected articles presented the completed pro forma and explained their judgments; and the whole team discussed any dilemmas or ambiguities to reach a consensus.
Finally, two independent reviewers checked a purposively selected 20% sample, across journal and review teams, to ensure accuracy and consistency in reporting.This process gave teams the opportunity to revise their extraction work in light of any new insights and ideas, thereby introducing an additional level of rigour and ensuring consistency.

| Synthesis
As advocated by Bradbury-Jones et al. ( 2019) for our FMRS, we directed the synthesis on examining the current state of MMR in nursing.In so doing, we sought to consider the completeness of its use in the literature, so the synthesis was limited to exploring the occurrence of the design and the quality of the research.We synthesized our findings across journals (Table 4) and therefore were able to identify and compare the nuances of each journal and extract and report on the patterns within and across journals.

| Overall profile
In total, 34 articles fulfilled the inclusion criteria and were included in the review (Table 5).The articles represented a good global spread, deriving from a total of 18 countries (Table 6).Some of the studies spanned multiple countries and there was a predominance of articles from Australia, the UK, and USA.Of the 34 articles, eight were rated as good, with 11 being of satisfactory standard and most (N = 15) rated as poor.

| Justification and claimed MMR design
A positive finding of our review was that 29 articles stated the specific MMR design that was used in the research, with 17 of these citing an underpinning methodological source that guided their work.
Some of the reasons for the lower quality assessment related to lack of reporting or regard for crucial elements of an MMR design.The most frequently used design was an explanatory sequential MMR design, whereby quantitative methods were followed by qualitative strands (e.g., Newton, Chandler, Morris-Thomson, Sayer, & Burke, 2015).Those that justified this approach, such as Alabdulaziz,

| Underpinning source
Of the articles that explicitly stated an underpinning methodological source, Creswell and Plano Clark and Creswell were the most commonly cited references.Exactly half of the included articles (17 out of 34) failed to mention any underpinning methodological source in the methods section.

| Integration phase
Only eight of the 34 articles gave full details of the integration phase.
Articles identified as good in this review (N = 8) gave very good signposting of where integration occurred, with the best offering a diagrammatic illustration (e.g., Desborough et al., 2018).Eleven articles were assigned to the satisfactory quality range and in most cases, these articles had weaknesses in the reporting of data integration and hence their assignment to the satisfactory range, rather than 'good'.

| Reflection on strengths and limitations of MMR
Most articles gave an account of the limitations of their studies, but this mainly lacked specific reflection on either the benefits or limitations of MMR design.Only four made an explicit statement about the limitations of the MMR approach that they had used, or some aspect of it, such as integration.Articles which commented on the limitations of quantitative and qualitative phases separately were graded 'no' for this criterion.

| What does good-quality reporting of MMR look like?
If researchers are to report on rigorous MMR that advances their field, they need to be able to recognize what 'good' looks like (Venkatesh, Brown, & Bala, 2013) and our FMRS should help with this.In our review, high-quality reporting in articles tended to include the justification for undertaking a MMR approach and also the drawing on theory to underpin the decision.For example, Burden et al. (2018) used pragmatism (Feilzer, 2010)  We have already established the necessity of integration in MMR (Fetters et al., 2013) since MMR is more than simply collecting multiple forms of qualitative and quantitative evidence (Klassen, Creswell, Plano Clark, Smith, & Meissner, 2012).It follows that articles identified as good in this review articulated how integration was accomplished and gave clear signposting of where it occurred in the study design.Well-reported studies, such as Bailie and Thomas (2017), also gave some reflective account on the beneficial use of MMR in their studies.All studies need to show appropriate theoretical depth and breadth of alignment to a recognized MMR design (Creswell, 2009;Creswell & Plano-Clark, 2007) and again, this was evident in good studies, which articulated their approaches to sequential phasing, data triangulation, integration, and synthesis.Of course, this could result merely in an operational approach to replication without advancing the method.In some papers this was taken further, and advanced integration or theoretical application was evident as illustrated in Figure 2.
The Mixed Methods Appraisal Tool (Hong et al., 2018)    an exemplary manner.In terms of the overall interpretation, both papers used tabular data displays as a visual means of drawing out new insights that moved beyond the results of the separate qualitative and quantitative components (Figure 2).In addition, these data displays gave a structure to understand conceptual similarities between quantitative variables and qualitative categories and the way they interacted, converged, or expanded.This approach culminated in an integrated model of patient satisfaction and enablement (Desborough et al., 2018) or an account of different levels of partner participation in care (Näsström et al., 2017).

| Poor quality
It is also important to consider what impedes the quality of MMR reporting so that future researchers avoid similar pitfalls.Our review shows that one of the weaknesses of poorly reported studies was the failure to describe the design in terms of the sequence of methods.For example, some articles appeared to have made this omission because while the authors give detail of their different data collection methods, they do not explain how these methods inform each other.This is contrary to the guidelines for Good Reporting of a Mixed Methods Study (O'Cathain, Murphy, & Nicholl, 2008) that stress the need to articulate the order of a design to show how decisions and inferences are made.
In this cluster of poorly rated articles in terms of their reporting, most claimed adherence to a MMR approach, frequently in the title of the article or in the methods section.However, despite asserting an MMR design, these authors clouded their position by failing to refer to key texts that could have steered the research approach.This was also the case in some of the papers that were rated as satisfactory, the difference being that the satisfactory The majority of the poorly reported articles neglected to chronicle the specific details of a MMR study.For example, one article claimed a convergent, parallel mixed method design but did not align this to the framework of an acknowledged methodological source and gave no indication of how integration was achieved.Due to these limitations in reporting, it is difficult to gain an appreciation of the level of alignment to an accepted MMR design in this group of articles.This does not necessarily mean that they all lack scientific benefit or are of poor quality, but indicates a lack of regard for rigorous reporting and perhaps an inclination by some journals to publish papers that do not satisfy best practice in defining the methodological criteria.
Overall, in our analysis the major weakness of the poor articles was the lack of data integration, especially during analysis and discussion.
This supports the findings of Younas et al. (2019).In most cases, authors reported findings separately for the quantitative and qualitative parts of their studies and failed to integrate the findings at any point.At least eight studies illustrated this shortcoming because although they gave a lot of detail of the separate parts of the study, they neglect to explore the interaction between phases.Therefore, they miss the opportunity for analytical density which could have given the new insights that integration offers.

| Changes over time
Overall, when comparing the quality of MMR studies included in our MMR studies published between 1994-2004-it is evident that quality has improved over the past decade.While lack of transparency of the MMR approach still compromises the quality of MMR studies, more than one third (14 out of 34) of articles in our FMRS showed evidence of at least partial integration of findings derived through qualitative and quantitative research methodology.For example, Kinley, Preston, and Froggatt (2018) maintain that they integrated data, but the detail of the process is confined to telling us that they 'followed a thread' of new concepts from the qualitative data back to the quantitative data.Similarly, Ngangana, Davis, Burns, McGee, and Montgomery (2016) state that the qualitative and quantitative components of their study were mixed in the interpretation stage, but they do not outline the process.
In their review of 294 MMR studies in nursing, published between 1998-2015, Beck and Harrison (2016) found that integration of the qualitative and quantitative components was minimal.Their assertions compare to those of O'Cathain et al. (2008) who stated that: Judgements about integration could rarely be made due to the absence of an attempt at integration of data and findings from different components within a study (p.92) The level of recognition of the integration phase in both of these reviews differs to the (at least) partial integration that was revealed in our findings.Since O'Cathain et al. ( 2008) reviewed Best practice in integration of results papers that extended beyond nursing, one explanation is that the nursing papers were the rare exceptions to their observations, and of higher quality than the other disciplines.However, as this is not borne out by Beck and Harrison (2016), it is more likely that the publication and widespread dissemination of reporting guidelines for MMR studies is the main reason for a positive development over time.This explanation is corroborated in our review where there is a chronological shift in quality, with no good papers published in 2015, five by 2017, and three in the first 5 months of 2018 alone.

| Journal conventions
It is worth raising the point that journal conventions can muddy the waters when considering the quality of MMR papers because of the approach that some journals take when classifying their articles.For example, we observed that JAN organizes its research papers under subheadings, including "ORIGINAL RESEARCH: EMPIRICAL RESEARCH -MIXED METHODS."Three papers that were initially retrieved during our scrutiny of the journal index featured under this heading.However, when we explored these in more detail, it was evident that they did not follow MMR conventions and indeed, the authors made no claim of an MMR study.
At this point we excluded the articles; however, they could easily have slipped through the net.Had this happened, they would have unjustifiably been rated as 'poor' MMR studies when in fact they were reporting on one phase of their MMR study (Lima, Jordan, Kinney, Hamilton, & Newall, 2016), a multi-method study (Stefana, Padovani, Biban, & Lavelli, 2018) and a Delphi study (Perry, Nicholls, Duffield, & Gallagher, 2017).There are for journals here that we refer to below.

| Limitations
There are inherent limitations to our review methodology because it gives a snapshot profile that is constrained by the focused element of the FMRS (Bradbury-Jones et al., 2019).However, in our view, it gives additional and deeper insights into the patterns and problems in MMR and complements earlier reviews on the subject (Younas et al., 2019).
If the review is to be repeated at a different point in time, or with a different set of included articles, the conclusions may well be different.However, our study holds some useful insights and even within the limitations of its focus, we have been able to discern some interesting shifts across time.We debated how to report on the articles that we deemed to exhibit poor reporting of MMR.We felt strongly that we did not want to 'name and shame' and give the details of such articles.After all this review is reporting on one particular analysis which may hold implicit bias (although we made every effort to eliminate this from the review process).Accepting the limitations of not having reported on a significant group of articles contained in the review, we have however highlighted what we consider 'best examples' which is likely to be more helpful to readers, than criticizing those deemed to be of poorer quality.

| CON CLUS ION
The quality of reporting of mixed methods research in the articles included in our review was mixed.Primarily, a poor rating was due to the absence of reporting an underpinning methodological approach to the study and/or limited detail of the theoretical application used to drive the crucial integration phase.This review gives useful guidance on best practice in conducting and reporting mixed methods nursing research and will help to ensure that nurses' endeavour in reporting MMR is of highest quality.The principle quality criteria are twofold: to make sure that the MMR design is well explained and that it includes details of an integration phase.This does not necessarily mean strictly adhering to current reporting convention.We know that MMR represents the complexity of inquiry very well (Flemming, 2007) and diversifying attempts to represent it could help to further advance the design and expand our ways of knowing.
Highly rated MMR articles in our review included a flow chart that showed how the different aspects of the mixed methods design were integrated.This is a simple strategy for MMR to continue to improve quality in reporting.

| Recommendations
Apart from the recommendations for authors to be clear and detailed in their reporting of MMR, we also call on journal editors to re-

ACK N OWLED G EM ENTS
Thanks to Vivian Afoko, PhD Candidate, University of Birmingham, for her contributions to the early stages of the review.

CO N FLI C T O F I NTE R E S T
There is no conflict of interest to declare.

a
Ranking in Scimago Journal & Country Rank for Nursing as of 10 May 2018.undertook an additional keyword search using the journals' search tools.Using the same time frame, we searched titles for the keywords 'mixed method' OR 'mixed-method' OR 'mixed methods' OR 'mixed-methods' (AND nurse OR nursing OR nurs* in the Journal of Mixed Methods Research).In addition, for journals where indexes included the subtitle mixed methods (or derivatives), we searched the abstracts of articles that featured under this subtitle and included them in the review.
gives a clear view of what is expected of high-quality MMR.It comprises five criteria to assess methodological quality in terms of: (a) providing adequate rationale for using a MMR design; (b) effectively integrating the different components to answer the research question; (c) the overall interpretation (meta-inferences) derived from integrating qualitative and quantitative findings; (d) divergences and inconsistencies found when integrating the findings; and (e) adhering to the quality criteria of each tradition.Since they adhered closely to these criteria, the papers by Desborough et al. (2018) and Näsström et al. (2017), both published in the Journal of Advanced Nursing (JAN), epitomized what a good-quality MMR study looks like.Desborough et al. (2018) conducted a concurrent MMR study on developing a positive patient experience with nurses in general practice and Näsström et al. (2017) used a convergent parallel mixed methods design to explore partners' perspectives on participating in home care for patients with heart failure.Although neither team of authors gave an explicit rationale for using an MMR design, they compared and integrated the results of the multilevel analyses in
through their observation of other important MMR elements.Stating the source of the mixed methods design is important because many different MMR designs exist, and researchers often use different terms to refer to these designs.This oversight leads to confusion and prevents methodological replication, which, according toCastro, Kellison, Boyd, and Kopak (2010) is one of the canons of scientific research.While not everyone may agree with Castro's assertion, we agree that clarity in reporting is crucial.

FMRS
with the results of O'Cathain et al. (2008)-who assessed 118 view their practices.Journals should give clear guidance for authors on the reporting of MMR; for example, the framework developed by O'Cathain et al. (2008) is currently recommended by the EQUATOR Network (2013), which aims to enhance the quality and transparency of health research.Moreover, we urge editors to avoid classifying articles as MMR unless claimed by authors.
., C.B.J., M.T.C., N.E., O.H., F.H., L.G., and A.T. made substantial contributions to conception and design, acquisition, analysis, and interpretation of data; F.I., C.B.J., M.T.C., N.E., O.H., F.H., L.G., A.T., and D.S. contributed to drafting the article and revising it critically for important intellectual content; T.H. made the section planning/ design of the study.Additionally, J.T. and T.H. contributed to revising the article critically for important intellectual content.