Public and patient involvement in quantitative health research: A statistical perspective

Abstract Background The majority of studies included in recent reviews of impact for public and patient involvement (PPI) in health research had a qualitative design. PPI in solely quantitative designs is underexplored, particularly its impact on statistical analysis. Statisticians in practice have a long history of working in both consultative (indirect) and collaborative (direct) roles in health research, yet their perspective on PPI in quantitative health research has never been explicitly examined. Objective To explore the potential and challenges of PPI from a statistical perspective at distinct stages of quantitative research, that is sampling, measurement and statistical analysis, distinguishing between indirect and direct PPI. Conclusions Statistical analysis is underpinned by having a representative sample, and a collaborative or direct approach to PPI may help achieve that by supporting access to and increasing participation of under‐represented groups in the population. Acknowledging and valuing the role of lay knowledge of the context in statistical analysis and in deciding what variables to measure may support collective learning and advance scientific understanding, as evidenced by the use of participatory modelling in other disciplines. A recurring issue for quantitative researchers, which reflects quantitative sampling methods, is the selection and required number of PPI contributors, and this requires further methodological development. Direct approaches to PPI in quantitative health research may potentially increase its impact, but the facilitation and partnership skills required may require further training for all stakeholders, including statisticians.

A bibliometric review of the literature reported strong growth in the number of published empirical health research studies with public involvement. 3 In a systematic review of the impact of PPI on health and social care research, Brett et al 4  . More significant tensions have been reported in communityacademic partnerships that use quantitative methods rather than solely qualitative methods, for example tensions with the community about having and recruiting to a "no intervention" comparison group. 5 Particular challenges for PPI have been reported in the most structured and regulated of quantitative designs, that is a randomized controlled trial (RCT), where there is little opportunity for flexibility once the trial has started 6 and Boote et al 3 concluded that researchers may find it easier to involve the public in qualitative rather than quantitative research.
If the full potential of PPI for health research is to be realized, its potential and challenges in quantitative research require more exploration, particularly the features of quantitative research which are different from qualitative research, for example, sampling, measurement and statistical analysis. Statisticians in practice have a long history of working with a variety of stakeholders in health research and have examined the difference between an indirect or consulting role for the statistician and a more direct, collaborative role, 7 yet their perspective has never been explicitly explored in health research with PPI. The objective of this study therefore was to critically reflect on the potential and challenges for PPI at distinct stages of quantitative research from a statistical perspective, distinguishing between direct and indirect approaches to PPI. 2

| SAMPLE S IZE AND S ELEC TI ON
Quantitative research usually aims to provide precise, unbiased estimates of parameters of interest for the entire population which requires a large, randomly selected sample. Brett et al 4 reported a positive impact of PPI on recruitment in studies, but the representativeness of the sample is as important in quantitative research as sample size. Studies have shown that even when accrual targets have been met, the sample may not be fully representative of the population of interest. In cancer clinical trials, for example, those with health insurance and from higher socio-economic backgrounds can be over-represented, while older patients, ethnic minorities and so-called hard-to-reach groups (often with higher cancer mortality rates) are under-represented. 8 This limits the ability to generalize the results of the trials to all those with cancer. There is evidence that a direct approach to PPI with sustained partnerships between community leaders, primary care providers and clinical trial researchers can be effective in increasing awareness and participation of underrepresented groups in cancer clinical trials 9,10 and therefore help to achieve the goal of a population-representative sample.
Collecting representative health data for some groups in the population may only be possible with their involvement.
Marin et al 11 reports on the challenges of identifying an appropriate sampling frame for a health survey of Aboriginal adults in Southern Australia. Access to information identifying Aboriginal dwellings was not publically available, making it difficult to randomly select participants for large population household surveys.
Trying to overcome this challenge involved reaching agreement on the process of research for Aboriginal adults with their local communities. An 8-month consultation process was undertaken with representatives from multiple locations including Aboriginal owned lands in one region; however, it was ultimately agreed that it was culturally inappropriate for the research team to survey this region. The study demonstrated the opportunities for PPI in quantitative research with a representative sample of randomly chosen Aboriginal adults (excluding those resident in one region) ultimately achieved but also the challenges for PPI. The direct approach to involvement in this study, after a lengthy consultation process, resulted in a decision not to carry out the planned sampling and data collection in one region with implications for generalization of results and overall sample size.
Of course, given the importance of representativeness in quantitative research, there may be particular challenges for statisticians and quantitative researchers in accepting the term patient or public representative with some suggesting PPI contributor as a more appropriate term. 6 PPI representative may suggest to a quantitative researcher that an individual patient or member of the public is typical of an often diverse population, yet there is evidence that the opportunities and capacity to be involved as PPI contributors vary by level of education, income, cognitive skills and cultural background. 12 Dudley et al carried out a qualitative study of the impact of PPI in RCTs with patients and researchers from a cohort of RCTs. 6 The types of roles of PPI contributors described by researchers involved in the RCTs were grouped into oversight, managerial and responsive roles. Responsive PPI was described as informal and impromptu with researchers approaching multiple "responsive" PPI contributors as difficulties arose, for example advising on patient information sheets and follow-up of patients. It was reported that contributions from responsive roles may carry more weight with the researchers in RCTs because it allowed access to a more diverse range of contributors who researchers saw as more "representative" of the target population.  16 There are therefore important statistical considerations to be taken into account in the design of longitudinal studies but these have to be balanced with input from PPI contributors on appropriate timing and frequency of data collection for potential participants.

| ME A SUREMENT
Lucas et al reported on how European birth cohorts are engaging and consulting with young birth cohort members. 17 Of the 84 individual cohorts identified, only eight had a mechanism for consulting with parents and three a mechanism for consulting with young people themselves (usually "one off" consultations). Very varied follow-up rates were reported from 13% to 84% more than 10 years after enrolment for individual data rounds of the birth cohorts. 17 Being motivated to continue to participate may be influenced by whether a participant believes the study is interesting, important, or relevant to them. 18 One of the key strategies for retention in the Australian Aboriginal Birth Cohort study was partnerships with community members with local knowledge who were involved in all phases of the follow-up. 19 Retention rates of 86% at 11-year follow-up and 72% at 18-year follow-up were reported which demonstrates the potential of a direct approach to PPI. Ethical approval for the study involved an Aboriginal Ethical Sub-committee which had the power of veto and a staged consent was used where participants had the right to refuse individual procedures at each wave. As with all missing data, this has implications for the statistical analysis yet only 10% of participants in this study chose to opt out of different assessments at follow-up.

| Statistical analysis
A report on the impact of PPI found that it had a positive impact at all stages of qualitative research including data analysis but that there was little evidence of its impact on quantitative data analysis. 20 It was concluded this lack of evidence may reflect a lack of involvement rather than an evidence gap. Booth et al 3 also suggested that the public may be more comfortable with interpreting interview and focus group data compared with numeric data. Low levels of numerical and statistical literacy in the general population may contribute to this.
Statistical analysis involves describing the data using appropriate graphical and numerical summaries (descriptive statistics) and using more advanced statistical methods to draw inferences about the population using the data from a sample (statistical inference). Choosing appropriate methods for statistical inference, testing the underlying assumptions and checking the adequacy of the models produced requires advanced statistical training and implementing them typically involves the use of statistical software or programming. Statisticians bring this expertise to quantitative health research and while it is important that the chosen methods are adequately communicated to all stakeholders, replicating this type of expertise in PPI contributors seems like an inefficient use of resources for PPI.
Quantitative data are, however, "not just numbers, they are numbers with a context" 21 and most practising statisticians agree that knowledge of the context is needed to carry out even a purely technical role effectively. 22 While many associate statistical analysis with objectivity, in practice, statisticians routinely use "subjective" external information to guide, for example the decision on what is a meaningful effect size; whether an outlier is an error in data entry or represents an unusual but meaningful observation; and potential issues with measurement of variables and confounding. 23 Gelman and Hennin argue that we should move beyond the discussion of objectivity and subjectivity in statistics and "replace each of them with broader collections of attributes, with objectivity replaced by transparency, consensus, impartiality and correspondence to observable reality, and subjectivity replaced by awareness of multiple perspectives and context dependence." 23 This debate within statistics is relevant for PPI where the perceived objectivity and standardization of statistical analysis can be used as a reason for lack of involvement.
External information and context are particularly important in statistical modelling where statisticians are often faced with many potential predictors of an outcome. The "best" way of selecting a multivariable model is still unresolved from a statistical perspective, and it is generally agreed that subject matter knowledge, when available, should guide model building. 24 Even when the potential pre- the process was also discussed with a suggestion to include patients "who were able to take a neutral view" and "at least five patients that differ significantly in their background," again highlighting the focus of quantitative researchers on bias and sample size. The role for this type of participatory modelling in informing debate on public health problems is increasingly being recognized, drawing on the experience of its use in other areas where optimal use of limited resources is required to address complex problems in society. 30

| CON CLUS IONS
Statistical analysis of quantitative data is underpinned by having a representative sample, and there is evidence that a direct approach to PPI can help achieve that by supporting access to and increasing participation of under-represented groups in the population. The direct approach has also demonstrated its potential in the retention of those recruited over time, thus reducing bias caused by missing data in longitudinal studies. At all stages of statistical analysis, a statistician continuously refers back to the context of the data collected. 22 Lay knowledge of PPI contributors has an important role in providing this context, and there is evidence from other disciplines of the benefits of including this knowledge in analysis to support collective learning and advance scientific understanding.
The direct approach to PPI where patients and the public have the power to make decisions also brings challenges and the statistician needs to be able to clearly communicate the impact of each decision on the scientific rigour and validity of sampling, measurement and analysis to all stakeholders. Decisions made on participation impact on generalizability. Participatory modelling requires facilitation and partnership skills which may require further training for all stakeholders, including statisticians.
The direct and indirect role for PPI contributors mirrors what happens for statisticians in practice. Statisticians can have a consultative role, that is answering a specific statistical question or a collaborative role where a statistician works with others as equal partners to create new knowledge, with professional organizations for statisticians providing guidance and mentorship on moving from consulting to collaboration to leadership roles. 7,31 Statisticians therefore bring very relevant experience and understanding for PPI contributors on the ladder of participation in health research. Further exploration is required on the impact of direct compared to indirect involvement in quantitative research, drawing on the evidence base for community-based participatory research in quantitative designs 9 and the framework for participatory health research and epidemiology. 32,33

Prof. Anne MacFarlane, Public and Patient Involvement Research
Unit, University of Limerick, for discussion of ideas and comments on drafts.

CO N FLI C T O F I NTE R E S T S
No conflict of interests.