Development and validation in Ecuador of the EPD Questionnaire, a diabetes‐specific patient‐reported experience and outcome measure: A mixed‐methods study

Abstract Introduction The global prevalence of diabetes in 2019 in adults was estimated to be 9.3%. This study developed in Ecuador, for the first time, instruments to assess patient‐reported outcomes and experiences. Methods The Experiences of the Person with Diabetes (EPD) Questionnaire is a diabetes‐specific instrument. A mixed‐methods study was conducted. First, a qualitative item development phase that included four focus groups and six semi‐structured interviews with patients was conducted in different rural and urban areas of Ecuador to obtain information on culture, beliefs, demographics, diet and social perspectives. A second quantitative phase for psychometric validation was carried out in primary care settings of rural and urban areas of Ecuador. Results Forty‐two and four hundred and eighty‐nine participants were included in each phase, respectively. The item development phase resulted in a questionnaire of 44 items (23 for perceived outcomes and 21 for experiences). In the validation study, most participants were women (58%) and from urban areas (57%). Exploratory factor analysis revealed three dimensions for each instrument. Outcomes instrument dimensions were symptoms and burnout, worries and fears and social limitations. Experiences instrument dimensions were information, patient‐centred care and care delivery. Cronbach's α values of the total score and dimensions were high, ranging between .81 and .93 in both instruments. Confirmatory factor analysis showed an acceptable fit of the data. Conclusion The EPD Questionnaire is probably the first instrument developed to assess patient‐reported experiences and perceived outcomes in a middle‐income country that included patients to capture all dimensions relevant for the intended population. Its psychometric properties are robust and could provide valuable information for clinicians and policymakers in the region. Patient or Public Contribution The development of these instruments has taken into consideration patients and the public since their conception. A qualitative approach gathered relevant information related to the cultural, social and economic burden of different populations in Ecuador. Before validation, a pilot test was carried out with users of the National Health Services to obtain their perspectives and insights of the developed instrument. Finally, during the data analysis, we have given special consideration to social variables such as rural and urban populations.


| INTRODUCTION
Chronic noncommunicable diseases are a challenge for all health systems. The global prevalence of diabetes in 2019 in adults  years) was estimated to be 9.3% (463 million people), expected to increase to 10.2% (578 million) by 2030. 1 However, more alarmingly, the prevalence of diabetes has increased more rapidly in low-and middle-income countries 2 with health systems where much remains to be done. It is estimated that by 2030, in America, 83 million people may have diabetes, an increase of 50% since 2000. 3 In Ecuador, the prevalence of diabetes is 7.3%. 4,5 In Ecuador, diabetes is the second most common cause of death, only after ischaemic heart disease, 6 which is also a well-established risk factor. Driven by obesity (6 out of 10 Ecuadorians are overweight and obese), unhealthy lifestyles (60% have a sedentary lifestyle) and increased life expectancy (74 years for men and 80 years for women), diabetes carries a high burden of disease, derived from its prevalence, complications and the associated comorbidity. 4,7 Ecuador's Health System is struggling to achieve all its objectives. Despite the significant increase in the number of medical consultations, this effort has not been reflected in a real impact in terms of health improvements for the population. 8 One of the strategies available in primary care is peer-related support groups such as the Club of Patients with Chronic diseases, led by a healthcare worker (medical doctor or nurse). These Clubs of Patients have been expanded in the country, aimed at providing education and counselling to patients with chronic diseases, especially diabetes and hypertension. 9 The prevention and control of diabetes represent a challenge for professionals, healthcare services and social systems and, above all, for the patients themselves, since diabetes significantly affects their quality of life, requiring a coordinated effort from different levels of care to alleviate its effects. In addition, diabetes has a high economic and social impact. 10 The promotion of a healthy lifestyle, control of risk factors (diet, overweight, physical exercise), diabetes education and patient self-care are essential elements in preventing the progress of the disease and the social and health overload that it represents. 11 For a better intervention in terms of health-disease processes, the patient with a chronic disease has to be an active protagonist. 12,13 Healthcare systems aiming to achieve person-centred coordinated care should systematically assess patients' perspectives. 14 A recent scoping review of patient-reported measures could not retrieve any in situ developed diabetes-specific tool in low-and middleincome countries. 15 All of them were developed in high-income countries with strong economies and robust healthcare systems. It is essential to incorporate the patient's voice in a healthcare organisation to obtain feedback and improve patient participation. 16 In low-and middle-income countries, out-of-pocket expenditure accounts for 48% and 31% of healthcare financing, respectively. 17 Specifically in Ecuador, out-of-pocket payments account for 40% of healthcare expenditure, 18 combined with difficult access to health services, especially in more distant communities; the economic burden is a common issue in the local context. 19,20 Although there is local production of first-hand supplies, the social and economic difficulties, especially among vulnerable populations, make them more likely to consume high-carbohydrate diets due to their lower cost.
Other cultural and educational factors result in people holding diverse beliefs about the pathogenesis of the disease, which complicates health education and management. Therefore, there might be a current need to develop diabetes-specific instruments in low-and middle-income countries, where social, cultural and economic contexts and access to healthcare are different.
The Triple Aim is to improve individual experiences, the health of populations and reduce per capita costs of care of populations. 21 Healthcare systems aiming to achieve person-centred coordinated care should be organized to support value-based healthcare centred on what matters most to patients. The incorporation of instruments known as patient-reported experience measures (PREMs) has made possible to make the patient visible within the context of health systems and include his or her experience in managing chronic pathologies. One of the most widely used relational PREMs is the Picker Patient Experience Questionnaire. 22

| MATERIALS AND METHODS
The development of the 'Experiencias de la Persona con Diabetes' (EPD) Questionnaire, which included an independent PREM and PROM instrument, followed a mixed-methods study. The protocol of this study considered the three phases used to create a scale described by Boateng et al. 31 (item development, scale development and scale evaluation), the standards and guidelines for validation practices summarized by Chan 32 and the COSMIN recommendations. 33 Ethics approval was obtained for both item development and validation phases from a local ethics committee (HCK-CEISH-19-0041) ( Figure 1).

| Item development
First, a scoping review of existing specific type 1 and 2 diabetes patient-reported outcome and experience measures was considered. 15 In a previous study, dimensions assessed by other measures, possible items and existing gaps were identified.
Second, four focus groups and six semi-structured interviews involving patients and professionals were conducted in Ecuador between May and August 2019. To obtain information on the culture, beliefs, demographics, diet, type of treatment and degree of engagement, participants from the highlands, coastal and urban or rural areas were included. Participants were recruited using a snowball approach and reached through phone calls. Direct conversations were held with the medical staff or the Club of Patients with Diabetes (a peer-related support group) of the healthcare centre. Eligibility criteria were adults over the age of 18 years with type 2 diabetes. Once they had been invited, participants were explained the aim of the focus groups and they decided whether to participate voluntarily. Informed consents were signed before the focus groups were conducted, and permission for audio recording was obtained. They were also informed that they had the right to leave the focus group at any time without providing a reason. None of the participants declined the invitation to be part of focus groups or interviews. Data collection was continued until data saturation.
The interview guide content was developed through the mentioned scoping review on diabetes 15 and was agreed by the team in consultation with professionals based in Ecuador (File S1). Experts in the qualitative research methodology provided input on the clinical approach and guidance. The focus groups were conducted in private rooms in the same health centres where participants used to receive medical attention to provide a familiar environment to generate trust.
None of the healthcare workers were part of the research team or were involved during data acquisition. Participants' anonymity and data confidentiality were always respected. The focus groups began with a brief description of the aim of this study and a brief introduction of each participant, after informed consent was obtained.

| Item scoring
The EPD Questionnaire included two instruments. The recall period was last month, and responses were rated on a four-point Likert scale

| Content and face validity
To determine the ease with which patients answered the EPD Questionnaire, face validity, content and feasibility were assessed. A pilot test with type 2 diabetes patients was performed to evaluate if any other theme was missing. Also, wording, completion time, importance, difficulty of responding to each item and if items addressed their experience and outcomes adequately were included. A 5-point Likert scale was used (0 = it is not understood to 5 = completely understood) and if items addressed their experience and outcomes adequately. Finally, they were asked if the response scale was appropriate for the instrument. Ten Participants were reached by their family doctor and had to be of legal age, diagnosed at least 5 years ago with type 2 diabetes and regular users of health services (patients for at least 1 year).

| Reliability and validity study
A paper-based validation study was conducted during January and March 2020 to collect data to assess the measurement and psychometric properties of the EPD Questionnaire. The study was conducted in the primary care settings of rural and urban areas of Ecuador. To be eligible for the study, participants had to be of legal age, been diagnosed with type 2 diabetes and be able to understand the questions. The sample size was established using the formula for finite universes considering the last available prevalence data of diabetes in Ecuador. 5 A 95% confidence level, 5% accuracy (p = q = 50) and 15% of lost data were established, and the criteria to involve at least 10 respondents per item 31 were applied. The selection process of the respondents was randomly performed by healthcare professionals working in each of the participating health centres.

| Item reduction
Exploratory principal component factor analysis (EFA) was used to evaluate the scales' latent content structure. In this process, the varimax rotation was used, as it usually produces explicit results that can facilitate the interpretation. Furthermore, items with crossloadings and factor loadings of less than 0.50 were dismissed using IBM SPSS Statistics for Windows, Version 25.0. Items were also reviewed by the research team composed of medical doctors, psychologists and experts in mixed-methods research and patients. 35

| Reliability
Internal consistency was measured using Cronbach's α and McDonald's ω. A minimum correlation of .70 was expected to establish that the instrument is internally consistent and has acceptable reliability.
The split-halves method was used as an alternative to test-retest.
Split-halves provide different results; we have considered Guttman's λ 4 as it provides the most significant split-half reliability for measures with 16 items or less. 36 Items were randomly and evenly distributed using the R package splitHalf. Lavaan was used. 37 We calculated the Pearson correlation coefficients between all items of the three factors of the PREM and PROM instruments and between each dimension and total score. This enabled us to evaluate the convergent and discriminant validity based MARTIN-DELGADO ET AL. | 2137 on the hypothesis that the correlations between each item would be stronger than those between factors and the total score.

| Structural validity
Additionally, the hypothesis that scores on the scale would be higher in urban areas compared to rural areas was tested. This is because in urban areas, patients have higher education levels and improved access to health. The Mann-Whitney U test was used.

| Responsiveness
To assess responsiveness, in both instruments, we evaluated how patients' responses changed according to years from diagnosis, hypothesising that scores in the experiences measure would be better in patients with a longer history of diabetes. Also, scores in the perceived outcomes measure, especially the dimension of fears and social limitations, could worsen over time. The Kruskal-Wallis test was used.

| Minimal clinically important change
The minimal clinically important change (MCIC) was estimated using three distribution-based approaches: 1 standard error of measurement, 0.5 standardized effect size and 0.5 responsiveness statistic. 38,39 The values enable estimation of the amount of change for each of the scales to be considered important.

| Item development
A total of 28 participants of the focus groups were reached through their medical doctors who were directly involved with their care.
Afterwards, 36 phone calls were made, and 14 more patients accepted the invitation (38%) to participate in the focus groups. Data saturation was achieved with the third focus group and six interviews, but the fourth focus group was conducted to comply with the methods of including a focus group from a rural coastal area. Detailed characteristics of the participants are presented in Table 1.
Among the most prevalent symptoms, thirst was described as a persistent cause of discomfort, along with fatigue.
Even if I drank water, I was thirsty. I was even more thirsty at night and dawn and used the bathroom a lot.
A proportion of the participants accepted their pathology, but not the treatment. This is motivated by a high prevalence of alternative treatments, lack of information, the low level of health literacy and 'fear' of insulin therapy. My fear was to go blind because you see people who go blind (…) being disabled is worrying.
What worries me most is getting to dialysis, but there are times when I forget about it.
By means of the scoping review 15  Participants were also given the opportunity to choose between two different 4-point Likert scale responses; 9 out of 10 picked the following scale (hardly ever to everyday). patients were hospitalized in the last year due to diabetes, with a mean of 9.5 (8-11) days of hospital stay. Table 2 shows the detailed characteristics of the participants.

| Item reduction
The EFA identified five factors for the PROM and three factors for the PREM, explaining 70% and 62% of the variance, respectively.
Items that did not show factor loadings of 0.50 or higher were removed. This process was repeated on two occasions, and 19 items were excluded. Furthermore, the remaining items were assessed for content redundancy. Following this approach, four items with similar content and connotation were discarded. In each case, the item with higher factor loading was retained. One item related to insulin intake was removed due to a low response rate.

| Reliability
Internal consistency coefficients measured by the Cronbach's α of the EPD Questionnaire (total score and dimensions) were high, ranging between .81 and .93 (

| Construct validity
To test the observed factor structure, CFA was used (

| Structural validity
Overall, the PREM and PROM results were better in urban rather than rural areas (Tables 7 and 8). This might be due to overall access to health and information. Persons in urban areas had better results related to patient-centred care and care planning (p < .05). This instrument has been found to have adequate validity and reliability and should provide accurate and targeted information for local policymakers and the region.

| Responsiveness
T A B L E 3 Rotated factor loadings of the patient-reported experience measure  It is fundamental to achieve person-centred care and to do, so individuals need to be part of the healthcare system and process.
Patient-centred care and cocreation of care were associated positively with satisfaction with care and the physical and social wellbeing of patients with multimorbidity in the primary care setting. 41 The development of this instrument has included the participation of patients since its beginning. A qualitative study was performed in different social, cultural and geographical areas of Ecuador. This approach had the objective of capturing patients' beliefs, values, way of living and how they experience 'living with diabetes,' as these could influence results. 42,43 The results of the validation study suggest that the EPD Questionnaire is psychometrically sound. The PREM ex-  This study reflects the convenience of combining the results of these instruments that include the patients' perspectives with other sources of objective evaluations of health outcomes such as glycosylated haemoglobin or population-based indicators related to chronic diseases. However, special attention must be paid to address the needs of rural populations or those who live far from the communities where health centres are located. 49 These instruments could also be used in other Latin American countries that share with Ecuador similar characteristics described previously. For an instance, coastal and andean population, similar customs and beliefs, and a health system where many remains to be done. At least, it can be assumed that this instrument is closer to the reality experienced in these countries than others originally designed and validated in developed countries, with more robust health systems, different beliefs and customs and primarily English-speaking populations.
Furthermore, using these types of instruments in health policy decision-making and incorporating patient perspectives, health planning decisions are legitimized, attention is brought closer to the needs identified by patients and improvements can be made to meet these needs. 50 Moreover, it is expected that health outcomes improve when the patient becomes more involved and participates more actively in his or her self-care. 51 At a time when, as a result of the COVID-19 pandemic, many of the consultations with patients have been interrupted, empowered patients have become more capable of taking care of their health in the absence of traditional consultation resources. The national health system in Ecuador has tried to adopt telemedicine, but only 44.5% of users reported a positive experience during 2020. 52 This situation has drawn attention to the downsides of patients being dependent on healthcare providers and not actively assuming a self-care role. 53 In the current process of reviewing the diabetes care programme, it is an opportunity to include, to some extent, the voice of the patients with an in situ developed instrument to systematically measure and tackle some of the current challenges (access to healthcare, urban and rural disparities, economic burden, health beliefs and patient education and empowerment).
We encountered some limitations during our study. The use of HbA1c to compare scores and assess predictive validity was not possible due to intrinsic difficulties such as access to health or lack of standardized laboratory procedures. After the national lockdown was decreed on 17th March due to the COVID-19 pandemic, the study had to be stopped, and test-retest assessment could not be performed. Another cross-culturally validated diabetes-specific instrument in the local context is not available, which did not allow comparison of our results with another measure. Additionally, we are aware of the limitations of a two-item factor, but this was retained to have a short instrument. The association of PROMs and PREMS with HbA1C is likely to be complex. Future research should include standardized HbA1c testing and validation of another measure for anchor-based MCIC and comparison with other scales.

| CONCLUSION
In summary, we believe that the EPD Questionnaire is the first PREM and PROM instrument developed in a middle-income country that included a mixed-methods study to capture all dimensions relevant for the intended population. Its psychometric properties are robust and could provide valuable information for clinicians, policymakers and middle management in Ecuador and other countries of South America. Further research is needed to assess predictive validity with a more objective outcome such as HbA1c.
participated in data collection in each of the participating health