Quality of life in coeliac disease: item reduction, scale development and psychometric evaluation of the Coeliac Disease Assessment Questionnaire (CDAQ)

Summary Background A better understanding of coeliac disease can be achieved by assessing health‐related quality of life alongside clinical factors. Existing patient‐reported outcome measures (PROMs) evaluating quality of life in coeliac disease have not been developed in accordance with the US Food and Drug Administration guidelines. Aim To develop a PROM in accordance with best practice guidelines, capturing all aspects of quality of life important to adults with coeliac disease. Methods Candidate items for the Coeliac Disease Assessment Questionnaire (CDAQ) were refined through item appraisal, expert review, cognitive interviews, and a translatability assessment. A cross‐sectional survey determined further item reduction and the CDAQ's structure. The final CDAQ was administered alongside the Short Form Health Survey Version 2 (SF?36v2) in a second survey to assess construct validity and test‐retest reliability. Results Pre‐testing the 64 candidate items revealed a range of issues which guided their refinement and reduction, resulting in the final CDAQ with 32 items representing 5 subscales: stigma (eight items), dietary burden (eight items), symptoms (five items), social isolation (five items), and worries and concerns (six items). Cronbach's alpha ranged between 0.82 and 0.88 for all domains. Further results showed CDAQ scores were more strongly correlated with the SF‐36v2's mental health dimensions, as expected. Intraclass correlation coefficients ranged from 0.79 to 0.89. Conclusion The CDAQ is a reliable and valid coeliac‐specific measure that captures all aspects of quality of life important to adults with coeliac disease. Further work is underway to assess the CDAQ's responsiveness to change.


| INTRODUCTION
Coeliac disease is a chronic autoimmune condition affecting approximately 1% of the population. 1 The immune response is triggered by the consumption of gluten, a protein found in wheat, barley and rye.
The only treatment currently available is a gluten-free diet, which is known to be burdensome, restrictive and challenging in terms of adherence. [2][3][4] Various aspects of daily life can be affected by following a gluten-free diet, including travelling, shopping and eating meals outside of the home. 5 Patient-reported outcome measures (PROMs) present a unique opportunity to systematically gain insight into patients' views, which may not overlap with clinical outcomes or biomedical markers. 6 A broader understanding of the impact of coeliac disease may help to direct care and improve clinical outcomes. 6 PROMs can also be used as endpoints in clinical trials, which are currently underway to develop and test pharmacological treatment alternatives to a glutenfree diet. 7 It is likely that the treatments under development will be supplementary to, rather than a substitute for, the diet. 8 PROMs that are to be used in clinical trials to support labelling claims should be developed by following the guidance of the US Food and Drug Administration (FDA). 9 In any case, this guidance is considered best practice for the development of PROMs regardless of their intended use. 10 The initial steps of development should include qualitative interviews or focus groups with people with the relevant disease to generate candidate items, which then undergo cognitive testing. Once the items have been determined, psychometric properties need to be assessed to evaluate the PROM's quality.
Specifically, PROMs must be reliable, valid and responsive to change. 11 Patient-reported outcomes in coeliac disease have predominantly been assessed using generic measures, such as the Short Form Health Survey (SF-36). 12 However, generic measures are less specific and can be less sensitive than disease-specific measures. Some coeliac-specific measures have been developed, for example, the Coeliac Disease Questionnaire (CDQ) 13 and the Coeliac Disease Quality of Life Survey (CD-QOL). 14 A systematic review identified four candidate coeliac-specific PROMs for use in clinical trials and concluded that none of these meet the standards of the US Food and Drug Administration. 15 Another systematic review 16 focused on patientreported symptom scores and identified two coeliac disease indices that have been approved by the US Food and Drug Administration.
Symptom indices are recommended as end points in clinical trials for new treatments for coeliac disease. 17 However, they are narrow in their focus and are unlikely to capture all aspects of quality of life that are important to people with coeliac disease. As such, there is a need for PROMs that include other health-related quality of life domains to capture the broader burden of disease.
Due to the limitations of existing coeliac disease-specific measures, our aim was to develop a new PROM, capturing all aspects of coeliac-specific health-related quality of life, for adults with coeliac disease using best current practice in instrument development.

| METHODS
The development of the Coeliac Disease Assessment Questionnaire (CDAQ) was undertaken in four stages ( Figure 1). In stage 1, qualitative interviews with 23 adults with coeliac disease informed the development of candidate items and is reported elsewhere. 18 In stage 2, candidate items were refined following item appraisal, expert review of the items, cognitive interviews and a translatability assessment. The items were amended as necessary after each step of pre-testing. In stage 3, data collected from a cross-sectional survey were used to reduce the number of items and identify the CDAQ's dimensions. In the final stage (stage 4), the reliability and validity of the CDAQ was assessed using data from a further cross-sectional survey. Stages 19 which aids the systematic assessment of questionnaires to identify and resolve common problems (eg, poor question clarity) at an early stage of development. Second, feedback on the retained items was obtained from experts in meetings and interviews in June 2013.

| Cognitive interviews
Cognitive interviews were conducted with people with coeliac disease in August 2013 to identify sources of response error within the questionnaire, such as incomprehensible questions. 20 Detailed notes of any identified problems and suggestions for improving the items were documented and guided item revision. All participants gave written consent.

| Translatability assessment
A translatability assessment was conducted to identify and address any cultural or linguistic translatability issues. The assessment was conducted in collaboration with PharmaQuest Ltd (now Corporate Translations, Inc.), a company specialising in the translation and linguistic validation of PROMs. Translators commented on the translatability of the CDAQ's instructions, items and response options, with comments used to further refine the CDAQ. The survey questionnaire included 51 CDAQ candidate items along with demographic and disease-related questions (eg, time since diagnosis, adherence to the gluten-free diet). CDAQ items ask about the past 4 weeks, with all items scored from 1 ("Never") to 5 ("Always"). No data imputation was undertaken.

| Analysis
Data were analysed using IBM SPSS Statistics 20. The aim of the analysis was to determine the structure of the CDAQ and, where necessary, to reduce the number of items.
Prior to principal components analysis, candidate items were considered for removal if more than 5% of data were missing on an individual item; if there was a floor or ceiling effect >50% (ie, at least 50% of respondents selected "never" or "always"); or if interitem correlations <0.2 or >0.8 (ie, items were measuring different constructs 21 or almost the same thing 22  A higher order factor analysis was conducted to determine the appropriateness of combining dimension scores to create an overall score.

| Stage 4-Assessing the reliability and validity of the CDAQ
Following the reduction in items, a second cross-sectional postal survey (Survey 2) was conducted with adult members of Coeliac UK.
Eight hundred members of Coeliac UK were invited to participate.
The same eligibility criteria and sampling strategy as the previous survey (Stage 3) were adopted. The survey questionnaire included the CDAQ, the Short Form Health Survey Version 2 (SF-36v2), 12 and demographic and disease-related questions.

| Test-retest reliability
To evaluate test-retest reliability, consenting respondents completed a second questionnaire, which included the CDAQ and the following question ("Compared with 2 weeks ago, how would you rate the impact of your coeliac disease on you and your health now?"), rated on a five-point response scale from "much better" to "much worse".
Respondents reporting that their health was unchanged were included in the analysis. A test-retest interval of 2 weeks was selected as it is generally considered short enough for no changes to have occurred, but long enough to minimise the risk of respondents recalling their previous answers.

| SF-36v2
The SF-36v2 is a 36-item generic measure of health-related quality of life addressing the eight domains: Physical Functioning, Role-Physical, Bodily Pain, General Health, Vitality, Social Functioning, Role-Emotional and Mental Health. In addition, two summary scores can be calculated, the Physical Component Summary and Mental Component Summary. The SF-36v2 was selected as it is considered to be the leading generic measure, 27 it has been used in previous studies in coeliac disease, for example, [28][29][30] and construct validity is commonly assessed against a generic measure. 31

| Analysis
Internal consistency reliability, test-retest reliability, and construct validity were evaluated. The internal consistency of each CDAQ dimension was assessed using Cronbach's alpha, with acceptable values ranging between 0.70 and 0.95. 26 Test-retest reliability was evaluated using the intraclass correlation coefficient (ICC), with values of 0.70 or above considered acceptable. 26,32 In terms of construct validity, convergent and divergent validity were assessed by comparing dimensions of the CDAQ and the SF- Based on the literature (eg, 33,34 ), it was hypothesised that the CDAQ overall index score would vary by gender, with women expected to report lower scores than men. An independent samples t test was used to test this hypothesis. In addition, it was expected that the CDAQ overall index score would discriminate between groups based on self-reported impact of coeliac disease (scored from 1 "no impact" to 5 "very severe impact"), with higher CDAQ scores for those reporting lower impact. A one-way ANOVA with Tukey-Kramer post-hoc test was used to determine whether the differences between severity groups were significant.

| RESULTS
Candidate items for the CDAQ were refined and pre-tested, prior to a survey being conducted for further item reduction and identifica-  items were amended, two items were added to address unclear constructs more specifically, and 16 items were deleted as they were conceptually too similar to other items.

| Expert review
Nine experts (including clinicians, researchers and Coeliac UK employees) participated in a meeting or interview. Suggestions to improve the candidate items included re-wording, broadening content (eg, acknowledging the presence of gluten in drinks as well as food), and adding statements to clarify that items should be answered in relation to coeliac disease. Experts also commented on the cultural translatability of some items (eg, "eating out" may not occur in all cultures). Overall, 13 items were amended on the basis of recommendations made by the experts.

| Cognitive interviews
Ten people (three men and seven women) with coeliac disease took part in a cognitive interview. Two rounds of interviews were conducted, following which it was deemed that no major issues remained. Participants were aged between 24 and 80 years, the majority of which were White British (n = 9), and had been diagnosed more than 6 years ago (n = 8), and half were married (n = 5) and in full-time employment (n = 5). Three types of problems were identified: (a) participants answered some questions in general rather than specifically about their coeliac disease; (b) participants interpreted specific words and phrases within the context of items in different ways to each other and (c) participants interpreted items in a different way than intended. Amendments were made to 11 items.
In general, participants felt that the CDAQ comprehensively covered all areas of health-related quality of life in relation to coeliac disease. While participants accepted the 4-week recall period, many commented that certain important and potentially problematic experiences, such as having medical tests or going on holiday, were unlikely to have occurred within this time frame.

| Translatability assessment
Sixty-four potential translatability issues were identified across 40 items (out of 51), which were broadly categorised as "cross-cultural" (n = 27) or "grammatical" (n = 37). The majority of cross-cultural issues identified arose due to a lack of equivalent vocabulary in the target languages (eg, words such as "condition") (n = 22), or translations of phrases (eg, "eating out") where no conceptually equivalent phrases exist in all the target languages (n = 4). One item was potentially difficult to translate due to sociocultural differences, with the concept of following a gluten-free diet out of personal choice not understood in all cultures. The majority of grammatical issues identified were as a result of inconsistent tenses across items (n = 28), structural errors (eg, missing verbs) (n = 7) and the wording of items not accurately or adequately expressing the underlying concept (n = 2). Sixty-nine amendments were made to 43 items to address these issues and improve the readability of items.  ("worried family member could develop coeliac disease") was the only item with a "not applicable" response option. As this response option was rarely endorsed by respondents (1%, n = 4), the 'not applicable' response option was removed. Bartlett After removing these eight items, a further principal components analysis was performed on the remaining 32 items (

| CDAQ overall score
A higher order factor analysis of the five CDAQ dimensions identified one factor with an eigenvalue >1, explaining 68.0% of the variance, indicating that it is appropriate to combine the

| Stage 4-Assessing the reliability and validity of the CDAQ
Survey 2 achieved a 34.5% (n = 276) response rate. Eight questionnaires were excluded as the respondents did not report receiving their diagnosis from a doctor, thus leaving 268 respondents in the analysis. The majority of respondents were female (n = 166, 61.9%), married (n = 159, 59.3%), and White British (n = 225, 84.0%). The mean age of respondents was 49.5 years (SD 18.9) and the mean duration since diagnosis was 7.49 years (SD 9.67). CDAQ and SF-36v2 scores are given in Table 3. Missing data for CDAQ items were very low, with the maximum amount of missing data for any one item being 1.1% (n = 3), which meets the PROMs quality criteria set out by Terwee et al. 26 and indicates that the CDAQ is acceptable to adults with coeliac disease.

| Internal consistency reliability
Internal consistency was assessed for each CDAQ subscale. Cronbach's alpha values were: stigma (0.87), dietary burden (0.87), symptoms (0.86), social isolation (0.86) and worries and concerns (0.82), indicating good internal consistency reliability. indicating that the CDAQ scores are stable over time when participants report no changes (see Supporting Information for CDAQ scores).

| Convergent and divergent validity
Correlations between CDAQ dimensions and the SF-36v2 are shown in Table 4. All correlations were in the expected direction (ie, positive). As expected, the CDAQ overall index score and subscale
The CDAQ overall index scores decreased from the "no impact"

| DISCUSSION
The aim of this research was to refine and reduce the number of candidate items, determine subscales, and assess the reliability and validity of the CDAQ. The final CDAQ has 32 items across five subscales: stigma (eight items), dietary burden (eight items), symptoms (five items), social isolation (five items) and worries and concerns (six items) and captures all aspects of quality of life of importance to adults with coeliac disease as identified in the qualitative phase. 18 The CDAQ has been shown to be a reliable and valid measure.
The methods used to develop the CDAQ have been shown to be effective in the development of similar questionnaires in the past 35,36 and are compliant with best practice guidance on the development of PROMs, such as guidance provided by the US Food and Drug Administration 9 and the International Society for Pharmacoeconomics and Outcomes Research (ISPOR). 37 Involving potential respondents in the refinement of questionnaire items through cognitive interviews enhanced content validity, which is considered one of the most important measurement properties of PROMs. 26  The final CDAQ subscales represent a modification to the conceptual framework developed from the qualitative interviews (encompassing six main themes: emotional health, gluten-free diet, relationships, impact on activities, symptoms and financial issues). 18 However, the themes from the qualitative interviews remain represented, albeit in a potentially less obvious way. For example, financial issues identified as a qualitative theme did not become a subscale in their own right, but an item on the cost of gluten-free food fit with the subscale of 'dietary burden'. The impact of coeliac disease on travel and holidays was also commonly reported in the qualitative interviews, but was difficult to include as an item because the majority of respondents are unlikely to have been on holiday in the 4 week time frame covered by the CDAQ. The concept is still covered indirectly in an item on difficulties experienced with finding suitable food away from home.
Following a gluten-free diet is known to be burdensome 2 and the results of this study found dietary burden to have the greatest impact on health-related quality of life. Therefore, it is essential that items addressing this burden are included in coeliac-specific PROMs that aim to comprehensively assess health-related quality of life.
While items assessing dietary burden will be appropriate to the majority of people with coeliac disease, they may be less relevant to those who are newly diagnosed, but yet to be treated (ie, not yet following a gluten-free diet). Further research is required to assess the reliability and validity of the CDAQ within this patient group.
This study also provides evidence that the CDAQ is a reliable and valid measure for assessing health-related quality of life in adults with coeliac disease. Across both surveys, the internal consistency of all five subscales was within the ideal range (0.7-0.95). 26 Other coeliac-specific measures have also been found to be internally consistent such as the CDQ 13,41-44 and the CD-QOL. 14 45 and at least comparable with the CDQ, for which some evidence of sufficient test-retest reliability was found. 13,[41][42][43] However, the quality of some of the studies for both measures was poor with small sample sizes or insufficient detail about study design.
Overall, correlations between the CDAQ and SF-36v2 were as expected, with dimensions of the CDAQ correlating more strongly with mental health dimensions of the SF-36v2 and the Mental Component Summary score.
A possible limitation is that all survey respondents are members of Coeliac UK, although there is no evidence to suggest that healthrelated quality of life in this population differs from that of the wider population of adults with coeliac disease. The cognitive interviews included participants who were not recruited through their membership of Coeliac UK. In addition, Survey 1 achieved a diverse sample (eg, 12.1% belonged to black and minority ethnic groups), which increases the likelihood that the final CDAQ will be of relevance to adults from all demographic groups. The response rate to Survey 2 (34.5%) was lower than that achieved in Survey 1 (52.0%), and is likely to be accounted for by its much greater length (as survey 2 also collected data on experiences of health care which will be reported elsewhere). Despite this, the response rate was similar to other Coeliac UK surveys. 46 Furthermore, participants were recruited on the basis of self-reporting their diagnosis and while they considered themselves to have been given a diagnosis of coeliac disease, it is possible that they may not have coeliac disease.
With clinical trials to develop treatments other than the gluten-free diet underway, it is important that valid and well-developed PROMs are available as potential endpoints for trials. 7 In addition to symptoms and histological improvement, health-related quality of life is regarded as a key outcome in assessing new therapeutic treatments and should be considered as a critical end point in relevant clinical trials. 17 The results of this study further highlight the need to include measures of health-related quality of life in clinical trials of treatments for coeliac disease, as the greatest impact on health-related quality of life was the burden of managing a gluten-free diet. As it is likely that the diet will remain relevant in combination with any treatments under development, 8 it is important to have available PROMs that assess coeliac-specific quality of life more broadly than symptoms alone. Furthermore, the CDAQ is suitable for use in clinical practice and research more broadly. Further research is underway to evaluate the CDAQ's responsiveness to change.