Quality-assessment of disease-specific quality of life questionnaires for rhinitis and rhinosinusitis: a systematic review

Authors


C. M. van Oene
Academic Medical Center – ENT
Postbus 22660
Amsterdam 1100 DD
the Netherlands

Abstract

The aim of this systematic review was to give a quality-assessed review of the existing disease-specific health related quality of life (QOL) questionnaires concerning rhinitis and rhinosinusitis for adults. The quality is assessed by defining the characteristics of a QOL questionnaire with assessment criteria. The results of the construction, description, feasibility, and the psychometric performance of the instruments are provided. We finally provide a clinician’s guide to choose a questionnaire based on the measurement goals, the discriminant validity, responsiveness and the points obtained in the quality assessment. Of the top scoring instruments regarding the overall quality assessment, only four health related QOL questionnaires for rhinitis and rhinosinusitis met our criteria for the discriminant validity and responsiveness.

Abbreviations:
AR

allergic rhinitis

CRS

chronic rhinosinusitis

HRQL

health related quality of life

M(C)ID

minimal (clinical) important difference

NA

not applicable

NR

not reported

Nr

number of patients

P

points

PAR

perennial allergic rhinitis

QOL

quality of life

SAR

seasonal allergic rhinitis

SF-36

Short form 36

SRM

standardized response mean

Rhinitis and rhinosinusitis encompass several diseases with a high incidence rate and chronic character (1–3). Rhinitis is a heterogeneous disorder, defined as inflammation of the lining of the nose, characterized by one or more of the following symptoms: nasal congestion, nasal discharge, sneezing and itching (4). Rhinosinusitis (including nasal polyps) is defined as inflammation of the nose and the paranasal sinuses characterized by two or more symptoms, one of which should be either nasal blockage/obstruction/congestion or nasal discharge (anterior/posterior nasal drip), ±facial pain/pressure, ±reduction or loss of smell; and either endoscopic signs of polyps and/or mucopurulent discharge primarily from middle meatus and/or; oedema/mucosal obstruction primarily in middle meatus and/or CT changes showing mucosal changes within the ostiomeatal complex and/or sinuses (5). In the USA, the term sinusitis is still widely used, for example by the FDA. In Europe, the term rhinosinusitis is used as sinusitis rarely occurs without rhinitis (6). Although the mentioned symptoms of rhinitis and rhinosinusitis are not life threatening, multiple studies indicated that they are associated with a dramatic reduction in quality of life (QOL) (7, 8). For example, Bousquet reported a significant impairment in eight of nine QOL dimensions of the Short form 36 (SF-36) in patients with perennial rhinitis compared with healthy subjects (9). For rhinosinusitis, van Agthoven reported SF-36 QOL scores all well below population norm scores (10). Wee et al. reported a (significantly) worse QOL relative to patients with diabetes mellitus, rheumatism, myocardial infarction or migraine, also assessed with the SF-36 (11). These findings indicate the importance of QOL in rhinitis and rhinosinusitis patients. However, controversy has surrounded defining QOL and measuring it reliably in these patient groups.

One definition of QOL is offered by the World Health Organization: ‘QOL includes psychological and social functioning as well as physical functioning and incorporates positive aspects of well-being as well as negative aspects of disease or infirmity’. Health-related quality of life (HRQL) focuses on the part of QOL that is influenced by health: the functional effects of an illness and its treatment on a patient, as perceived by the patient (12). To measure HRQL, questionnaires have been developed to provide a standardized, quantified and summarized version of the patients’ physical symptoms and the functional and psychosocial consequences of the disease and treatment. Traditionally, QOL studies have focussed on assessment of generic QOL. The generic HRQL questionnaires, such as the SF-36, allow comparison among patients with different diseases (11). However, generic instruments may be unresponsive to small – but to the patient important – changes in HRQL. Juniper (13) was the first to publish the development of disease-specific HRQL instruments for rhinitis and Piccirillo (14) was the first regarding rhinosinusitis. In recent years, multiple other HRQL questionnaires have been developed specifically for rhinitis and rhinosinusitis. Several reviews described HRQL questionnaires concerning rhinitis and/or (rhino)sinusitis (15–22). Despite these laudable efforts, little attention has been paid to the quality of the psychometric properties of these questionnaires. As good properties are the key to the successful use of questionnaires, the current review was undertaken to assess the quality of the disease-specific HRQL questionnaires specific for rhinitis and rhinosinusitis for adults.

Review Methodology

Identification of questionnaires

The following selection criteria were applied: (i) the questionnaire must be specifically designed for adults with rhinitis and/or rhinosinusitis. (ii) The questionnaire should measure all three aspects of HRQL: physical, functional and psychosocial; thereby excluding symptom lists.

Identification of articles of the questionnaires

The following selection criteria were applied: (i) the primary objective of the article must be the development and validation of the questionnaire. Articles addressing other objectives (e.g. treatment evaluation) were excluded. (ii) The article must be written in English.

Literature search

We reviewed the literature up to May 2007 in Pubmed, EMBASE and Medline, to identify studies of interest. We performed a search strategy using the terms: QOL, disease specific, health related questionnaire, instrument, sinusitis, rhinitis, rhinosinusitis, sinonasal, nasal polyposis; filter: diagnostic test validation and limits: English, adults. The reference lists were cross-referenced to identify new articles. Moreover the database of the AAAAI QOL Resources (http://www.aaaai.org/professionals/quality_of_life; accessed 10 July 2007) and the Patient-reported Outcome and QOL Instruments Database (http://www.proqolid.org; accessed 10 July 2007) were searched.

Quality assessment of the questionnaire: characteristic definitions and criteria

The development of a HRQL questionnaire involves multiple steps (23–25). Each step has its influence on the final quality of the questionnaire. The quality of the following characteristics of each questionnaire was evaluated: (i) construction: measurement goals, item generation and item reduction; (ii) description; (iii) feasibility; (iv) validation study; (v) psychometric properties: reliability, validity, responsiveness and clinically significant change.

The characteristics and criteria for the quality-assessment were defined. A scoring system was additionally constructed. A total of 18 points could be obtained (Table 1). Two researchers independently scored the articles and additionally discussed the discrepancies to achieve consensus.

Table 1.   Characteristics and criteria for quality-assessment
PropertyPartCriterionPoints
Construction
Measurement GoalsTargeted patient populationIf provided1
Purpose:
 discrimination and/or evaluation
If provided1
For use in:
 (clinical) trial or clinical practice
Used for level of reliability-
Item generationSources:
 literature (incl. questionnaires)
 clinician
 patients
If all three sources are used1
Item reductionApproach:
 conceptual
patient feedback
statistical analysis
If all three methods are used1
Scale construction:
 conceptual
patient feedback
statistical analysis
If all three methods are used1
Description
 Items, domains, response, scoreIf all four are provided1
TimeframeIf provided1
Feasibility
 Feedback of patientsIf obtained1
Completion timeIf provided1
Validation study
 Kind of patientsIf representative of target patient population1
Number of patientsIf ≥1001
Psychometric properties
ReliabilityInternal reliabilityAt group level: Cronbach’s α ≥ 0.7 or
At individual level: Cronbach’s α ≥ 0.9
1
Test-retest(significant t-test and Pearson/Spearman) or (ICC):
At group level: correlation ≥0.7 or
At individual level: correlation ≥0.9
1
ValidityContent validityIf confirmed (qualitative)1
Convergent validityIf correlation is between 0.4 and 0.81
Discriminant validityIf the purpose is:
 evaluation: this item is NA
discrimination: P-value <0.05

NA
1
ResponsivenessIf the purpose is:
 evaluation: P-value <0.05 or responsiveness statistic is ≥0.5
discrimination: this item is NA

1
NA
Clinically significant changeIf the purpose is:
 evaluation: used method and outcome provided
discrimination: this item is NA

1
NA

Construction

Measurement goals.  A point can be obtained for the description of the target patient population. Another point can be obtained for the description of the purpose of the questionnaire: discrimination between patients (measuring differences between subjects at one point in time) and evaluation (measuring changes within subjects over time). A questionnaire can be used in a clinical trial (comparison at group level) or in clinical practice (comparison at individual level). No points are awarded for this item. This information is only used for determining the level of reliability, i.e. group or individual level.

Item generation.  A point can be obtained for consulting the three sources that can be used for devising items: research literature (including existing questionnaires), clinicians and patients. If the questionnaire is a standardized or shortened version of an existing questionnaire, it will receive the points of the parent questionnaire. If an existing questionnaire is modified, for example for a different target population, it is judged as a separate questionnaire.

Item reduction.  The number of items can be reduced on conceptual grounds, based on either patient feedback or by statistical analysis. The conceptual basis refers to theoretical and/or clinical insight. A point can be obtained when all three approaches are used. At the end of the reduction phase, the items can be combined to form domains or scales. This can be done by the same approaches as used for item reduction: conceptual ideas, patient feedback or statistical analysis. Again, a point can be obtained for using the three approaches. If the questionnaire is a standardized or shortened version of an existing questionnaire, it will receive the points of the parent questionnaire, if the domains remained the same. If an existing questionnaire is modified, for example for a different target population, it is judged as a separate questionnaire. If all items are summed into one overall domain, this characteristic is not applicable.

The description of the questionnaire.  A point can be obtained for the description of the number of items, number of domains (or scales), the response options (continuous or categorical), and the scale level of the score (domain and/or composite); a point is obtained if all these four aspects are described. The timeframe (recall period) is the period that the patient has to refer to for the representation of the complaints. Another point can be obtained for the description of the timeframe.

The feasibility of the questionnaire.  The instrument’s feasibility (ease of administration) depends on the patients’ understanding and the burden of completing the questions. The feasibility can be assessed by pretesting the questionnaire in a number of patients who give their feedback on the wording and the comprehensibility. A point can be obtained if such patient feedback is obtained. Another point can be obtained for providing the completion time.

The validation study of the questionnaire.  The questionnaire can only be properly tested in patients, if they are the same as or representative of the target population. A point can be obtained if the patient population included in the validation study is the same as or representative for the target population. The psychometric properties can only be tested adequately if the sample size is sufficiently large. Another point can be obtained if the number of patients used in the validation study is at least 100. As this criterion is arbitrary, the number of patients is provided in the table.

The psychometric properties of the questionnaire

Reliability.  Reliability concerns the extent to which a questionnaire is free from random or systematic error. It is mostly measured by two components: internal reliability and test-retest reliability. Internal reliability measures the correlation among questions and reflects the tendency for items within a domain or scale to measure a common concept. It is usually estimated by calculating Cronbach’s α (range 0–1). If a scale has an alpha of at least 0.7, the scale is considered to be reliable for group level comparison and a value of 0.9 or more for assessment at the individual level (26). In these cases, a point can be obtained. Test-retest reliability is the ability of a measure to give similar results under similar conditions – also called reproducibility. This may be assessed by a t-test and a Pearson or Spearman correlation coefficient or the intra-class correlation coefficient. A correlation coefficient runs between −1 and 1; where 1 means perfect positive association. A correlation coefficient of a magnitude of at least 0.70 is adequate for group level comparisons and of 0.9 or more for comparison at the individual level (23). A point can be obtained if there is no statistically significant change with a t-test and a correlation coefficient of at least 0.70 or 0.90 depending on the measurement goals (group or individual level).

Validity. If the instrument measures what it purports to measure it is called valid. Different types of validity can be distinguished. Content validity is the appropriateness of an instrument for a particular task and is generally assessed by having experts and/or patients with the target condition review an instrument for its breadth of coverage (27–29). One point is obtained, if the content is confirmed by experts or patients. Construct validity is confirmed, if an instrument behaves according to underlying hypotheses, i.e. whether the measured variables behave in a consistent way with theoretical and/or clinical expectations. Construct validity is often divided into convergent and discriminant validity. Convergent validity is confirmed if the instrument correlates with another instrument of the same concept, i.e., correlations range from 0.4 to 0.8 (23). The instruments are considered too similar if the correlation is 0.8 or more; the tested instrument has no added value. A point is obtained if the correlation falls within this intermediate range. Discriminant validity or known-groups method refers to the instrument’s ability to differentiate between populations that are known or expected to differ. A point can be obtained if the discriminant validity is confirmed: a statistically significant difference (P < 0.05) between distinct groups of patients. If the measurement goal is evaluation over time, the discriminant validity is not applicable.

Responsiveness (30). Responsiveness is the ability of an instrument to detect change when the change occurs. There are many responsiveness statistics (24, 31–33). A point can be obtained, if the responsiveness is confirmed by a statistically significant change (P < 0.05) or if a responsiveness statistic of at least 0.5 (meaning: moderate to highly sensitive to change) (34).

Clinically significant change (30). Clinical significance goes beyond statistical significance and refers to the extent to which the statistically significant difference is large enough to have implications for patient care (35). There are two families of methods to detect clinically significant change: anchor-based and distribution based methods (36). We will only describe anchor-based methods, and the Minimal (Clinical) Important difference (M(C)ID) specifically. These methods link the observed change to an independent interpretable criterion, such as the patient’s perception of the magnitude, importance, or desirability of this change or clinical status (31). The M(C)ID establishes the smallest amount of change that a patient would say results in benefit (37). Juniper has shown that for instruments that use a 7-point scale, the M(C)ID is approximately 0.5 (37). A point can be obtained if the method and outcome of the clinically significant change are provided, irrespective of its magnitude.

Clinician’s guide: choosing a questionnaire

Clear measurement goals are necessary to use a questionnaire appropriately. For a questionnaire to offer accurate interpretations, it must also yield adequate levels of reliability, validity and responsiveness. Finally, establishment of clinical significance is needed. For recommendation, we singled out four criteria: the measurement goals, discriminant validity, responsiveness and the obtained score in the quality assessment of the instrument. If the purpose is discrimination, a maximum score of 16 can be obtained and for evaluation 17. If the purpose is discrimination and evaluation, the maximum score is 18.

Results

Identification of questionnaires

Thirteen questionnaires were identified: six for rhinitis, five for rhinosinusitis and two were miscellaneous (Fig. 1). Four questionnaires were excluded because they were designed for children, six because they were generic and 13 questionnaires did not fulfil the criteria for HRQL measurement.

Figure 1.

 Selection of Questionnaires. Abbreviations in order of appearance: AdolRQLQ, Adolescent RQLQ; PRQLQ, Paediatric RQLQ; SN-5, Sinonasal survey-5; ISAAC questions, International study of asthma and allergies in children rhinitis symptom questionnaire; SF-36, Medical Outcomes Study 36-Item Short form health survey; SF-12, Medical Outcomes Study 12-Item Short form health survey; SIP, Sickness impact profile; NHP, Nottingham health profile; EQ-5D, EuroQol; CSS, Chronic sinusitis survey; RSI, Rhinosinusitis symptom inventory; Chronic sinusitis TyPE, Technology of patient experience; FNQ, Fairley nasal questionnaire; SNAQ, Sinonasal assessment questionnaire; SOQ, sinusitis outcomes questionnaire; WPAI-Allergy specific, work productivity and activity impairment; RSUI, rhinitis symptom utility index. *There is no validation study of the SNOT-22 or the HRQLQ Rhinosurgery. **EQ-5D, WPAI- allergic rhinitis and RSUI are designed for cost-effectiveness studies.

Quality assessment of the characteristics of the questionnaires

We divided the results of the quality assessment into two tables for convenient arrangement. The results of the construction, description and feasibility of the instrument can be found in Table 2 and the results of the validation study and psychometric performance of the instrument in Table 3. The questionnaires were clustered per topic: rhinitis, rhinosinusitis and miscellaneous; within each cluster, the questionnaires appear in chronological order of publication.

Table 2.   Construction, description and feasibility of the instrument
QuestionnaireConstruction*DescriptionPFeasibilityPPoints
Measurement GoalsPItem generationPItem reductionP
  1. P, points; NA, not applicable; NR, not reported.

  2. *If all items are summed into one overall domain, the scale construction is not applicable.

RQLQ
Rhinoconjunc-tivitis Quality of Life Questionnaire (13)
Patient population: rhinoconjuctivitis
Purpose: evaluation
Fur use in: trial
1

1
Sources: literature clinician patients1Approach: patient feedback
Scale construction: conceptual
0

0
Items: 28
Domains: 7 (nasal, eye, nonhay-fever, sleep disturbance, activity limitations, practical and emotional problems)
Response: continuous: 7 point scale
Score: composite
Timeframe: 1 week
1





1
Feedback: Yes
Completion time: 5–15 min
1
1
7
Rhinitis QOLQ
Rhinitis Quality of Life Questionnaire (45)
Patient population: PAR
Purpose: evaluation
Fur use in: trial
1
1
Sources: literature0Approach: conceptual
Scale construction: conceptual
0
0
Items: 24
Domains: 6 (nasal-, nonrhinitis, sleep-, practical problems, activities, emotional)
Response: continuous: 7 point scale
Score: domain, composite
Timeframe: 1 week
1




1
Feedback: No
Completion time: NR
0
0
4
RQLQ (S)
Standardized version of RQLQ (38)
Patient population: SAR and PAR
Purpose: evaluation
For use in: trial
1

1
Sources: As parent RQLQ1Approach: conceptual
Scale construction:
As parent RQLQ
0
0
Items: 28
Domains: 7; see RQLQ
Response: continuous: 7 point scale
Score: composite
Timeframe: 1 week
1



1
Feedback: Yes
Completion time: NR
1
0
6
MiniRQLQ
Mini version of RQLQ (52)
Patient population: rhinoconjunctivitis
Purpose: evaluation and discrimination (of impairment)
For use in: trial
1

1
Sources: As parent RQLQ1Approach: conceptual patient feedback statistics
Scale construction: conceptual
1


0
Items: 14
Domains: 5; activity limitations, practical problems, nose-, eye- and other symptoms
Response: continuous; 7 point scale
Score: domain, composite
Timeframe: 1 week
1





1
Feedback: Yes
Completion time: NR
1
0
7
ROQ
Rhinitis Outcomes Questionnaire (57)
Patient population: AR
Purpose: evaluation
For use in: practice
1
1
Sources: literature clinician0Approach: conceptual
Scale construction: statistics
0
0
Items: 26
Domains: 4; nose, eye, chest, system
Response: continuous: 6 point scale
Score: domain
Timeframe: present
1



1
Feedback: Yes
Completion time: <5 min
1
1
6
NRQLQ
Nocturnal RQLQ (59)
Patient population: nocturnal AR
Purpose: evaluation
 and discrimination
 (of severity)
For use in: practice
1

1
Sources: literature clinician patients1Approach: patient feedback
Scale construction: conceptual
0

0
Items: 16
Domains: 4; sleep problems, symptoms during sleep time, symptoms on waking, practical problems
Response: continuous: 7 point scale
Score: domain, composite
Timeframe: 1 week
1





1
Feedback: Yes
Completion time: NR
1
0
6
RSOM-31
Rhinosinusitis Outcome Measure (14)
Patient population: rhinosinusitis
Purpose: evaluation and discrimination (disease vs no-∼)
For use in: practice
1

1
Sources: literature clinician patients1Approach: conceptual patient feedback statistics
Scale construction: conceptual statistics
1


0
Items: 31
Domains: 7; nasal, eye, ear, sleep, general, emotional, functional problems
Response: continuous: 4 + 6 point scale
Score: domain, composite
Timeframe: 2 week
1




1
Feedback: No
Completion time: 20 min
0
1
7
SNOT-16
Sinonasal outcome test (62)
Patient population: rhinosinusitis
Purpose: evaluation and discrimination (disease vs no-∼)
For use in: practice
1

1
Sources: As parent RSOM-311Approach: conceptual patient feedback
Scale construction: NA
0

-
Items: 16
Domains: 1
Response: NR
Score: composite
Timeframe: NR
0



0
Feedback: No
Completion time: NR
0
0
3
SNOT-20
Sinonasal outcome test (64)
Patient population: rhinosinusitis
Purpose: evaluation and discrimination (disease vs no-∼)
For use in: practice
1

1
Sources: As parent RSOM-311Approach: conceptual
 patient feedback statistics
Scale construction: NA
1


-
Items: 20
Domains: 1
Response: continuous: 6 point scale
Score: composite
Timeframe: NR
1



0
Feedback: No
Completion time:
10 min
0
1
6
RSDI
Rhinosinusitis Disability Index (65)
Patient population: rhinosinusitis
Purpose: evaluation and discrimination (disease vs no-∼)
For use in: practice
1

1
Sources: literature clinician patients1Approach: statistics
Scale construction: statistics
0
0
Items: 30
Domains: 3; emotional, functional, physical
Response: continuous: 5 point scale
Score: NR
Timeframe: NR
0




0
Feedback: No
Completion time: <5 min
0
1
4
RhinoQOL
(68)
Patient population: sinusitis
Purpose: evaluation and discrimination (sinusitis and rhinitis)
For use in: practice
1

1
Sources: literature clinician patients1Approach: conceptual patient feedback statistics
Scale construction: conceptual patient feedback
1



0
Items: 17
Domains: 3; symptom frequency, bothersomeness, impact scale
Response: continuous 5 + 11 point scale
Score: domain
Timeframe: 1 week
1




1
Feedback: Yes
Completion time: 7 min
1
1
8
GNPI
General nasal patient inventory (69)
Patient population: rhinology patients
Purpose: evaluation
For use in: practice
1

1
Sources: patients0Approach: patient feedback
Scale construction: NA
0

-
Items: 45
Domains: 1
Response: continuous: 4 point scale
Score: composite
Timeframe: NR
1



0
Feedback: No
Completion time: NR
0
0
3
Rhinastma (70) Patient population: rhinoconjuctivitis and asthma
Purpose: evaluation and discrimination (rhinitis vs rhinitis and asthma)
For use in: practice
1


1
Sources: literature clinician patients1Approach: patient feedback statistics
Scale construction: statistics
0

0
Items: 30
Domains: 3; names not specified
Response: continuous: 5 point scale
Score: NR
Timeframe: NR
0



0
Feedback: No
Completion time: NR
0
0
3
Table 3.   Validation study and psychometric properties of the instrument
QuestionnaireValidation studyPPsychometric properties*Points
ReliabilityPValidityPResponsivenessPClinical significanceP
  1. P, points; NA, not applicable; NR, not reported; n, number of patients; t, time.

  2. *If the measurement goal is solely evaluation the discriminant validity is not applicable.

RQLQ
Rhinoconjunc-tivitis Quality of Life Questionnaire (13)
Patients: pollen rhinoconjuctivitis
Number of patients: 60
1
0
Internal reliability: NR
Test-retest: ICC(overall score):0.86 (n = 60; t = 2 weeks)
0
1
Content validity: NR
Convergent validity: NR
Discriminant validity: NA
0
0
-
t-test: P < 0.05 (n = 60) (overall and domains)1Correlation: change in diary and change in RQLQ
Pearson = 0.31–0.59
14
Rhinitis QOLQ
Rhinitis Quality of Life Questionnaire (45)
Patients: perennial rhinitis
Number of patients: 58
1
0
Internal reliability: NR
Test-retest: NR
0
0
Content validity: NR
Convergent validity: Correlation with diary Pearson 0.63–0.78
Discriminant validity: NA
0
1

-
Repeated measures anova (domains) (n = 58) 1NR03
RQLQ (S)
Standardized version of RQLQ (38)
Patients: rhinoconjunctivitis
Number of patients: 100
1
1
Internal reliability:α (overall score): 0.93
Test-retest: ICC(domains): 0.91– 0.97 (n = 44; t = 1–4 weeks)
1


1
Content validity: Yes
Convergent validity: Correlation with diary
Pearson : 0.53–0.69
Discriminant validity: NA
1
1


-
Responsiveness index = 0.75 (n = 83) (overall and domains)1MID = 0.48 ± 0.9318
MiniRQLQ
Mini version of RQLQ (52)
Patients: rhinoconjuctivitis
Number of patients: 100
1
1
Internal reliability:α (overall score): 0.90
Test-retest
ICC(overall score):0.93 (n = 44; t = 1–4 weeks)
1


1
Content validity: Yes
Convergent validity: Correlation with diary Pearson: 0.73
Discriminant validity: NR
1
1

0
Responsiveness index = 0.83 (n = 83) (overall and domains)1MID: 0.7018
ROQ
Rhinitis Outcomes Questionnaire (57)
Patients: allergic rhinitis
Number of patients: 175
1
1
Internal reliability:α (domains): 0.80–0.92
Test-retest: NR
0

0
Content validity: Yes
Convergent validity: NR
Discriminant validity: NA
1
0
-
t-test: P < 0.05 (overall and items)1NR04
NRQLQ
Nocturnal RQLQ (59)
Patients: nocturnal allergic rhinitis
Number of patients:106
1
1
Internal reliability: NR
Test-retest: ICC(overall score):0.87 (n = 80; t = 1 week)
0
0
Content validity: Yes
Convergent validity: Correlation with diary and RQLQ (domains): Pearson 0.38–0.82
Discriminant validity: NR
1
1


0
Responsiveness index = 0.93 (n = 77; t = 3 weeks) (overall and domains)1NR05
RSOM-31
Rhinosinusitis outcome measure (14)
Patients: rhinosinusitis
Number of patients:142
1
1
Internal reliability:α (overall score): 0.95
Test-retest: t-test without correlation
1

0
Content validity: Yes
Convergent validity: Global QOL question anova, P < 0.001
Discriminant validity: Compared to hearing loss patients: t-test <0.05
1
1

1
anova
P < 0.0001 (overall and domains)
1MCID = 1
 or 30%
 different
18
SNOT-16
Sinonasal Outcome Test (62)
Patients: rhinosinusitis
Number of patients: 22
1
0
Internal reliability:α (overall score): 0.89
Test-retest: NR
1

0
Content validity: NR
Convergent validity: Correlation with overall QOL question: Spearman 0.42
Discriminant validity: Compared to hearing loss patients: t-test <0.05
0
1


1
SRM: 0.69 (overall)1NR05
SNOT-20
Sinonasal Outcome Test (64)
Patients: rhinosinusitis
Number of patients: 46
1
0
Internal reliability:α (overall score): 0.90
Test-retest: t-test <0.05 with Pearson (overall score): 0.90 (n = 15; t = 2–4 weeks)
1

1
Content validity: Yes
Convergent validity: With Global QOL questionanovaP = 0.002
Discriminant validity: Compared to hearing loss patients: t-test <0.05
1
1

1
SRM: 0.4 (n = 46) (overall)0MCID > 0.817
RSDI
Rhinosinusitis Disability Index (65)
Patients: rhinitis or rhinosinusitis
Number of patients: 87
1
0
Internal reliability:α (overall score): 0.95
Test-retest: Spearman (overall score): 0.69 (n = 25; t = 5 days)
1

0
Content validity: NR
Convergent validity: NR
Discriminant validity: Compared to no nasal problems: t-test <0.05
0
0
1
NR0NR03
RhinoQOL (68) Patients: rhinosinusitis
Number of patients: 115
1
1
Internal reliability:α (domains): 0.28–0.89
Test-retest
ICC(domains): 0.57–0.67 (t = 2 weeks)
0


0
Content validity: Yes
Convergent validity: Correlation with SF-12
Pearson 0.06–0.57
Discriminant validity: compared to asymptomatic and AR patients: t-test <0.05
1
0


1
Effect size: 3.4–4.3
Responsiveness index: 2.3–3.0 (t = 4 weeks) (overall and domains)
1Correlation between change in Global Rating Score and Change in QOL- Score
P < 0.001
16
GNPI
General Nasal Patient Inventory (69)
Patients: general rhinology
Number of patients: 153
1
1
Internal reliability: NR
Test-retest: NR
0
0
Content validity: NR
Convergent validity: Correlation with Fairley nasal Q: Pearson 0.79
Discriminant validity: NA
0
1


-
NR0NR03
Rhinastma (70)Patients: AR and asthma
Number of patients: 103
1
1
Internal reliability:α (domain): 0.76–0.93
Test-retest: Pearson (domains) 0.61–0.92 (n = 31; t = 1 week)
0

0
Content validity: Yes
Convergent validity: NR
Discriminant validity: AR vs AR with asthma
14/30 items significantly different
1
0
0
Wilcoxon 18/30 items significantly different (t = 3 weeks) 0NR03

All analysed articles reported the measurement goals; 10 (of 13) met our criteria of the sources of item generation; however, only four instruments met our criteria for item reduction and none for the scale construction. Five articles did not report a full description (of items, domains, response options, score and timeframe). The reports on the feasibility varied, only three received the maximum 2 points.

All the validation studies included patients who were the same as or representative of the target population of the questionnaire; eight studies included more than 100 patients. Only three questionnaires met our criteria of good reliability, for rhinitis: the RQLQ(S) and the mini-RQLQ, for rhinosinusitis the SNOT-20. Content and convergent validity were reported for eight questionnaires adequately. Of five questionnaires, the purpose was evaluation only; therefore, the item of discriminant validity is not applicable. Of the other eight questionnaires, five reported adequate discriminant validity. The responsiveness was measured of 10 instruments, of which eight did meet our criteria. However for only six cases the clinical significance of this change was reported.

Clinician’s guide: choosing a questionnaire

The obtained overall score of the quality assessment ranged from 6 to 15 points (out of 17 or 18) (Table 4). We identified seven HRQL questionnaires with adequate levels of discriminant validity (where applicable) and responsiveness: for rhinitis, the RQLQ; rhinitis QOL; RQLQ(S) and the ROC and for rhinosinusitis: the RSOM-31, SNOT-16 and the RhinoQOL. However, the rhinitis QOL, the ROC and the SNOT-16 obtained a low overall score.

Table 4.   Clinician’s guide: choosing a questionnaire
QuestionnaireMeasurement goalsDiscriminant validityResponsivenessPoints*
x + y = z/m
Patient populationPurpose
  1. *x is the number of points obtained for the construction, description and feasibility; y is the number of points obtained for the validation study and psychometric properties; z is the total number of points obtained; m is the maximum number of point the questionnaire could have obtained (note: if the purpose is only evaluation the maximum number is 17; if the purpose is discrimination and evaluation the maximum number is 18; if the number of domains is 1 the maximum score decreases by 1).

Questionnaires for rhinitis
RQLQRhinoconjunctivitisEvaluationNot applicableYes7 + 4 = 11/17
Rhinitis QOLPerennial rhinitisEvaluationNot applicableYes4 + 3 = 7/17
RQLQ(S)Seasonal and perennial rhinitisEvaluationNot applicableYes6 + 8 = 14/17
MiniRQLQRhinoconjunctivitisEvaluation; discrimination (of impairment)NoYes7 + 8 = 15/18
ROQAllergic rhinitisEvaluationNot applicableYes6 + 4 = 10/17
NRQLQNocturnal allergic rhinitisEvaluation; discrimination (of disease severity)NoYes6 + 5 = 11/18
Questionnaires for rhinosinusitis
RSOM-31RhinosinusitisEvaluation; discrimination (disease vs no-∼) YesYes7 + 8 = 15/18
SNOT-16RhinosinusitisEvaluation; discrimination (disease vs no-∼)YesYes3 + 4 = 7/17
SNOT-20RhinosinusitisEvaluation; discrimination (disease vs no-∼)YesNo6 + 7 = 13/17
RSDIRhinosinusitisEvaluation; discrimination (disease vs no-∼)YesNo4 + 3 = 7/18
RhinoQolSinusitisEvaluation; discrimination (sinusitis and rhinitis)YesYes8 + 6 = 14/18
Questionnaires linked to rhinitis and rhinosinusitis
GNPIRhinology patientEvaluationNot applicableNo3 + 3 = 6/16
RhinasthmaRhinoconjunctivitis and asthmaEvaluation; discrimination (rhinitis vs rhinitis + asthma)NoNo3 + 3 = 6/18

The highest scores points were obtained for rhinitis by the RQLQ(S) and the mini-RQLQ; for rhinosinusitis by the RSOM-31, the SNOT-20, and the RhinoQOL. However, the mini-RQLQ did not measure discriminant validity and the responsiveness statistic did not exceed 0.5 for the SNOT-20.

Discussion

On the basis of our systematic review of the existing disease-specific HRQL questionnaires concerning rhinitis and rhinosinusitis for adults, we identified of the top scoring instruments only four HRQL questionnaires with adequate levels of discriminant validity (where applicable) and responsiveness: for rhinitis, the RQLQ and RQLQ(S) and for rhinosinusitis: the RSOM-31 and RhinoQOL. In conclusion of the results, the development of a questionnaire was given limited attention in general, and the item reduction, scale construction, reliability testing and clinical significance in particular.

A number of limitations of the current review merit attention. First, we limited the search to those articles whose primary objective was the development and validation of the questionnaire. We necessarily discarded the many articles that used the questionnaire and that may have provided additional information on the psychometric properties. Secondly, we chose our criteria on the basis of the literature and to the best of our abilities. However, if a questionnaire does not meet our criteria it is not necessarily a poor instrument. Our systematic review is meant as a tool for choosing the most appropriate questionnaire, in a specific context. For example, if the questionnaire is to be used in clinical practice, the reliability is the deciding factor. Moreover, the clinician’s guidelines are meant to aid in making a proper choice for an instrument. For example, in the context of a longitudinal study, the responsiveness is of major importance, whereas in a cross-sectional study, the discriminant validity is the determining factor for choosing the most appropriate questionnaire.

The current review points to areas that would merit from further study. Whereas the literature on the reliability testing, responsiveness and clinical significance of the rhinitis and rhinosinusitis instruments is growing, this information is not available for all instruments. The field would benefit when such information would become available for those instruments as well.

In this review, we have focused on disease-specific instruments designed for evaluation and discrimination. These instruments are of interest to clinicians in a research context and during consultation. An emerging field is cost-effectiveness analysis that relates the relative expenditure (costs) to outcomes (effects) of two or more courses of action. The results will be used by policy makers and health insurance companies. The growing prevalence of rhinitis and rhinosinusitis with their impact on HRQL and the increasing demand for health care increase the interest in cost-effectiveness studies and therefore, in instruments designed specifically for that purpose.

The measurement of HRQL and the development and testing of HRQL questionnaires for rhinitis and rhinosinusitis are (still) a growing field. The current review, with its quality assessment of the existing disease-specific HRQL questionnaires concerning rhinitis and rhinosinusitis for adults, assists in making a deliberate choice for the most appropriate questionnaire, in a specific context.

Ancillary