Edited by: Wytske Fokkens
A new instrument for the assessment of patient-defined benefit in the treatment of allergic rhinitis
Article first published online: 1 DEC 2010
© 2010 John Wiley & Sons A/S
Volume 66, Issue 5, pages 665–670, May 2011
How to Cite
Franzke, N., Schäfer, I., Jost, K., Blome, C., Rustenbach, S. J., Reich, K., Reusch, M., Maurer, M. and Augustin, M. (2011), A new instrument for the assessment of patient-defined benefit in the treatment of allergic rhinitis. Allergy, 66: 665–670. doi: 10.1111/j.1398-9995.2010.02515.x
- Issue published online: 7 APR 2011
- Article first published online: 1 DEC 2010
- Accepted for publication 2 November 2010
- allergic rhinitis;
- benefit assessment;
- patient benefit index;
- patient-reported outcomes;
- quality of life
To cite this article: Franzke N, Schäfer I, Jost K, Blome C, Rustenbach SJ, Reich K, Reusch M, Maurer M, Augustin M. A new instrument for the assessment of patient-defined benefit in the treatment of allergic rhinitis. Allergy 2011; 66: 665–670.
Background: Allergic rhinitis (AR) is a common disease that affects health-related quality of life (HRQoL). Current and future health policy demands the assessment of patient-relevant treatment benefit for evaluation of treatments.
Methods: We developed, validated and tested a standardized instrument for the assessment of patient-relevant needs and benefits in AR. In an open survey of patients with AR, 100 need and benefit items were generated. The items were condensed to a 25-item list by an expert panel. On this list, patient rates the personal importance of 25 treatment needs on a scale ranging from ‘not at all’ to ‘very’ before treatment (Patient Needs Questionnaire, PNQ). At the end of the treatment, patient rates the extent, to which these needs were achieved by treatment from ‘did not help at all’ to ‘helped a lot’ (Patient Benefit Questionnaire). The patient benefit index (PBI) is computed to provide a global weighted benefit parameter. This disease-specific instrument was validated in n = 104 patients with AR.
Results: The PBI-AR showed good acceptability and feasibility in clinical routine. Reduction in nose and eye symptoms was rated most important. The PBI amounted to 2.2 (PBI ranges from 0 = ‘no benefit’ to 4 = ‘maximum benefit’). Reliability of the PNQ was high (Cronbach’s alpha = 0.9).The PBI was significantly correlated with relevant external validation criteria, such as patient satisfaction (R = 0.54) and HRQoL (R = 0.26).
Conclusion: The PBI-AR is a feasible, reliable and valid instrument for the standardized assessment of patient-relevant benefits in the treatment of AR.
Allergic rhinitis (AR) is one of the most common diseases in Western Europe, with a prevalence of up to 20% (1). Because of severe burden of disease and substantial reduction in health-related quality of life (HRQoL) in many patients, the treatment of AR is of considerable socio-economic relevance (2–5).
A broad spectrum of treatments of AR, such as intranasal topical preparations, system therapeutics and hyposensitization treatments, has been available in the last decade (2). Nevertheless, many of the patients are dissatisfied with treatment (6, 7), because comprehensive patient information and appropriate consultation is lacking. Moreover, knowledge of adequate treatment options and patient needs is still missing.
Evidence of treatment benefit gains importance and is demanded explicitly by legislators. In the past, this benefit evaluation was based exclusively on the assessment of medical efficacy. Today, according to the rules of procedure of the GBA (German Federal Committee) and the IQWiG (German Institute for Quality and Efficiency in Medicine), priority is given to patient-relevant benefit (8–10). According to these rules of procedure and the German social laws, attributes of patient-relevant benefit are the reduction in mortality and morbidity, the improvement of quality of life, treatment satisfaction and decrease in treatment burden.
Because of these health policy conditions, cost and innovation pressure in the treatment of allergic diseases have increased – as a consequence, health services are further restricted (11). Appropriate methods for the evaluation of the demanded patient-relevant benefit in the clinical routine of AR management are still missing. This study was performed to develop and validate an instrument for the assessment of patient-relevant benefit in the treatment of AR and – in a second step – to test the feasibility and acceptability of this instrument in clinical routine.
Materials and methods
Data protection and ethics
The study was approved by the Ethics committee of the German medical association in Hamburg and conducted in accordance with data protection regulations.
Assessment of patient benefit
The instrument for the assessment of patient benefit in rhinitis allergica was adopted from a validated generic patient benefit assessment instrument in dermatology (12). While the method of assessment was essentially the same for both instruments, the items of the generic instrument were replaced by disease-specific items taken from an item pool. The following paragraphs describe the item derivation and the assessment method in more detail.
Compilation of the item pool
First, an item pool of patient-relevant treatment needs and benefits was generated in an open patient survey (n = 50) in accordance with the internationally accepted standards of psychometric and biometric test development (13–15). To generate this pool of items, patients with AR were asked in an open text form about their burden of disease and treatment needs.
Compilation of the item list
The pool of more than 100 items was reduced to a 25-item list by a six-member expert panel made up of an allergologist, two dermatologists, one epidemiologist and two patients with AR. In the panel-discussion, each item was examined for (1) representativeness, (2) uniqueness and (3) suitability for use in the questionnaires. The fully consented version finally consisted of 25 items nonredundant and nonoverlapping patient-relevant therapy needs and benefits. A protocol was written of the expert meeting, including the item discussion. Each of the 25 items on the list was supplemented with a five-point Likert scale, and a response option ‘does not apply to me’ and used to build two questionnaires – both with 25 items in identical format – given before and at the end of treatment with different instructions: At the start of treatment, the patient is requested to rate each of the 25 treatment items for his individual need (Patient Needs Questionnaire; PNQ) on a scale of 0 ‘not important at all’ to 4 ‘very important’. At the end of treatment, the patient is requested to rate the extent benefit has been achieved with the current or at last used treatment (Patient Benefit Questionnaire; PBQ), scaled from 0 ‘treatment did not help at all’ to 4 ‘treatment helped very much’.
Computation of the benefit index
The patient benefit index (PBI) is a global score of patient-relevant benefit. The PBI is computed by dividing each rating on a need item (PNQ) by the sum of all ratings in the PNQ, and multiplying this fraction with the respective benefit rating (PBQ). The PBI is the sum of these products and ranges from 0 ‘no benefit’ to 4 ‘maximum benefit’. According to a methodological study in psoriasis (16), a PBI ≥ 1 is considered as a threshold of ‘relevant benefit’.
Validation and testing of PBI-AR in clinical routine
The instrument was validated in a sample of 104 patients with AR.
- 1Study design: Cross-sectional observational multi-centre study.
- 2Patients and study centres: Questionnaires were distributed by mail to five study centres, including dermatological–allergological university outpatient clinics and dermatological and otorhinolaryngological practices. Data were collected between August and December of 2007. Patients of both sexes ≥18 years of age with intermittent and persistent AR were included.
- 3Data collection: The 13-page patient questionnaires included items of following domains: socio-demographic characteristics, anamnesis of disease, treatment satisfaction, willingness to pay, the Rhinitis Quality of Life Questionnaire (17) and the PBI.
The validation criteria and procedures were chosen in accordance with the guidelines of the German Society of Dermatology and the Association of the Scientific Medical Societies in Germany (AWMF) for the assessment of Quality of Life in Dermatology (18).
Analyses were performed with spss 15.0 for windows (SPSS Inc., Chicago, Illinois, USA). Means, Standard deviations, Median and range were computed to describe the sample and distributions of variables. Reliability of the PNQ was computed as internal consistency, and Spearman rank correlations were computed to validate the PBI in relation to chosen criteria.
Data were obtained from 104 patients. 62.5% of the patients (n = 65) were female. The mean age was to 40.2 ± 12.5 years (Table 1).
|Body mass index (male; female)||25.7; 22.9||3.7; 2.9||21.9; 25.5||22–35; 18–34||38; 64|
|Duration of disease (in years)||17.0||10.2||17.0||0–39||98|
|Doctor visits because of allergic rhinitis (AR) (n, in the last 12 months)||5.1||7.1||2.0||0–50||96|
|Treatment burden (0–4)||0.7||1.0||0.0||1–4||100|
|Evaluation of AR care (1 = very good to 5 = poor)||2.1||0.7||2.0||1–4||102|
|Satisfaction with AR treatment received (1 = very satisfied; 4 = very dissatisfied)||2.4||0.8||2.0||1–4||102|
|Willingness to pay (singular payment; €)||2743||5910||950||0–40 000||78|
|Patient benefit: patient benefit index (0–4)||2.3||1.1||2.3||0–4||100|
|Quality of life: Rhinoconjunctivitis quality of life questionnaire (0–6)||1.9||1.5||1.8||0.0–5.3||104|
The PBI-AR was well-accepted by the patients and proved practicable in clinical routine. The percentage of missing data was lower than 1% (median 0%). Several patients who did not receive treatment at the time of the study gave notification that the questionnaire did not apply to their situation at the moment.
The ratings in the PNQ show a broad spectrum of patient-relevant needs (Table 2): The mean importance was at 2.7 or higher on all treatment need items. The mean needs most important were ‘to be able to stay outdoors without symptoms’ (mean = 3.7), as well as to have less symptoms: ‘runny or stuffed-up nose’; ‘itch’; ‘burning or watery eyes’; ‘to be healed of all symptoms’ (means at 3.6). With a mean of 2.7, the needs ‘to be less burdened in your partnership’ and ‘to be able to have a normal sex life’ were least important to the patients.
|Needs||Mean*||SD*||Per cent quite/very important*||Per cent afflicted patients||n|
|…to be able to stay outdoors without symptoms||3.7||0.6||93.2||96.1||103|
|…to no longer have a runny or stuffed-up nose||3.6||0.7||90.4||98.1||104|
|…to not have itching on the eyes. nose or palate anymore||3.6||0.7||85.4||93.2||103|
|…to not have burning or watery eyes anymore||3.6||0.8||76.0||83.7||104|
|…to be healed of all symptoms||3.6||0.8||89.2||98.0||102|
|…to be able to breathe through my nose more freely||3.5||0.8||88.5||97.1||104|
|…to feel less fatigued or groggy||3.5||0.8||75.5||81.4||102|
|…to have confidence in the therapy||3.5||0.8||76.7||88.3||103|
|…to have an easily applicable treatment||3.4||0.9||77.5||95.1||102|
|…to be able to sleep better||3.4||0.9||61.8||71.6||102|
|…to experience a greater enjoyment of life||3.4||0.9||47.6||56.3||103|
|…to be able to engage in normal leisure activities||3.4||0.8||69.9||79.6||103|
|…to feel less depressed||3.3||1.1||46.1||53.9||102|
|…to be able to concentrate better at work||3.3||0.9||56.9||66.7||102|
|…to not have sneezing impulses||3.2||1.1||77.7||96.1||103|
|…to have no fear that the disease will become worse||3.2||1.0||50.0||66.7||102|
|…to be more productive in everyday life||3.2||0.9||60.2||75.7||103|
|…to be less dependent on doctor and clinic visits||3.2||1.1||64.1||79.6||103|
|…to have fewer side effects||3.2||1.1||61.2||81.6||103|
|…to feel less irritated||3.1||1.1||43.7||63.1||103|
|…to have fewer out-of-pocket treatment expenses||3.0||1.3||59.2||86.4||103|
|…to need less time for daily treatment||2.9||1.3||50.0||77.5||102|
|…to feel more comfortable showing yourself||2.8||1.2||33.3||48.0||102|
|…to be less burdened in your partnership||2.7||1.4||33.0||51.5||103|
|…to be able to have a normal sex life||2.7||1.4||27.5||42.2||102|
The reliability (internal consistency) of the PNQ was high, Cronbach’s alpha amounted to 0.91 (n = 94).
The benefits ‘to have an easily applicable treatment’ (mean 2.7) and ‘to have confidence in the therapy’ (mean 2.6) were achieved best through the respective treatment (Table 3). Least achieved benefits were ‘to have fewer out-of-pocket treatment expenses’ (mean 1.8) and ‘to be healed of all symptoms’ (mean 1.4).
|Treatment benefit||Mean*||SD*||Per cent helped rather/very much||n|
|…to have an easily applicable treatment||2.7||1.4||66.7||96|
|…to have confidence in the therapy||2.6||1.2||58.3||97|
|…to be able to engage in normal leisure activities||2.5||1.2||60.9||97|
|…to be able to have a normal sex life||2.5||1.6||53.3||94|
|…to be able to breathe through my nose more freely||2.4||1.3||51.6||98|
|…to feel more comfortable showing yourself||2.4||1.4||53.7||97|
|…to need less time for daily treatment||2.4||1.5||52.0||96|
|…to not have sneezing impulses||2.3||1.3||48.9||98|
|…to no longer have a runny or stuffed-up nose||2.3||1.3||48.4||97|
|…to be able stay outdoors without symptoms||2.3||1.4||52.3||98|
|…to experience a greater enjoyment of life||2.3||1.3||46.0||97|
|…to be more productive in everyday life||2.3||1.3||48.6||97|
|…to be able to concentrate better at work||2.3||1.3||49.2||96|
|…to not have itching on the eyes. nose or palate anymore||2.2||1.3||43.5||97|
|…to have fewer side effects||2.2||1.4||47.9||97|
|…to feel less irritated||2.1||1.3||41.7||96|
|…to not have burning or watery eyes anymore||2.1||1.3||41.8||96|
|…to be able to sleep better||2.1||1.3||37.3||98|
|…to have no fear that the disease will become worse||2.1||1.4||40.0||96|
|…to be less burdened in your partnership||2.1||1.4||41.9||97|
|…to be less dependent on doctor and clinic visits||2.1||1.4||38.2||95|
|…to feel less fatigued or groggy||1.9||1.4||32.4||96|
|…to feel less depressed||1.9||1.3||31.9||97|
|…to have fewer out-of-pocket treatment expenses||1.8||1.5||31.9||96|
|…to be healed of all symptoms||1.4||1.4||26.4||96|
Distribution of the PBI
The patients achieved a mean PBI of 2.3 ± 1.1 (Median: 2.3, n = 100) and ranged from 0 to 4. The PBI was slightly higher for men (2.5 ± 1.0) than for women (2.1 ± 1.1).
Applying a threshold of PBI ≤ 1, n = 86 patients (87.8%) attained relevant benefit from current treatment (Fig. 1).
The PBI was significantly correlated to treatment burden, positive evaluation of treatment and treatment satisfaction. Because of the scaling of the validation criteria, all correlations appear numerically negative. The interpretation of these correlations is ‘the more satisfied and the less burdened patients were, the higher the patient benefit, measured by PBI, was.’ Additionally, the PBI was significantly associated with all subscales and the total score of the Rhinoconjunctivitis quality of life questionnaire (RQLQ) (Table 4). As high scores on this instrument indicate high impairment, high quality of life was accompanied by high treatment benefit.
The PBI was not systematically associated with the frequency of visits to dermatologists, family doctors, otorhinolaryngologists, or internal specialists, or with the frequency of visits to other physicians (data not shown). There also was no association to age, body mass index, duration of disease, or willingness to pay (Table 5), which indicates discriminant validity.
|Body mass index||−0.04||98||0.730|
|Duration of the disease||−0.12||94||0.240|
|Burden of treatment||−0.31||97||0.002|
|Evaluation of the AR care over the last years||−0.56||98||0.001|
|Treatment satisfaction (present AR treatment)||−0.54||99||0.001|
|Willingness to pay (singular payment in €)||−0.09||95||0.390|
The aim of this study was the development, validation and feasibility testing in clinical routine of an instrument for the assessment of patient-relevant benefit in the treatment of AR.
The feasibility of the PBI-AR in clinical routine was good with regard to patient acceptability and comprehensibility. No study patient had relevant rates of missing values or posed directly negative comments. The distribution of items means in the Patient Needs Questionnaire (PNQ) revealed that patients with AR show a broad spectrum of treatment needs, which should be considered in the evaluation of anti-allergic therapy.
For interpretation of the benefits of treatment and the validation results, limitations of the study have to be taken into consideration. This first study was cross-sectional in design, and patients received various treatments at the time of study, and no treatment was evaluated across time. Additionally, some patients were burden-free at the time of study and thus probably achieved better HRQoL values in the RQLQ as they would have after an allergy-relevant week. The different stages of treatment and varying severities of disease are clearly reflected in the distribution of the PBI (Fig. 1). Nevertheless, the PBI was positively associated with patient-relevant validation criteria and varied reasonably with current treatment.
The instrument was developed by an interdisciplinary team in cooperation with patients with AR, in accordance with international standards for the development of psychometric and biometric instruments (13–15, 19–23) and the AWMF guidelines of the HRQ (German Society of Dermatology) for the assessment of quality of life in dermatology (18).
The methodology is a refinement of ‘goal attainment scaling’ techniques (24) and goal-oriented outcome measurement (25). Compared to these methods, the advantages of the PBI lie in the Likert scaling of need and benefit items and the computation of a single-weighted benefit index.
In conclusion, the PBI-AR is a feasible, reliable and valid instrument for the assessment of patient-relevant benefit in AR. The PNQ revealed a broad range of patient-relevant needs, and the PBQ appropriately captures treatment benefits achieved. Thus, the instrument permits a differentiated need and benefit assessment for treatment planning and evaluation.
Because multidimensional patient-relevant needs and benefits are combined into a single benefit parameter, the PBI is especially suited for outcomes research on comparative efficacy and effectiveness of treatments. It is recommended to combine multiple outcome methods including a clinical score, quality of life questionnaires and patient-relevant benefit assessment (PBI-AR).
This study was supported by a research grant from Essex Pharma, Munich, Germany.
- 8Methoden. Version 2.0 vom 19.12.2006. Cologne: Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (DE), 2007 (German)., , , , , et al.
- 9Bundesministerium für Gesundheit und Soziale Sicherung (DE). Bekanntmachung [1883 A] eines Beschlusses des Gemeinsamen Bundesausschusses über eine Verfahrensordnung vom 20.September 2005 [Federal Joint Committee Rules of procedure]. Bundesanzeiger 2005;242:16998 (German).
- 10Bundesministerium für Gesundheit (DE). Gesetz zur Stärkung des Wettbewerbs in der gesetzlichen Krankenversicherung (GKV-Wettbewerbsstärkungsgesetz – GKV-WSG) vom 26.03.2007 [Law to strengthen competition in statutory health insurance]. Bundesanzeiger 2007;66:3613 (German).
- 13Measuring disease: a review of disease-specific quality of life measurement scales. Buckingham: Open University Press, 1995..
- 14Testaufbau und Testanalyse. Weinheim: Beltz, 1998., .
- 15Measuring health: a practical approach. Chichester: John Wiley, 1988..
- 20Quality of life measures in health care and medical ethics. In: SpilkerB, editor. Quality of life and pharmacoeconomics in clinical trials. Philadelphia: Lippincott-Raven, 1996:487–510..
- 21Economic evaluation in health care. Oxford: Oxford University Press, 2001., .
- 22Integrating economics evaluations and quality of life assessments. In: SpilkerB, editor. Quality of life and pharmacoeconomics in clinical trials, 2nd edn. Philadelphia: Lippincott-Raven, 1996:85–95., , .
- 25Ergebnisqualität psychomatischer Rehabilitaion: Zielerreichungsskalierung auf der Basis einer strukturierten Therapiezielliste [Outcome quality of psychosomatic rehabilitation: goal attainment scaling on the basis of a structured catalogue of therapy goals]. Rehabilitation (Stuttg) 2004;43:219–232 (German)., , , , , .