Validation of the ISAAC video questionnaire (AVQ3.0) in adolescents from a mixed ethnic background
Gibson Airway Research Centre, Department of Respiratory Medicine, John Hunter Hospital, Newcastle, Locked Bag 1, Hunter Mail Centre NSW 2310, Australia.
The International Study of Asthma and Allergies in Childhood (ISAAC) has developed an international version of the asthma video questionnaire (AVQ3.0) to measure asthma prevalence. This questionnaire has not been validated in adolescents from a mixed ethnic background.
The aims of this study were to compare the video questionnaire with a written questionnaire in the detection of airway hyperresponsiveness to hypertonic saline in a population of adolescents from a mixed ethnic background, and to establish the repeatability and psychometric properties of the asthma video questionnaire.
The study was conducted in four secondary schools in Sydney, an area with a high proportion of people from a non-English speaking background. Four hundred and seventy-five students from schools 1 and 2 completed the video questionnaire and a subgroup of these students (n = 170) completed the written questionnaire and a hypertonic saline inhalation challenge. Reproducibility of the questionnaire was evaluated by administering the questionnaire to a subsample of students 2 weeks later. The psychometric properties of the video questionnaire were examined in 852 students at two other schools (schools 3 and 4).
One hundred and sixty-nine students aged 13.5 (sd 1.3) years completed both written and video questionnaires, and the hypertonic saline challenge. The students had widely different cultural backgrounds including Asian, South Pacific, Middle Eastern, European and African countries. There was good agreement between the questionnaires for wheeze (kappa 0.42). Questions on the video questionnaire concerning wheezing had good sensitivity (90%) and specificity (68%) for airway hyperresponsiveness to hypertonic saline. The video questionnaire was reproducible (kappa 0.82), had good internal consistency (Cronbach's alpha 0.81) and each question pertained to a single construct explaining 58% of the variance in total score.
This study has validated the international version of the ISAAC video questionnaire against airway hyperresponsiveness to hypertonic saline in adolescents from a mixed ethnic background, and identified that the questionnaire has good psychometric properties. The ISAAC video has proved to be a valuable tool for the assessment of asthma prevalence in populations of ethnic diversity.
Asthma is now recognized as a major public health issue because of the reported increases in prevalence, morbidity and mortality [ 1]. The International Study of Asthma and Allergies in Childhood (ISAAC) is investigating asthma prevalence in more than 40 countries [ 2, 3]. This important study will define geographical variation in asthma prevalence and permit exploration of the reasons for this [ 1]. A key component of ISAAC is the accurate assessment of asthma prevalence in children from differing language and cultural groups. Valid comparison between countries requires rigorous evaluation tools. Epidemiological surveys of asthma rely heavily on self-administered written questionnaires. However, a translated written questionnaire may not have the same precision as the original questionnaire. In order to overcome the potential problems of imprecise translation and vocabulary differences between countries that occur with written questionnaires [ 4, 5], a video questionnaire has been developed which depicts five scenes of young people manifesting the clinical features of asthma (wheeze, night cough, exercise dyspnoea) [ 6].
The earlier European versions of this questionnaire (AVQ1.0 and AVQ2.0) used only Caucasian people. These versions were found to be more reproducible, and have equivalent sensitivity and specificity in predicting airway hyperresponsiveness (AHR) to methacholine than a written questionnaire when administered to adolescents from a Caucasian background [ 7, 8]. An international version of the ISAAC video questionnaire has been developed which depicts young people from different ethnic origins (AVQ3.0). This has been validated in Chinese school children against methacholine AHR [ 9].
Hypertonic saline challenge is a useful clinical and epidemiological tool for the assessment of airway responsiveness in asthma [ 10, 11]. It differs from methacholine in that it is a nonpharmacological indirect stimulus to airway narrowing and has been shown to correlate with airway inflammation in an epidemiological setting [ 10]. AHR to hypertonic saline is highly specific for asthma and can be satisfactorily performed in children in a school setting [ 10–12]. The relationship of the ISAAC video questionnaire to AHR to hypertonic saline has not been examined previously, and the validity of the international version of the questionnaire in different ethnic groups has not been established. Therefore, the aims of this study were to compare the ISAAC video questionnaire (AVQ3.0) with a written questionnaire in the detection of airway hyperresponsiveness to hypertonic saline in a population of adolescents from a mixed ethnic background.
The study was conducted in secondary schools (n = 4) in Western Sydney. This area has a high proportion (up to 90%) of the population from a non-English speaking background. Two schools (schools 1 and 2) were used to evaluate agreement between written and video questionnaires, and the relationship of the video questionnaire to airway hyperresponsiveness. Two other schools (schools 3 and 4) were used to evaluate the psychometric properties of the video questionnaire. In school 1 a subsample of children participated on a reproducibility study of the questionnaire. At schools 1 and 2, four hundred and seventy-five students attending Years 7, 8 and 9 completed the ISAAC AVQ3.0 during a school period. Approximately 1 week later, a subgroup of these students completed the ISAAC questionnaire, an asthma medication questionnaire, and underwent spirometry and hypertonic saline inhalation challenge. Bronchial challenge testing was performed on a random selection of students responding negatively to all five questions concerning wheezing in the last 12 months, and all of those who responded positively to at least one of the video sequences concerning wheezing in the last 12 months. Reproducibility of the questionnaire was evaluated in school 2 by administering the questionnaire to 36 students, and repeating this 1 week later.
The psychometric properties of the AVQ3.0 were examined following administration of the questionnaire to 852 high school students at two high schools (schools 3 and 4). Students completed the AVQ3.0 during a single class where project staff read out the instructions to the students and distributed an instruction sheet and the written component of the questionnaire. The video was screened and the students completed the questionnaire. Permission to conduct research was obtained from the Department of Schools Education, the school principals, and the Hunter Area Health Service and University of Newcastle Research Ethics Committees. Written informed consent was obtained from parents and students.
ISAAC International Video Questionnaire (AVQ3.0)
The ISAAC video questionnaire (AVQ3.0) consists of five video sequences of young people with different asthma symptoms. The first three sequences show various scenes of wheezing while the final two sequences display other asthma symptoms. The scenes depicted:
(1) Moderate wheezing at rest (a Caucasian girl)
(2) Wheezing after exercise (a Maori boy)
(3) Waking at night with wheezing (a Caucasian girl)
(4) Waking at night with coughing (an Asian boy)
(5) A severe attack of asthma with wheezing and breathlessness at rest (an Indian woman)
After each video sequence, students recorded whether their breathing had ever been like that shown in the video and if so, the frequency of such symptoms (past month, past year, ever), using a one-page printed answers sheet. The video questionnaire took about 7 minutes to administer, and the term ‘asthma’ was not mentioned during this time.
ISAAC Written Questionnaire (WQ)
The ISAAC WQon asthma comprises five questions that correspond to the five sequences depicted in the video questionnaire [ 6]. The questions are based on a standard asthma prevalence questionnaire [ 4] and prior ISAAC survey instruments. The questions are:
1 Have you had wheezing or whistling in the chest in the last 12 months?
2 In the last 12 months, has your chest sounded wheezy during or after exercise?
3 In the last 12 months, how often, on average has your sleep been disturbed due to wheezing?
Never woken with wheezing [ ]
Less than one night per week [ ]
One or more nights per week [ ]
4 In the last 12 months, have you had a dry cough at night, apart from a cough associated with a cold or chest infection?
5 In the last 12 months, has wheezing ever been severe enough to limit your speech to only one or two words at a time between breaths?
An additional question was asked: ‘Have you ever had asthma?’. The written questionnaire was presented to students in English by an ethnic health care worker who was based at the school. The child was asked to complete the questionnaire and to ask for clarification from the ethnic worker if required. When assistance was requested by the student this was provided in both English and in the native language of the student.
A poster board containing actual asthma medications devices was prepared. Each student was asked if they were using any of these medications and if so, to indicate which medication and how often it was used.
Hypertonic saline challenge
The height and weight of each student was measured, followed by baseline spirometry. Students with FEV1 values > 80% of predicted underwent hypertonic saline challenge [ 10–12]. Hypertonic saline (4.5%) was inhaled for doubling time periods (30 secs, 1 min, 2 min, 4 min) from either a Timeter® MP500 ultrasonic nebuliser (Oregon, Pike, PA, USA) or an ULTRA-NEB® DeVilbiss 2000. The equipment included the use of 23 cm of corrugated tubing and a Hans Rudolph 2700 2-way, nonrebreathing valve box (Hans Rudolph Inc, Kansas City, USA) with rubber mouthpiece, and nose clip. FEV1 was measured in duplicate 60 s after each saline dose. Between each dose of saline the students were encouraged to expectorate into a collection container. The test was stopped if the FEV1 fell by greater than 20% from baseline, if 15.5 cumulative minutes of nebulization time elapsed, or the student requested to finish. If the FEV1 fell by greater than 20% during the challenge, salbutamol 200 μg was administered using a pressurized metered dose inhaler and valved holding chamber (Breath-A-Tech, Scott-Dibben, Australia). The dose of 4.5% saline delivered was assessed by weighing the nebuliser cup, tubing and valve-box, before and after each challenge. Airway hyperresponsiveness was defined as a PD20 < 15 mL.
Agreement between questionnaires was estimated using the kappa statistic. The sensitivity and specificity (with 95% CI) of questionnaire responses for predicting AHR for each question, and for combinations of the questions in the video questionnaire were determined. The validity of these questions was determined by Youden's Index, which was calculated by adding the sensitivity and specificity together and then subtracting one from the sum [ 13]. A Student's t-test was used to compare data from the two questionnaires, with significance accepted at the P < 0.05 level. The psychometric properties of the AVQ3.0 were analysed using STATA for windows V4.0. Construct validity was assessed using principal components (factor) analysis which examines whether the questionnaire items are correlated with each other in conceptually meaningful groupings. This is numerically assessed by the amount of variance explained by the factor groupings and by the eigen value of each factor, with eigen values greater than 1.0 indicating a relevant construct. Internal consistency was assessed using item analysis and Cronbach's alpha.
Assessment of AHR
One hundred and seventy students attended for testing and 169 completed the hypertonic saline challenge. One student withdrew after the first dose of hypertonic saline because of throat discomfort and apprehension. The subjects were aged 13.5 (SD 1.3) years and 67 (39%) were male. The majority of students (94.5%) were from a non-English speaking background. Sixty-eight (40%) of the students came from homes where English was the only language. Seventy-one (42%) came from homes where no English was spoken, while 23 (14%) reported that English as well as other languages were spoken at home. The range of languages is displayed in Table 1. The students were from widely different cultural backgrounds, including Asian, South Pacific, Middle Eastern, European and African countries. The duration of residence in Australia had a bimodal distribution. One group had lived in Australia for 0–2 years, and another for between 12 and 15 years. Fifty-two (31%) had previously been diagnosed by a doctor as having asthma, and 31(18%) reported using asthma medications. β2-agonists were used by 20 (12%) students and inhaled corticosteroids by 13 (8%). Twenty (12.2%) of the students had AHR to 4.5% saline.
Table 1. .
Language background of students: languages spoken at home
Agreement between written and video questionnaires
There was agreement between responses to the video and written questionnaires in 60% to 70% of subjects for each question. Each question demonstrated significant agreement beyond chance ( Table 2). The best agreement was for the ‘wheeze at rest’ and ‘wheeze during exercise’ questions that had kappa values of 0.41 and 0.42, respectively. Agreement was comparatively poor for dyspnoea at rest and nocturnal cough and wheeze, with kappa values of 0.21, 0.25, and 0.28, respectively. The main sources of disagreement occurred where there were positive responses to the written questionnaire, but negative responses to seeing the symptom depicted in video format.
Table 2. .
Agreement between Written and Video Questionnaires n
< 0.05 indicating significant agreement beyond chance.
The responses to individual questions from the video and written questionnaires were compared to airway responsiveness to hypertonic saline and the results were summarized as sensitivity, specificity and Youden's Index ( Table 3). There were no significant differences between the corresponding written and video questions in their association with airway hyperresponsiveness (P > 0.05). A history of ‘ever wheezing at rest’ had the highest Youden's Index (0.58), with a sensitivity of 90% and a specificity of 67.8% on the video questionnaire. The three questions concerning wheezing had the best test characteristics in terms of sensitivity and Youden's Index. Combinations of positive responses did not perform any better than single questions alone ( Table 4). While specificity improved by combining questions, there was a reduction in sensitivity and consequently Youden's Index did not increase.
Table 3. .
Sensitivity, specificity and Youden's Index with 95% confidence intervals for airway hyperresponsiveness of the ISAAC International Video Questionnaire (AVQ3.0) and the ISAAC Written Questionnaire (WQ). Symptoms in the past 12 months
Table 4. .
Sensitivity, specificity and Youden's Index intervals for airway hyperresponsiveness of combinations of ISAAC Video Questionnaire (AVQ3.0) Symptoms in the past 12 months* *Rest — indicates wheeze at rest.
The validity of the written and video questionnaires was also assessed in the 69 children reporting English-only as their spoken language. The sensitivity and specificity of recent wheeze (past 12 months) were 50% (31,69) and 74% (61,87), respectively, for the video questionnaire. For the written questionnaire, the sensitivity and specificity of recent wheeze were 65% (47,83) and 63% (49,78), respectively. For the remaining students, the sensitivity and specificity of the video questionnaire were 29% (11,47) and 73% (63,84), whereas the sensitivity and specificity of the written questionnaire were 29% (11,47), and 73% (63,84). Both the video and written questionnaires tended to have reduced sensitivity for AHR in the NESB students, however, the confidence intervals were overlapping indicating that the values were not statistically significant.
Each of the questions in the AVQ3.0 had moderate to good reproducibility, that was significantly greater than chance (P < 0.001). The kappa values for questions 1–5 were: 0.82, 0.92, 0.80, 0.63, 0.91, respectively.
Five hundred and eight (508) students from Years 7, 8 and 10 in School 3, and 344 students from years 7, 8, 9, 10 and 11 in School 4 completed the AVQ3.0. The response rate was 98%. Fifty-six percent (56%) and 27% of the students reported that a language other than English was spoken at home in School 3 and School 4, respectively. The prevalence of asthma was approximately 19% in both schools. Other than language spoken at home, both the schools were similar in demographic characteristics. Internal consistency of the video questionnaire was excellent with an overall Cronbach's alpha value of 0.81. Principal component analysis for construct validation revealed that the five questions belonged to a single factor (eigen value 2.91), termed ‘asthma’ which explained 58.1% of the variance in the total questionnaire score.
The novel aspects of this study are the validation of the international version of the ISAAC video questionnaire against airway hyperresponsiveness to hypertonic saline, and the description of the questionnaire's psychometric properties in adolescents for a mixed ethnic background. There was good agreement between the written and video ISAAC questionnaires, and that the AVQ3.0 was equivalent to the ISAAC written questionnaire in its ability to predict airway hyperresponsiveness to hypertonic saline. The AVQ3.0 had excellent psychometric properties.
The ISAAC video has proved to be a valuable tool for the assessment of asthma prevalence [ 3]. The questionnaire has been shown to validly detect symptomatic asthma in homogenous populations of Caucasians [ 7] and Chinese school children [ 9]. The population in Western Sydney provides a further opportunity to test the validity of the ISAAC video questionnaire. Successive periods of migration by people from different parts of the world to Australia has resulted in a population of diverse ethnic composition. Over half of the school children came from homes where English was not the primary language, and 32 different languages were spoken. Despite this marked ethnic diversity, the video questionnaire performed well as an instrument to assess asthma prevalence. This is an important observation since some migrating populations experienced rising asthma prevalence with increasing residence in Australia [ 14]. These results support the use of the video questionnaire in future aims of ISAAC, namely, identifying geographical and population characteristics that may explain the increase in asthma prevalence seen in Western countries [ 1].
The ISAAC written questionnaire has previously been evaluated against AHR to hypertonic saline and found to be sensitive and specific [ 15]. Similar results were obtained in this study for the ISAAC video questionnaire. Airway hyperresponsiveness to hypertonic saline is a good marker for asthma since it is highly specific and closely correlated to airway inflammation [ 10–12, 15].
In conclusion, this study demonstrated that the international version of the ISAAC video questionnaire, AVQ3.0, is a valid and reliable tool for the assessment of asthma prevalence and symptoms in a population of diverse ethnic background.
We would like to thank Professor Richard Beasley, the Wellington Asthma Research Group and the ISAAC study team for providing the video questionnaire and funding to conduct this study, Professor A Bauman and Dr H Mamoon for assistance with factor analysis and Kellie Fakes, Fiona Collier, Robyn Hankin, Hassan Mamoon, and Anne Stuckey for assistance with data collection. We would like to acknowledge the cooperation of the students and teachers of Holroyd, Granville South, Christian Community and Trinity High Schools, and the assistance provided by the school principals Ms Dorothy Hoddinott and Mr Ron Miles.