Clinical & Experimental Allergy

Validation of the Chinese translated version of ISAAC core questions for atopic eczema

Authors


Christopher K.W. Lai, Room 1403 Takshing House, 20 Des Voeux Road, Central, Hong Kong. E-mail:: keilai@netvigator.com

Abstract

Background The International Study of Asthma and Allergies in Childhood (ISAAC) was designed to allow international comparison of epidemiological data on atopic conditions in childhood. In so doing, further aetiological information would be obtained that in turn would provide a framework for future studies. The global ISAAC results on the prevalence of atopic dermatitis indicated a 60-fold variation recorded in different countries. Such a degree of difference may be partially due to the translated questionnaires that were not validated in all of the involved countries.

Objective To validate the Chinese version of the ISAAC core questions for atopic eczema.

Methods One thousand nine hundred and twenty children aged between 3 and 5 were randomly recruited from 13 kindergartens in Hong Kong. Using a dermatologist's clinical examination as the gold standard, we validated the Chinese version of the ISAAC core questions for atopic eczema. The Youden's Indexes obtained in our study were compared with those obtained in the United Kingdom's validation study.

Results The Youden's Indexes obtained in our study were significantly lower than those from the United Kingdom. The low scores were likely to be due to a reduction in the sensitivity of the Chinese questionnaire, which ranged from 23.5% to 70.6%.

Conclusion Our findings indicate that the translated questionnaire is less effective than the English version in assessing the prevalence of atopic eczema. The indication of a low prevalence of atopic eczema among the Chinese population reported in previous studies was at least partially due to problems with the translated questionnaire.

Introduction

The International Study of Asthma and Allergies in Childhood (ISAAC) was designed to allow international comparison of the epidemiological data on asthma and other atopic conditions in childhood. It is divided into three phases. The aim of Phase 1 of the ISAAC study was to use a standardized method to identify large relative prevalence differences in different countries. This in turn would provide a framework for further aetiological research (Phases 2 and 3 of the ISAAC study). The results on the global prevalence of atopic eczema have just been published [1,2]. Over 68 000 children from 56 countries were involved in the study. A 60-fold variation in the 12-month prevalence was recorded in different centres. Such degree of variation may suggest significant differences in the aetiology of atopic eczema among populations. Another important factor that may have contributed to this degree of variation is the study design. Despite using a standardized protocol, cultural and linguistic differences among populations could affect the outcome [1]. This particularly applies to the English questionnaire, which was translated by bilingual translators into 39 different languages. In some of these versions, the terms used in the translated questionnaire may not be well understood, and validation of these translated questionnaires is therefore necessary. We present our findings on the validation of the Chinese version of ISAAC core questions on atopic eczema.

Methods

One thousand nine hundred and twenty children aged between 3 and 5 were randomly recruited from 13 kindergartens in Hong Kong, which were randomly chosen from a total of 666. The kindergartens were located in different areas (four from Hong Kong island, three from Kowloon and six from the New Territories) and truly represented the social and economic status of Hong Kong children. All children involved were ethnic Chinese and their parents could read and understand Chinese. The education levels of their parents were: 6% primary school; 19.8% lower secondary school; 39.3% upper secondary school; and 39.3% tertiary institution. Their children's mean age was 3.99 and the male : female ratio was 0.9 : 1. Written parental consent was obtained prior to the study, and The Ethical Committee of the Chinese University of Hong Kong gave its approval.

The ISAAC questionnaire has previously been described in detail [3]. Seven questions were used to assess the prevalence and severity of atopic dermatitis:

1 Has your child ever had an itchy rash, which came and went for at least 6 months? If no, please skip to question 7.

2 Has your child had this itchy rash at any time in the last 12 months? If no, please skip to question 7.

3 Has this itchy rash at any time affected any of the following areas: the folds of the elbows, behind the knees, the front of the ankles, under the buttocks, or around the neck, ears or eyes?

4 At what age did this itchy rash first occur? Under 2, age 2–4, age 5 or above?

5 Has this rash cleared completely at any time during the last 12 months?

6 In the last 12 months, how often, on average, has your child been kept awake at night by this itchy rash? Never, 1 night per week, 1 or more nights per week?

7 Has your child ever had eczema?

The questionnaire was translated into Chinese following the guidelines provided by the ISAAC steering committee [4]. In brief, a bilingual translator translated the English questionnaire into Chinese, which was subsequently back-translated into English by another translator. Together with the consent form, kindergarten staff distributed the questionnaire to all the parents. A follow-up letter was sent if they did not respond within 2 weeks.

Two weeks later, in the usual manner, a dermatologist with an interest in paediatric dermatology visited the school to perform clinical examinations and look for signs of atopic dermatitis. Children were examined in a private room and were free to refuse the examination regardless of parental consent. The child's family was informed if any skin disease was found and offered further consultation at the Prince of Wales Hospital.

Statistical analysis

The completed questionnaires were coded and entered into a database (Dbase 5.0 for DOS). All statistical analyses were carried out by SPSS Version 8.0. Results were expressed in terms of sensitivity, specificity, positive and negative predictive values and relative value (Youden's Index).

Results

Of the 1920 children recruited, 1570 were given consent for both the questionnaire and clinical examination. There were no significant differences between the responders and non-responders in terms of age and gender. Parental education level was significantly higher among the responder group than in the non-responder group (chi square P = 0.017 and 0.008 for paternal and maternal education levels, respectively). The questionnaire was validated using the dermatologist's diagnosis as the gold standard (see Table 1). Our results are compared with those of the UK validation study [5]. The point prevalence for atopic dermatitis in our local population of 3–5-year-olds was 7.6%.

Table 1.  Number of positive responses, sensitivity, specificity, Youden's index, positive predictive value and negative predictive value of each of the seven questions relating to eczema symptoms

Questions
No. of positive
responses
Sensitivity
(%)
Specificity
(%)
Youden's
index
Positive predictive
value (%)
Negative predictive
value (%)
Rash ever22355.588.90.44429.695.9
Rash in the last 12 months19854.690.60.45232.895.9
Flexural areas17149.692.10.41734.595.6
Rash cleared11241.295.50.36743.895.1
Sleep disturbance10132.895.60.28438.694.4
Age of onset (< 2 y)7723.596.50.20036.493.7
Eczema ever55570.666.60.37215.196.4

Discussion

Epidemiological studies are important, as by identifying population differences, the aetiology of atopic eczema can be further understood. Although many studies have looked at the prevalence of atopic eczema, there are few large-scale multicentre studies that have used the same methodology to examine population differences in childhood atopic eczema [6–9]. ISAAC is the only international study that, in a standardized manner, attempted to look at the global pattern of the prevalence of atopic diseases. The world-wide variation in the prevalence of atopic eczema and other atopic diseases has just been published. The findings of the ISAAC steering committee indicated a 60-fold variation in the prevalence of atopic eczema among countries [1,2]. This large degree of variation may indicate a real difference in the aetiological factors of atopic eczema globally. Another important factor, which would at least partially contribute, is the validity of the written questionnaire. If the translated questionnaires were found to be invalid, then the aim of Phase 1 of the ISAAC study would have been compromised. Validation of the different languages' written questionnaires is therefore necessary.

Using clinical examination as the gold standard, Williams et al. validated the ISAAC core questions for atopic eczema [4,5]. They showed that the sensitivity of the questions ranged from 47% to 80%, whereas specificity ranged from 84% to 97%. Our data indicated that the Youden Indices of the Chinese questionnaire are significantly lower than those of the UK version, suggesting that the translated questionnaire is less effective in assessing the prevalence of atopic eczema. The low Youden Index scores are likely to be due to a reduction in the sensitivity of the Chinese questionnaire, which ranged from 23.5% to 70.6%. A lower degree of sensitivity implies that underestimation of prevalence could occur. This would partially explain the low 12-month prevalence of atopic eczema symptoms observed in ISAAC studies (13–14 years) conducted in Hong Kong (2.7%) and China (0.8% to 1.6% depending upon the region) [1]. Interestingly, in Singapore in a study involving predominately Chinese children aged 13–14 years, where the English version of the ISAAC questionnaire was used, the 12-month prevalence of atopic eczema was much higher (7.4%) [1].

Validation studies of the UK diagnostic criteria for atopic eczema have been performed in several other countries. Their findings are summarized in Table 2. Using the same study protocol as the UK validation study, Popescu et al. examined the validity of the UK diagnostic criteria for atopic eczema in Romanian schoolchildren and found that the sensitivity and specificity of the questionnaires were 74% and 99%, respectively [10]. They concluded that the diagnostic criteria were applicable in settings outside the UK. However, findings by most others groups did not support such a view. Indeed, a low sensitivity of the ISAAC core questions seems to be a common feature.

Table 2.  Validation studies of the UK diagnostic criteria for atopic eczema
StudySubjectsSensitivity/
nAge (yr)specificity (%)
Williams et al. 19966953–1180/97
Popescu et al. 199811146–1274/99
Marks et al. 199924914–1843/95
Firooz et al. 1999416< 4, 4–10, > 1010/98.3
Kramer et al. 19981511651/93.6

Kramer et al. looked at the influence of cultural and educational factors on the validity of the ISAAC questions for atopic eczema [11]. They performed a cross-sectional study of 1511 6-year-old children in Germany and found that sensitivity ranged from 31.1% to 68.9%. Interestingly, the sensitivity of the ‘itchy rash of more than 6 months duration’ question was only 51.1%, suggesting that almost half of the patients with atopic eczema did not complain or have the itchy skin condition noticed by their parents. Translation error was thought to be the reason for this low sensitivity. The Australian group identified a similar degree of sensitivity (43%) [12].

More recently, Firooz et al. validated the UK diagnostic criteria in Iran [13]. They performed a cross-sectional study looking at 416 patients (60 atopic eczema patients and 356 with other skin diseases) in a private dermatology clinic, and concluded that the UK diagnostic criteria for atopic dermatitis were not sensitive enough to be used for large population-based epidemiological studies. In an editorial article, William raised several factors that could have affected the outcome of Firooz et al.’s study, including the following [14]:

• The criteria questions and examination protocol might not have been used as recommended.

• The translation might not have followed a strict format.

• The criteria were not tested in the appropriate setting (population based).

• The assessor might not have been blinded.

• The dermatologist's gold standard was not compared with others.

The first three factors would not apply to our study as we strictly followed the ISAAC protocol. The fourth factor is difficult to overcome in any validation study, as the UK diagnostic criteria are well publicized and most dermatologists are now aware of them. The final factor also applies to our study and, while testing for diagnostic consistency is important, we doubt that it would significantly alter our results. A resulting low sensitivity when the ISAAC questions were translated into Chinese is not unique to the eczema questions. Our co-authors (CKWL, GW) were involved in comparing the ISAAC video (AVQ3.0) and written questionnaires, for the purpose of estimating asthma-associated bronchial hyper-reactivity [15]. They found that with the exception of the ‘asthma ever’ question, all the other five questions had a sensitivity ranging from 13% to 69%. The reason for this low degree of sensitivity is not yet established, but cultural and linguistic differences are likely to be important.

One of the major problems in validation studies is that one is validating a questionnaire designed to assess period prevalence against point prevalence, which in our case is the dermatologist's diagnosis at one point in time. For atopic eczema, a disease characterized by its fluctuating course, the period prevalence could be much greater than the point prevalence. This also applies to our study, which showed that while the point prevalence by clinical examination was 7.6%, the period prevalence (rash in the flexural area) was 10.8%, giving a systematic error (ratio of the prevalence of the test compared with the prevalence of the examination) of 1.4. Given the fact that underestimation is likely to have occurred in our study, the period prevalence could be several times greater than the point prevalence. As it would be difficult to validate the questionnaire by period prevalence (examination by a dermatologist at different times over a 12-month period), there is no realistic way to avoid this design fault. A number of dermatologists assessing the same children or a subgroup of the whole cohort would help to minimize this error. However, in our study it is important to take systemic error into consideration during data interpretation. For example, our results indicated the period prevalence to be much higher than the point prevalence, creating the false impression that the questionnaire overestimated rather than underestimated the prevalence of atopic eczema in Hong Kong.

Popescu et al. performed a similar community validation study in Romanian children and found that the systemic error of prevalence estimated for atopic dermatitis tends to be inversely related to the true prevalence [10]. As a result, they concluded that the UK diagnostic criteria (which the ISAAC atopic dermatitis core questions are based upon) will overestimate if the true prevalence is below 2% and underestimate if it is above 10%. Their findings seem to contradict our statement of underestimation, given the fact that our true prevalence was 7.6%. In order to validate the questionnaire as a measure of point prevalence, Popescu et al. used symptoms that were present in the last 2 weeks instead of 12 months. As a result, their questionnaire was assessing a much shorter period prevalence (2 weeks) than the ISAAC core questions (12 months). Therefore, their data could not be applicable to our study.

The low sensitivity of the questionnaire could be partially explained by this design fault. This particularly applies to the ‘rash in the last 12 months’ question, as the rash could be absent at the time of examination. However, translation error or cultural difference must still be responsible for this low sensitivity. This is evidenced by the fact that the sensitivity of the ‘rash ever’ question was only 56% (compared to 86% in the UK study and 77.7% in the Romanian study), suggesting that almost half of the children diagnosed by the dermatologist as having atopic eczema had never had a rash. This also applies to the question regarding flexural rash, a hallmark of atopic eczema, which only carries a sensitivity of 49% (compared to 76% in the UK study and 74% in the Romanian study). The reason for this is unknown, but several factors may be responsible.

In Hong Kong, it is common practice for upper- and middle-class families to employ a foreign maid to look after their children, while both parents work full-time. Unfortunately in some cases the maid is more aware of the well-being of the children than the parents. In the case of significant eruption, the parents would notice or be informed by the maid. However, for milder cases the parents may not be aware or may not be properly informed. Furthermore, admitting that the child had or has a rash may be considered by some parents to be a form of ‘child neglect’ on their part. Indeed, some might even consider the rash in the flexural area as a form of dirty skin. Cultural factors such as these could explain why the local population was so reluctant to admit to a rash, leading to a low response rate and a subsequent low sensitivity.

Translation and educational factors are also important determining factors. In the German study a translation difficulty leading to ‘rash continuously present for 6 months’ being confused with ‘rash coming and going for at least 6 months’ might have been responsible for the low sensitivity of the question ‘itchy rash of more than 6 months duration’[11]. Furthermore, German families with lower educational levels were associated with lower rates of positive response. The translation of the Chinese version of the questionnaire followed ISAAC guidelines and we did not detect a similar type of translation fault. In Hong Kong, while the educational standard should allow most parents to understand the translated questionnaire, difficulty could arise for parents that are new immigrants from mainland China. Since the cultural revolution, mainland China has adopted the use of a simplified version of the written Chinese language. The mainland Chinese are taught to use a simplified form of the Chinese character that differs significantly from the traditional type used in the questionnaire. Our questionnaire did not allow accurate identification of the exact numbers of families that migrated from the mainland. Information from the headmasters of the kindergartens indicated a wide degree of variation that ranged from 1% to 30% depending upon the location of the kindergarten.

The key issue that needs to be addressed is whether the aim of Phase 1 of the ISAAC study has been compromised. Its aim was not to accurately assess the prevalence of atopic diseases in each country or city, but to identify patterns of prevalence differences so that further aetiological research could be carried out. Our findings indicate that the questionnaire has limitations and could compromise such an aim. One possible means to avoid linguistic or cultural problems is to use an objective criterion, such as visible flexural dermatitis, to serve as the crude tool in assessing the prevalence of atopic dermatitis. This is the approach currently used in Phase 2 of the ISAAC study.

Our data on the prevalence of atopic disease as compared with older children (aged 6–7 and 13–14 years) [16,17] within our population will be published separately. Such a comparison allows us to obtain further information regarding the aetiology and future trends of these atopic disorders.

In conclusion, we have validated the Chinese version of ISAAC core questions for atopic eczema. We find that the questions are less effective than the English version in assessing the prevalence of atopic eczema. A low prevalence of atopic eczema previously reported in other studies among Chinese populations is at least partially due to the low sensitivity of the translated questionnaire.

Ancillary