SEARCH

SEARCH BY CITATION

Keywords:

  • Cronbach's alpha;
  • General Health Questionnaire;
  • internal consistency;
  • intraclass correlation coefficient;
  • test–retest reliability and validity

Abstract

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. ACKNOWLEDGMENTS
  9. REFERENCES

Abstract This present study was undertaken to validate the English version of the General Health Questionnaire (GHQ-12) in urological patients. Validity and reliability were studied in patients with lower urinary tract symptoms (LUTS) and patients without LUTS. Reliability was evaluated using the test–retest method and internal consistency was assessed using Cronbach's alpha. Sensitivity to change was expressed as the effect size in the pre-intervention versus post-intervention score in additional patients with benign prostatic hyperplasia (BPH) who underwent transurethral resection of the prostate (TURP). Internal consistency was excellent. A high degree of internal consistency was observed for each of the 12 items with Cronbach's alpha value of 0.37–0.79, while total scores was 0.79 in the population study. Test–retest correlation coefficient for the 12 items score were highly significant. Intraclass correlation coefficient was high (0.35–0.79). It showed a high degree of sensitivity and specificity to the effects of treatment. A high degree of significant level between baseline and post-treatment scores were observed across all 12 items in the treatment cohort but not in the control group. The GHQ-12 is suitable, reliable, valid and sensitive to clinical change in urological disorders.


INTRODUCTION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. ACKNOWLEDGMENTS
  9. REFERENCES

The 12-item General Health Questionnaire (GHQ-12)1 has become a commonly used instrument in multicenter, international clinical trials designed and widely used to detect non-psychotic psychiatric disorders. The GHQ-12 is an instrument to identify states of depression and psychiatric morbidity. It is aimed at detecting psychiatric disorders which may have relevance to a patient's presence in a medical clinic so its focus must be on psychological compounds of ill health.2 Its discriminating ability can be influenced by factors such as the presence of physical illness, comorbidity with other psychiatric disorders and the presence of cognitive impairment which are more frequent in the elderly.3 The presence of somatic symptoms, irrespective of etiology, was associated with increased social and psychiatric morbidity across disparate culture.4 The validation of GHQ-12 has been established.5

The GHQ-12 has been used to detect psychiatric disorders secondary to urological disorders, such as benign prostatic hyperplasia (BPH).6 Lower urinary tract symptoms (LUTS) from BPH is a common disorder, which is rarely life-threatening but may affect the patient's perceived quality of life (QOL) in psychological aspects such as anxiety, depression and psychiatric morbidity.6 It is well-known that LUTS can be very bothersome for patients and interference with daily activities and especially symptoms such as frequency, nocturia, urgency, urge incontinence and dribbling can psychologically affect the patient's quality of life.6–8

Existing self-report measures of psychiatric disorders such as the General Health Questionnaire (GHQ-12) has been widely used in Western countries and therefore need to be validated for the local population. The GHQ-12 developed by Goldberg is a multidimensional tool and has been used widely in the studies of psychiatric morbidity in many countries both in the community-based study as well as the clinical studies. The 12-item of GHQ-12 is easy to administer, simple, short and requires less than 7 min to be completed by respondents.1

The present study was conducted at University Hospital, Kuala Lumpur, Malaysia, and was designed to assess the reliability and validity of the GHQ-12 among urological patients.

PATIENTS AND METHODS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. ACKNOWLEDGMENTS
  9. REFERENCES

Study sample

The study was carried out by the Health Research Development Unit and Department of Surgery, University Hospital, Kuala Lumpur, Malaysia. The psychometric properties of the GHQ-12 were assessed in three different groups: patients with LUTS undergoing surgical treatment (transurethral resection of the prostate, TURP) (n = 79), patients with LUTS undergoing medical treatment (n = 108) and control group of patients (renal stones) (n = 50). Validity and reliability were studied in a group of patients with LUTS (medical group) and the control group of patients with renal stones, and sensitivity of change was assessed in a group of patients who were diagnosed with LUTS admitted for TURP. Management decisions was entirely done by the urologist (AHR) in all cases based on clinical criteria such as medical history, physical and rectal examinations.

The patients were selected based on the inclusion and exclusion criteria set in the present study. Patients with LUTS were chosen because it was found that LUTS contributed to psychological effects among the patients. For patients with LUTS in the medical and surgical groups, the inclusion criteria were that patients must in stable clinical condition and literate (understand and able to answer) and able to give informed consent, while the exclusion criteria were patients who were treated with surgical and medical treatment for LUTS prior to this study. Patients less than 40-year-old who were unable to read, write or understand were excluded, as were patients with any major medical history and those who were physically disabled. For the control group of patients, the inclusion criteria included patients who were free from all major chronic and acute diseases and those with renal stones with minimal severity who did not seek treatment for LUTS, while the exclusion criteria were those seeking treatment for urological problems which included BPH and urinary tract infections (UTI). Renal stones patients were chosen as the control group in order to have a closer urologic condition for comparison. All patients in the surgical group (TURP) were patients who had been diagnosed from various indications, such as acute and chronic retention, recurrent UTI, severe symptoms, failure of medical treatment, and renal impairment secondary to bladder outlet obstruction. The study protocol was approved by the Ethics Committee, University Hospital Kuala Lumpur, Malaysia.

Data collection

All patients in the surgical group (TURP), medical group and control group of patients gave their written informed consent to participate in the study. Patients then completed the GHQ-12. All questionnaires were self-administered, although assistance was available from one of the authors (KFQ) to help if necessary. All patients included in the validity study were retested at 12 weeks after the first administration of the GHQ-12 (baseline). In the sensitivity to change study, patients completed the questionnaires 1 week before the surgical treatment and were retested at 3 months after TURP.

Data analysis

The internal consistency of the GHQ-12 was assessed by calculating the Cronbach's alpha coefficient.9 The internal consistency shows the resulting values of Cronbach's alpha for the scale when individual items are excluded from the analysis. Test–retest reliability was assessed using the intraclass correlation coefficient (ICC), which is derived from analysis of variance. Values of ICC vary from 1 (perfectly reliable) to 0 (totally unreliable).10 Sensitivity to change of the GHQ-12 was assessed in patients undergoing TURP. Sensitivity to change was analyzed by calculating the difference between GHQ-12 before and after TURP and dividing it by the standard deviation of the GHQ-12 before TURP (effect size).11

Mean differences in GHQ-12 scores before and after TURP was also calculated for each individual item by means of a paired t-test or by dividing it by the standard deviation of stable patients with LUTS before treatment (medical group) (Guyatt statistic).12

In treatment responsiveness, sensitivity and specificity of the instrument was evaluated by comparing the change between baseline and end-point scores following treatment. Sensitivity is the proportion of cases correctly identified, while specificity is the proportion of non-cases correctly identified. In the present study, sensitivity of the GHQ-12 was assessed by comparing the mean pre-treatment and post-treatment of patients who had undergone TURP, while specificity was assessed by comparing pre-treatment and post-treatment in subjects rated as control (renal stones).

RESULTS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. ACKNOWLEDGMENTS
  9. REFERENCES

A total of 237 respondents participated in the validity and reliability study. The age of the patients in the medical group was 63.67 ± 8.57 years, in the surgical group 70.01 ± 8.17 and in the control group 50.04 ± 12.29. In ethnicity, Chinese formed the largest ethnic groups in all three groups: medical (66.70%), surgical (75.90%) and control (58.00%), followed by Indian (28.70%, 22.80% and 42.00%) and others (4.60%, 1.30%, 0%). Patients with LUTS and higher GHQ-12 scores indicating psychiatric morbidity. Of the 237, 108 patients with LUTS had total scores of 10.39 ± 3.44, while 79 patients undergoing TURP has the total scores of 13.25 ± 4.56.

Internal consistency for the GHQ-12 was high for all the items of GHQ-12, indicating a high level of homogeneity among items in the scale. Test–retest reliability was assessed in 108 patients after a 12 week interval. The total GHQ-12 had an ICC of 0.79 (P < 0.001) and a Pearson's product moment correlation of 0.68 (P < 0.01) (Table 1).

Table 1.  Validity and Reliability: Mean test–retest score, intraclass correlation coefficient and internal consistency for individual General Health Questionnaire-12 items (medical group)
          95% CI
GHQ-12 itemsICC*Pearson's product moment correlation coefficient**Internal consistencyMean test scoreSDMean retest scoreSDMean difference§SDLowerUpper
  • *

    P < 0.001 for all ICCs;

  • **

    P < 0.01 for all Pearson's coefficient;

  • Cronbach's alpha value given for each item represents the effect of removing that item from from the calculation of the alpha value (e.g. if item 1 is omitted, the resulting value for the scale is 0.61, if item 2 is omitted, it is 0.60, etc);

  • §

    t-test for paired comparisons not significant.

  • ICC, Interclass correlation coefficient; SD, standard deviation; 95% CI, 95% confidence interval; GHQ-12 items, General Health Questionnaire-12 items.

10.600.440.611.190.441.110.420.0830.46–0.0030.17
20.600.440.600.880.780.780.660.100.77–0.040.25
30.700.530.691.060.331.060.3000.31–0.060.06
40.490.350.511.070.291.000.270.0740.330.010.14
50.730.590.740.730.610.780.570.0460.54–0.150.05
60.710.570.720.720.650.860.600.140.59–0.250.03
70.630.470.641.220.461.150.410.0740.45–0.010.16
80.350.230.371.070.380.980.330.0930.440.0010.18
90.710.550.710.730.730.680.640.0560.65–0.070.18
100.760.610.760.430.570.370.540.0560.49–0.040.15
110.620.450.620.280.510.220.460.0560.51–0.040.15
120.640.480.631.050.421.030.320.0190.39–0.050.09
Total scores0.790.660.7910.393.9110.043.440.353.08–0.230.93

Table 2 shows pre- and post-scores, mean difference, effect size, and the Guyatt statistic for individual items, and for global scores. The pre-intervention score on the total GHQ-12 was 13.25 ± 4.56, while the post-intervention score was 9.82 ± 3.04 (p < 0.0001), giving an average improvement after TURP on the psychiatric morbidity of 3.43. Overall effect size, and therefore sensitivity to change, was found to be high, which indicated TURP-induced urinary symptom improvement in these patients.

Table 2.  Sensitivity to change:mean scores before and after transurethral resection of the prostate (TURP), effect size and Guyatt statistic
 Pre-TURPPost-TURP
GHQ-12 itemsMeanSDMeanSDMean differences*SDEffect sizeGuyatt statistic
  • *

    P < 0.0001;

  • **

    P < 0.05.

  • Effect size, mean difference/SD pre-TURP; Guyatt statistic, mean difference/SD of stable lower urinary tract symptoms patients (medical treatment group). GHQ-12 items, General Health Questionnaire-12 items.

11.330.471.050.220.280.550.560.64
21.220.590.780.470.430.710.730.55
31.290.511.040.190.250.520.490.76
41.200.401.040.190.16**0.460.400.55
51.110.580.860.420.250.610.430.41
61.240.650.950.480.290.820.450.45
71.590.651.060.370.530.760.821.15
81.200.400.990.190.210.410.530.55
91.040.650.520.530.520.810.800.71
100.480.530.320.490.16**0.760.300.28
110.380.580.180.420.20*0.760.340.39
121.110.390.990.110.13**0.400.330.31
Total scores13.254.569.823.043.434.740.750.88

All items of the GHQ-12 demonstrated a high degree of sensitivity and specificity to the effects of treatment (Table 3). Significant changes were observed across all items in the TURP group. The lowest magnitude of change was noted in item 12. In contrast, except for items 1 (P < 0.0001) and 3 (P < 0.05), no comparison in the treatment of the control subjects approached significance (P < 0.09–0.77).

Table 3.  Sensitivity and specificity: GHQ-12 items characteristics of patients undergoing transurethral resection of the prostate (TURP) and the control group
GHQ-12 itemsNo.Mean changesSEMt statisticsP value
  1. GHQ-12 items, General Health Questionnaire-12 items.

Sensitivity
1790.280.0624.470.0001
2790.430.085.390.0001
3790.250.0584.340.0001
4790.160.0523.150.002
5790.250.0693.690.0001
6790.290.0923.160.002
7790.530.0866.180.0001
8790.210.0464.630.0001
9790.520.0925.670.0001
10790.160.0851.930.05
11790.200.0852.380.02
12790.130.0452.790.007
Total scores793.430.536.430.0001
Specificity
1500.260.0634.150.0001
2500.140.0811.730.09
3500.080.0392.060.044
4500.060.0441.350.182
5500.120.0841.430.159
6500.020.0670.300.766
7500.100.0511.940.058
8500.040.041.000.322
9500.060.0720.8290.411
10500.060.0720.8290.411
11500.020.0530.3750.709
12500.020.021.000.322
Total scores500.680.411.670.102

DISCUSSION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. ACKNOWLEDGMENTS
  9. REFERENCES

The GHQ-12 has proved to be a valid and a reliable tool in patients with LUTS. Its ability to discriminate between patients with LUTS and those without showed high levels of sensitivity and specificity. The large effect size obtained when the questionnaire was administered before and after an intervention of known efficacy (TURP) indicated a high degree of sensitivity to change. Similar findings were also noted in other studies.11,12

These findings provide substantial assurance that scores obtained using the GHQ-12 are reliable. The present findings on the reliability and validity was consistent with scores of GHQ-12 validated in various countries.13

The psychometric properties of the GHQ-12 validated in the present study and GHQ-12 validated in other countries13 showed that they are virtually identical with respect to their measurement properties. Although test–retest exhibited statistical significance at P < 0.05 in some items of GHQ-12, this is not surprising given that the test–retest was done at a 12 week interval which allowed the urinary symptoms and psychiatric morbidity to improve or worsen. In contrast, the minimal changes of symptoms and psychiatric morbidity would occur if test–retest were done at 1 week, 2 weeks or at the latest 1 month after the initial assessment. The reason for the test–retest to be carried out at a 12 week interval was due to the fact that most of the patients in the surgical group (TURP) could only achieved maximum benefits or total symptom improvement at 8–12 weeks and thus comparison could be made between patients with LUTS on medical treatment, patients undergoing TURP and the control group.

CONCLUSION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. ACKNOWLEDGMENTS
  9. REFERENCES

The reliability and validity of the GHQ-12 for 237 urological patients were tested. The ICC for total scores of the GHQ-12 indicated a high intraclass reliability. Likewise, the high Cronbach's alpha coefficient indicates that GHQ-12 exhibited a good internal consistency. One could then conclude that the GHQ-12 is a useful and accurate tool for assessing the severity of LUTS among urological patients.

ACKNOWLEDGMENTS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. ACKNOWLEDGMENTS
  9. REFERENCES

We would like to thank the University Malaya's Post Graduate Scholarship Committee as well as the Research and Development Management Unit for providing the financial support and research grant for this study.

REFERENCES

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONCLUSION
  8. ACKNOWLEDGMENTS
  9. REFERENCES
  • 1
    Goldberg D. General Health Quesionnaire (GHQ-12). NFER-Nelson Publishing Co. Ltd, Windsor, 1992.
  • 2
    Goldberg DP & Williams P. User's Guide to the General Health Questionnaire. NFER-Nelson Publishing Co. Ltd, Windsor, 1988.
  • 3
    Papussotiropoulos A, Heun R, Maier W. Age and cognitive component influence the performance of the general health questionnaire. Compr. Psyc. 1997; 38: 335340.
  • 4
    Kiseley S, Goldberg D, Simon G. A comparison between somatic symptoms with and without clear organic cause: Results of an international study. Psychol. Med. 1997; 27: 10111019.DOI: 10.1017/s0033291797005485
  • 5
    Goldberg D. Identifying psychiatric illness among general medical patients. Br. Med. J. 1985; 291: 161162.
  • 6
    Quek KF, Low WY, Razack AH, Loh CS. Psychological effects of treatment for lower urinary tract symptoms. BJU Int. 2000; 86: 630633.DOI: 10.1046/j.1464-410x.2000.00839.x
  • 7
    Lukacs B, Leplege A, McCarthy C, Comet D. Construction and validation of a BPH specific health related quality of life scale, for medical outcome reseach studies. In: Cockett ATK, Khoury S, Aso Y, Chatelain C, Denis L, Griffiths and Murphy G (eds). Proceedings of the Second International Consultation on Benign Prostatic Hyperplasia (BPH). SCI, Paris, 1994; 139143.
  • 8
    Barry MJ, Fowler FjJr, O'Leary MP et al. The American Urological Association symptom index for benign prostatic hyperplasia. J. Urol. 1992; 148: 15491557.
  • 9
    Cronbach LJ. Coefficient alpha and the internal structure of test. Psychometrika 1951; 16: 297.
  • 10
    Deyo RA, Dichr P, Patrick DL. Reproducibility and responsiveness of health status measures. Control. Clin. Trials 1991; 12 (Suppl. 1): 142S158S.
  • 11
    Cohen J. Statistical Power Analysis for the Behavioral Sciences. Academic Press, New York, 1977.
  • 12
    Guyatt GH, Walter S, Norman G. Measuring changes overtime: Assessing the usefulness of evaluative instruments. J. Chronic Dis. 1987; 40: 171178.
  • 13
    Goldberg DP, Gater R, Sartorius N, Ustun TB, Piccinell OG, Rutter C. The validity of two versions of the GHQ in the WHO study of mental illness in general health care. Psychol Med. 1997; 27: 191197.DOI: 10.1017/s0033291796004242