The evidence base to select a method for assessing glaucomatous visual field progression

Authors


Paul J. G. Ernest, MD
University Eye Clinic Maastricht
PO Box 5800
6202 AZ Maastricht
the Netherlands
Tel: + 31 43 387 5346
Fax: + 31 43 387 5343
Email: p.j.g.ernest@gmail.com

Abstract.

A large number of methods have been developed for assessing glaucomatous visual field progression, but their properties have not yet been systematically evaluated. In this systematic literature review, we summarize the evidence base for selecting a method by providing answers to ten relevant questions on the variety, validity and reproducibility of methods. In total, we found 301 different methods in 412 articles. The majority of studies (54%) used the Humphrey Field Analyzer. No data have been published about the reproducibility of methods. Although there is no gold standard to assess glaucomatous visual field progression, we found evidence on validity for 48 different methods. Some methods were less capable of distinguishing between progressive and nonprogressive patients. Choosing among twelve methods is supported by some evidence of their validity. These methods still differ in sensitivity, specificity and predictive values of test results within studies comparing several methods. In conclusion, the current evidence base is not perfect. A selection should be made from a limited number of methods, according to the clinical purpose of progression assessment. Methods that quantify the rate of visual field progression seem to be the most appropriate for guiding subsequent medical actions in individual patients. Future studies should investigate whether using one method to monitor patients is superior to another method in preventing loss of quality of life.

Introduction

An important goal in the management of open-angle glaucoma is to detect clinically relevant visual field progression early and reliably. The methods to assess progression, however, are a continuing subject of debate (Nouri Mahdavi et al. 1997; Katz et al. 1999; Lee et al. 2002; Vesti et al. 2003; Mayama et al. 2004; Heijl et al. 2008). In the last years, the relevance of monitoring the rate of visual field progression has increasingly being recognized (Rossetti et al. 2010). Together with the patient’s life expectancy and disease stage, the rate of progression can be used to estimate individual risk of lifetime visual disability (Bengtsson & Heijl 2008). However, from all methods to determine visual field progression, it may still be difficult to select a specific one.

This article provides a systematic review of the evidence base for selecting a method to assess glaucomatous visual field progression. The clinical use of a good method to assess progression should ultimately optimize quality of life (QoL) of patients with glaucoma, without under- or overtreatment. With this aim in mind, we searched the literature to provide answers to ten relevant questions.

Ten Questions and Answers

1. How many methods can we choose from to assess visual field progression?

A total of 301 different methods were used in 412 articles.

We classified all methods that we found in the literature. There are 15 different perimeters used to assess progression in the literature. A majority of 222 studies (54%) used the Humphrey Field Analyser (HFA) (Carl Zeiss Meditec, Dublin, CA, USA), increasing to 77% of the articles published since 2000. We therefore focused on methods for the HFA for the rest of this article. HFA methods could be further classified into qualitative and quantitative methods. A qualitative method implies that the ophthalmologist decides on the occurrence of progression, whereas a quantitative method uses numeric units for defining progression. Qualitative methods have been used 32 times (8%), and quantitative methods, 355 times (92%). Quantitative methods that calculate a rate of progression were used 166 times (47% of quantitative methods). However, most of these studies dichotomized the rate of progression because they aimed to compare different progression methods or estimated treatment effects in a large group of patients. Therefore, even these methods did not really quantify the rate of progression needed for decision making in individual patients.

2. Which method to assess visual field progression can predict the loss of QoL?

The prediction of loss in QoL has not been shown for any method.

The ultimate goal of glaucoma management is to prevent the loss of QoL. A method to assess progression should therefore identify patients who will lose vision-related QoL in the future if treatment is not intensified. Although this constitutes the essential goal of monitoring progression, it has not been addressed in empirical research. However, its clinical relevance is increasingly being recognized (Rossetti et al. 2010). Empirical research should ideally randomize patients to different monitoring strategies with a subsequent long follow-up period to evaluate differences in QoL. Future studies should address this issue with the inclusion of methods quantifying the rate of progression. What we do know is that the degree of visual field loss and QoL are strongly related (Gutierrez et al. 1997; Viswanathan et al. 1999; Kobelt et al. 2006; McKean Cowdin et al. 2008; van Gestel et al. 2010).

3. What is the gold standard to assess glaucomatous visual field progression?

There is no gold standard to assess visual field progression.

A gold standard is a method that is closest to anatomical and consequent functional changes, against which new developments should be compared (Higgins & Green 2005). There is no universally accepted solution in diagnostic research when faced with a missing or an imperfect gold standard (Reitsma et al. 2009). In the literature about methods to assess glaucomatous visual field progression, most of the studies compared the incidence of progression from different methods. Although these studies give insight into the methods that give either high or low estimates of the incidence of progression, they do not contribute in the selection of the best method. We summarized the results of these studies in an earlier review article (Ernest et al. 2010). Other studies dealt with the problem of the lack of a gold standard using another reference standard against which methods were compared. Different reference standards have been used that are of some value to substitute for a gold standard.

4. Which methods have been compared with a substitute gold standard of visual field progression or stability?

Several methods have been compared with a substitute gold standard to assess visual field progression. In the field of clinimetrics, this is termed concurrent validity (Streiner & Norman 2008a). The resulting sensitivities, specificities, positive predictive values (PPVs), negative predictive values (NPVs), likelihood ratios (LRs) and diagnostic odds ratios are shown in Table 1. The first thing to note is that there is much variation in several accuracy measures within studies and between studies. There seems to be no superior method, although some have a lower diagnostic odds ratio when compared with other methods within one study. These methods are less capable to distinguish between progressive and nonprogressive patients.

Table 1.   Accuracy measures of different methods to assess visual field progression.
StudyReference standardNumber of eyesMethodSenSpecPPVNPVLR+LR−OR
  1. If sensitivities and specificities of methods were available, we calculated positive and negative predictive values, likelihood ratios, and odds ratios using formulas that have earlier been described (Knottnerus 2002). Positive and negative predictive values were calculated for a presumed incidence of progression of 20% in six years, based on a recent meta-analysis (Ernest et al. 2010). Cut-off points for several methods are shown between brackets (e.g. the number of test points or visual fields that had to be progressive before the total visual field series is considered to be progressive).

  2. * These studies used a preselected sample of unequivocally progressive and nonprogressive patients after excluding patients with equivocal results.

  3. Simulated visual field series.

  4. Stable eyes are compared with eyes changed in either direction.

  5. Sen = sensitivity; Spec = specificity; PPV = positive predictive value, which is the probability that a patient has ‘true progression’ when the method indicates a patient as progressive; NPV = negative predictive value, which is the probability that the patient is ‘true nonprogressive’ when the method indicates a patient as nonprogressive; LR+ = likelihood ratio of a positive test result, which indicates how much a positive test result will raise the pretest probability of progression; LR− = likelihood ratio of a negative test result, which indicates how much a negative test result will lower the pretest probability of progression; OR = diagnostic odds ratio, which describes the odds of a positive test result in patients with progression compared with the odds of a positive test result in those without progression; AGIS = Advanced Glaucoma Intervention Study; CIGTS = Collaborative Initial Glaucoma Treatment Study; EMGT = Early Manifest Glaucoma Trial; GCP = Glaucoma Change Probability; TNT = Threshold Noiseless Trend program; FI = focality index; PLR = pointwise linear regression analysis; MD = mean deviation; CSS = clinical scoring system based on scotomas; CNTGS = Collaborative Normal-Tension Glaucoma Study.

Concurrent validity
Comparisons with substitute gold standards of visual field progression and stability
 Heijl et al. (2008)Masked subjective judgment*172AGIS0.580.980.880.9029.00.4367.7
CIGTS0.750.990.950.9475.00.25297.0
EMGT0.960.900.710.999.60.04216.0
 Katz et al. (1999),  Katz (2000)Masked subjective judgment67AGIS0.360.960.700.869.40.6714.1
CIGTS0.570.910.600.896.10.4712.8
EMGT0.500.870.490.873.80.586.6
GCP (3 points, 3 fields)0.790.790.490.943.80.2714.0
 Diaz Aleman et al. (2009)Masked subjective judgment*47EMGT0.270.950.570.845.40.777.0
EMGT suspected progression0.530.850.470.883.50.556.0
TNT0.800.710.410.932.80.289.8
TNT suspected progression0.860.600.350.942.20.239.2
TNT with FI >40th percentile0.670.900.630.926.70.3718.3
 Mayama et al. (2004)Masked subjective judgment for sensitivities* and computer simulated stability for specificities10510 000PLR according to Noureddin et al. (1991)0.910.260.240.921.20.333.8
PLR according to Fitzke et al. (1996)1.000.110.221.001.10.001173.1
PLR according to Bhandari et al. (1997)0.850.720.430.953.00.2114.4
PLR according to Nouri- Mahdavi et al. (1997)0.850.720.430.953.10.2114.5
MD slope (p < 0.1)0.740.890.630.936.90.2923.9
MD slope (p < 0.05)0.520.950.700.899.50.5018.9
MD slope (p < 0.025)0.460.970.800.8816.30.5629.2
  Method according to Werner et al. (1988)0.700.950.760.9312.90.3239.9
AGIS0.311.001.000.853050.00.704388.1
AGIS (2 fields)0.471.000.990.88467.00.53875.3
 Vesti et al. (2003)Computer simulated stability76AGIS1.00
CIGTS0.97
GCP (4 points, 2 fields)0.92
GCP (8 points, 2 fields)0.95
GCP (4 points, 3 fields)0.99
PLR (2 points, 3 fields)0.82
PLR (3 points, 3 fields)1.00
 AGIS (1994)Stability owing to short follow-up756AGIS (1 field)0.84
 Gillespie et al. (2003)Stability owing to short follow-up607CIGTS (1 field)0.90
Comparisons with other parameters of disease progression
 Girkin et al. (2000)Progressive optic disc cupping47AGIS (1 field)0.341.001.000.863380.00.665105.2
CSS0.590.880.550.904.90.4610.6
Sustainability
Ability to sustain progressive in future visual fields
 Heijl et al. (2008)Sustainability after 4 years172AGIS0.95
CIGTS1.00
EMGT0.93
 Lee et al. (2002)Sustainability after 2 more visual fields48Modified GCP (1 field)0.26
Modified EMGT (1 field)0.31
GCP (3 points, 1 field)0.16
EMGT (1 field)0.16
AGIS (1 field)0.33
CIGTS (1 field)0.18
 AGIS (2004)Sustainability during follow-up752AGIS (2 fields)0.45
AGIS0.62
 Schulzer (1994)Mathematical model456CNTGS single sequence0.43
CNTGS double sequence0.98
 Nouri-Mahdavi et al. (2007)Sustainability after 4 years156PLR (3 points, 3 fields)0.91
GCP (3 points, 3 fields)0.88
AGIS0.80

Some studies made use of expert judgement of visual field series as a substitute for a gold standard (Katz 2000; Mayama et al. 2004; Heijl et al. 2008; Diaz Aleman et al. 2009). However, these qualitative judgements can be seen as arbitrary gold standards, because they have not shown to be superior to other methods. If subjective judgements are used, it is important that these judgements are masked for other clinical information. This was the case in all studies. Three studies calculated sensitivities and specificities with use of a preselected sample of unequivocally progressive and nonprogressive patients after excluding patients with equivocal results (Mayama et al. 2004; Heijl et al. 2008; Diaz Aleman et al. 2009). This may have resulted in overestimated accuracy measures.

As another substitute for a gold standard, some studies used groups of patients with glaucoma that were likely to be stable (i.e. not showing progression) to calculate the specificity of a method (The AGIS Investigators 1994; Gillespie et al. 2003; Vesti et al. 2003; Mayama et al. 2004). Using a computer simulation, two studies constructed a substitute gold standard by simulating stable visual field series with a physiological degree of variability (Vesti et al. 2003; Mayama et al. 2004). Two other studies approached the specificity of a method by testing whether progression criteria were fulfilled within a follow-up period that was too short for the visual field to change (i.e. 1 month) (The AGIS Investigators 1994; Gillespie et al. 2003).

5. Which methods have been compared with other parameters of disease progression?

We found one study that used progressive optic disc cupping as a reference standard. The Advanced Glaucoma Intervention Study (AGIS) method had a very high diagnostic odds ratio and a very high likelihood ratio for a positive test result (Table 1) (Girkin et al. 2000).

Instead of comparing methods to assess glaucomatous visual field progression with a substitute gold standard to assess visual field progression, outcomes of methods can also be compared with other parameters of disease progression like changes in the optic nerve head. This is also a form of concurrent validity.

6. Which methods give a good prediction of future visual field loss?

Several methods have a high sustainability. These include the AGIS, Collaborative Initial Glaucoma Treatment Study (CIGTS), Early Manifest Glaucoma Trial (EMGT), Collaborative Normal-Tension Glaucoma Study (CNTGS) and pointwise linear regression analysis (PLR) methods (Table 1).

When methods have shown progression, they should remain to do so when additional follow-up visual fields are acquired because glaucomatous nerve fibre loss is assumed to be irreversible. The positive test results must therefore be sustained in future assessments. However, the results of assessments that do not show progression are not supposed to sustain because the patient could progress afterwards. In clinimetrics, this is termed predictive validity, but sustainability is a more appropriate term as used by the authors (Streiner & Norman 2008a).

One way to investigate the sustainability of progression is to use the outcomes after a limited number of follow-up years to predict outcomes after a longer period, both using the same baseline as a reference (Lee et al. 2002; The AGIS Investigators 2004; Nouri Mahdavi et al. 2007; Heijl et al. 2008). Furthermore, another study estimated the sustainability of the CNTGS method by a mathematical model based on repeated testing (Schulzer 1994).

Instead of looking at the sustainability of positive test results, Bengtsson and coworkers used correlations to validate the continuous Visual Field Index (VFI) rate (Bengtsson et al. 2009). They investigated whether the VFI rate in the initial 3.3 years could reliably predict the VFI after a mean follow-up time of 8.2 years. A correlation coefficient of 0.78 was found when the predicted VFI was compared with the actual last VFI (Bengtsson et al. 2009).

7. Which methods have shown to be related with a presumed prognostic factor of glaucomatous progression?

In total, 20 different methods have been studied in relation with mean intraocular pressure (IOP) in 21 articles. Thirteen methods (65%) found a positive relationship between mean IOP and glaucomatous visual field progression (Fig. 1). Six of these methods (30%) showed a statistically significant positive difference (p < 0.05) in mean IOP between the progressive and nonprogressive groups, including the EMGT method (Bengtsson et al. 2007), another method based on point-wise event analysis (Lee et al. 2008), a method based on PLR (Nouri Mahdavi et al. 2004) and three qualitative methods (Stewart et al. 1993, 2000; Diestelhorst et al. 1998) (Fig. 1).

Figure 1.

 Difference in mean intraocular pressure between progressive and nonprogressive patients according to 20 different methods to assess progression. A positive difference indicates a positive relationship between visual field progression and mean intraocular pressure during follow-up. The grey bars represent the differences that were tested statistically significant, while the white bars represent the differences that were not statistically significant. Each bar is labelled by the method that was used and by a reference of the study from where we derived the data. When one method was investigated in two studies, we calculated the weighted mean difference between the progressive and nonprogressive groups based on the number of patients in each study. Bars Subj1 to Subj4 represent qualitative (i.e. subjective) methods, bars MD1 to MD3 represent methods based on the mean deviation index, bars PLR1 to PLR2 represent methods based on pointwise linear regression analysis, and bars PWE1 to PWE3 represent methods based on pointwise event analysis. Modified Anderson represents a method adapted from Chen & Park (2000). Blindness represents a method that uses blindness as end-point. Blumenthal represents a method derived from Blumenthal et al. (2000). IOP = intraocular pressure, EMGT = Early Manifest Glaucoma Trial method, AGIS = Advanced Glaucoma Intervention Study method, CIGTS = Collaborative Initial Glaucoma Treatment Study method, GCP = Glaucoma Change Probability method.

A method that is assumed to assess glaucomatous visual field progression should find a relationship between progression and a presumed risk factor for glaucomatous progression. In clinimetrics, this is termed construct validity (Streiner & Norman 2008b). An example could be the relationship with mean IOP during the follow-up period. The reason for this is that IOP is a consistent prognostic factor for glaucoma and the focus of treatment (Coleman & Miglior 2008). Although the four qualitative methods resulted in a high positive association between mean IOP and visual field progression, the outcome assessment in these studies was not masked for other clinical information (Stewart et al. 1993, 2000; Diestelhorst et al. 1998; Brauner et al. 2006). Moreover, two of them included other parameters like visual acuity and optic disc criteria in the judgement of progression (Stewart et al. 1993, 2000).

8. Which methods have shown to be reproducible?

To our knowledge, no studies about the reproducibility of methods to assess visual field progression have been conducted.

However, we found 21 articles that studied cross-sectional reproducibility of visual field measures that were derived from the HFA (McMillan et al. 1992; Fitzke et al. 1995; Katz et al. 1995; Bengtsson & Heijl 1998a,b; Chauhan & Johnson 1999; Bengtsson 2000; Sekhar et al. 2000; Spry et al. 2000, 2001, 2003; Artes et al. 2002, 2005; Blumenthal et al. 2003; Gillespie et al. 2003; Bjerre et al. 2004; Jampel et al. 2006; Tattersall et al. 2007; Wyatt et al. 2007; Schiefer et al. 2009; Wall et al. 2009). In general, these studies showed that mean deviation (MD) values have a higher reproducibility than pointwise values.

9. Taking into account the evidence above, which method should we select from the 301 available methods?

The selection from 301 methods is limited to 48 different methods for which data on validity were present (see questions 4–7). Excluding the different cut-off points, the selection is limited to twelve methods being the AGIS, CIGTS, PLR, MD, Glaucoma Change Probability (GCP), EMGT, VFI, Threshold Noiseless Trend (TNT), Werner, clinical scoring system (CSS), CNTGS, and subjective methods. It is important to keep in mind that methods were validated with the use of different reference standards and different study designs.

Methods based on the AGIS generally have a high discriminative ability, which is shown by the high odds ratios in Table 1. However, we found an inverse relationship between AGIS progression and mean IOP during follow-up (Fig. 1). This is probably a biased result because IOP may have been lowered owing to changes in treatment among those with progression. Moreover, mean IOP during the first follow-up years has earlier shown to be associated with subsequent progression on the AGIS score (The AGIS Investigators 2000). The CIGTS method also showed high odds ratios and PPVs. Methods with a high PPV, like the AGIS and the CIGTS methods, could, for example, be used before performing glaucoma surgery, where one wants to be certain that a patient is really progressing.

Pointwise linear regression analysis methods showed highly variable accuracy measures ranging from a low odds ratio of 3.8 to a much better odds ratio of 1173.1 (Table 1). This is caused by the use of many different cut-off points for these methods in the literature. However, PLR, AGIS or CIGTS methods may be impractical in clinical practice, because they are more time-consuming owing to the need to interpret several test locations. These methods could possibly be made more usable by computerisation of the analysis.

Methods based on the VFI, MD, GCP or EMGT may be more usable, because the required information is available on the printed output of the HFA. Among them, the EMGT method is the only method that has shown to correlate with mean IOP during follow-up (Fig. 1). Methods based on MD and EMGT seem to perform well in the studies of Heijl and Mayama, although these studies probably overestimated the accuracy of methods (Table 1) (Mayama et al. 2004; Heijl et al. 2008). The odds ratio of the EMGT method was relatively low in the other studies (Katz 2000; Diaz Aleman et al. 2009).

Qualitative methods could also be useful, although the interpretation of results is dependent on the capacity of the observer. This may cause high interobserver variability (Werner et al. 1988). Nonetheless, these methods have frequently been used as a substitute for a gold standard. In these cases, however, the assessment was based on the judgement of more than one observer. Qualitative methods have also shown to correlate well with mean IOP, but these findings could be biased because these qualitative assessments were not masked for other clinical information.

10. In the end, what do we really want to know?

The current evidence base is not perfect but seems to be fair for a few methods that have been validated. As numerous methods are available, we should probably stop developing many new methods to assess visual field progression. The ultimately relevant question, whether using one method to monitor patients is superior to another in preventing loss of QoL, has not been answered. Methods that quantify the rate of visual field progression seem to be the most appropriate for guiding subsequent medical actions in individual patients, because they can be used to estimate individual risk of lifetime visual disability. This should ideally be studied in prospective studies with long follow-up periods. If this is not feasible, new research should be directed to other types of validation, like the ability of methods to predict future visual field loss, and to the relationship between outcomes of methods and structural progression or prognostic factors.

Literature Search

To obtain an overview of all methods to assess glaucomatous visual field progression, we performed a systematic literature search in April 2009. We searched in PubMed, EMBASE and all databases and registers of The Cochrane Library with the use of the following keywords: (glaucoma*) AND (prognos* OR predict* OR progress* OR longitudinal OR cohort OR follow-up) AND [perimetr* OR visual field* OR HFA OR Octopus OR Humphrey (not in author)]. A total of 2450 articles were identified. Based on predefined exclusion criteria, we selected studies reporting on patients with glaucoma who were followed for a minimum of 1 year with the use of standard visual field examinations, so that progression could be assessed (Table 2). We finally included 412 articles (Fig. 2). From this search, we also selected 21 articles that used the HFA and studied mean follow-up IOP as a prognostic factor for glaucomatous visual field progression.

Table 2.   Number of articles excluded according to different criteria.
Exclusion criteriaNumber
  1. Ten exclusion criteria are listed with their accompanying number of excluded articles.

  2. VF = visual field.

No intention to measure progression of VF damage471
Unconventional or single perimetric measurement434
No visual field damage at the beginning of follow-up395
Patients without glaucomatous disease244
No original study172
Perimetric follow-up <1 year (median or mean <1 year)124
Case report (<10 patients)80
Patients aged <18 years35
Animal study17
Other languages than English, French, Dutch or German8
Total1980
Figure 2.

 The selection process of studies found in the first systematic literature review is shown in this flow chart. Four hundred and twelve articles were finally included. HFA = Humphrey Field Analyzer.

We studied the reproducibility of progression methods in patients with glaucoma by performing a second systematic search in PubMed in April 2009: glaucoma [Mesh] AND visual fields [Mesh] AND (observer variation [Mesh] OR reproducibility of results [Mesh]).

Acknowledgements

No authors have any financial/conflicting interests to disclose.

Ancillary