Characteristics of randomised trials in ophthalmology using a single eye per person design

  • Protocol
  • Methodology

Authors


Abstract

This is the protocol for a review and there is no abstract. The objectives are as follows:

To describe how often studies that use a single eye design report the rationale for choosing that design and report the methods used to select which eye to use.

Background

Description of the problem or issue

Randomised controlled trials (RCTs) in ophthalmology generally involve the examination of eyes for a variety of clinical signs or measurements. Unlike cardiology, hepatology and many other clinical specialties, individuals have two eyes which might be examined. In addition, unlike kidneys or lungs, both eyes are easily accessible for assessment and both are frequently assessed. In some cases the data from one eye may be unavailable, resulting in data sets with information on one eye for some individuals and on two eyes for others (Murdoch 1998).

If information is available on both eyes of the same individual, the findings in the left eye are generally (but not always) likely to be more similar to those in the right eye of the same individual than to those in another eye from a different individual. This is because a multitude of factors, including environmental and genetic, act at the level of the individual and thus have an impact on the probability of the finding occurring in both eyes (Murdoch 1998).

The degree to which a given finding in the first eye predicts the probability of the same finding in the fellow eye varies considerably, however, for different conditions. Some conditions like choroidal melanoma, herpes simplex keratitis in immunocompetent patients, or severe ocular trauma, characteristically only ever occur in one eye (99%, 98% and 98% of cases are unilateral, respectively) (Castella 1995; Dunn 1992). Other conditions, such as inflammatory ocular surface disorders like blepharitis or allergic conjunctivitis, almost always affect both eyes (Murdoch 1998; Seal 1995) and the finding in the first eye almost perfectly predicts the finding in the fellow eye. Other conditions such as keratoconus may affect both eyes, but show considerable asymmetry between fellow eyes.

Correlations between observations are a recognised statistical problem in many areas of medical research (Qaqish 1992), including ophthalmology (Newcombe 1987). If such correlations are not taken into account in statistical analyses, there may be errors in the results obtained, usually resulting in falsely precise confidence intervals and falsely small P values.

Description of the methods being investigated

Because including both eyes can present issues in statistical analyses, ophthalmic researchers often choose to analyse only one eye per individual. Selecting a single eye avoids the problem of inter-eye correlation: selection might be at random or one eye may be selected based on clinical grounds, or a systematic decision may be taken, such as including all right eyes only or all left eyes only. For conditions that are equally likely to affect either eye, analyses based on only right eyes, or on only left eyes, or on a randomly selected eye are statistically equivalent. Analyses based on a single eye per individual are convenient in that standard statistical methods can be employed. The major disadvantage of this approach is a loss of information. Thus, the use of data from only one eye per individual is statistically valid but is likely to be inefficient (the power and precision of the analysis are less than optimal) (Murdoch 1998).

Another inconvenience of this analysis is the potential for bias arising through the choice of which eye to use. Often, if data are only available on a single eye in an individual because of incomplete data on the other eye for various reasons, that single eye is included in the analysis. Bias could occur if some individuals have data on both eyes, only one of which is 'randomly' selected for analysis, while other individuals have data on only one eye which is automatically included (that is, not randomly selected). The non-random selection of eyes can introduce bias. Similar caveats may apply to the choice of the first eye with disease, worse/better eye or operated eye (Murdoch 1998).

In conclusion, the use of one eye per individual in the analysis means that between eye correlation is not a problem, but that there may be considerable 'waste' of available data. If the eye is selected after an outcome is observed or simply because it is the only eye available, there is potential bias which should be acknowledged and, when possible, its likely extent assessed.

Why it is important to do this review

As stated above, unit of analysis issues may act as a potential source of bias in clinical trials studying ophthalmic conditions (Murdoch 1998). While a single eye per patient design can be correct and efficient for some conditions and interventions, for other instances it may be inefficient, as information from the eye not included is lost (Murdoch 1998; Newcombe 1987). Because one-eye selection might be done legitimately to avoid statistical complexity, or illegitimately (for example one eye shows positive results, the other does not), we feel that it is of value to review trials to identify whether researchers clearly state why a single-eye design was chosen and to identify if they are explicit as to how that eye was selected. A methodology review is needed to describe common practice in ophthalmic trial design and raise awareness of the need to be explicit in specifying the unit of analysis when conducting an eye-related trial.

Objectives

To describe how often studies that use a single eye design report the rationale for choosing that design and report the methods used to select which eye to use.

Methods

Criteria for considering studies for this review

Types of studies

In this review we will consider RCTs published in general ophthalmological journals. We will include studies which report data on one eye per participant and we will exclude studies which report on two eyes from any participant, or that do not mention the number of eyes included per participant from the review.

Types of data

We will include RCTs studying the effect of clinical interventions on one or more clinical outcomes, in patients with ophthalmic conditions, analysing only one eye in all the individuals included in the study.

We will extract information about the methods used in the trial reports and the following details:

  • condition (unilateral versus bilateral, specific conditions);

  • interventions (surgical, topical drugs, systemic drugs, optic devices, laser surgery, others);

  • outcomes (visual acuity, intraocular pressure, signs and symptoms scales, quality of life, etc.);

  • reporting a rationale for using a single eye per individual design (yes versus no; and specific rationale);

  • reporting the methodology for eye selection, and a rationale for using that methodology (yes versus no; and specific methodology used);

  • risk of bias;

  • country where the study took place and country of the investigating team;

  • other information (such as funding).

Types of methods

We will compare the following methods between studies:

  1. using a rationale for using a single eye per individual design;

  2. reporting the method used for eye selection.

Types of outcome measures

Primary outcomes
  • Reporting (with description) a rationale for using a single eye per individual design

Secondary outcomes
  • Reporting (with description) the methodology for eye selection

Study factors
  • Condition (unilateral versus bilateral; specific conditions)

  • Intervention (surgical, topical drugs, systemic drugs, optic devices, laser surgery, others)

  • Outcomes (visual acuity, intraocular pressure, signs and symptoms scales, quality of life, etc.)

  • Risk of bias

Search methods for identification of studies

Electronic searches

All trials meeting the inclusion criteria included in the CEVG register and published during 2012 will be included in the review.

Searching other resources

For this review, we will not be searching any other resources.

Data collection and analysis

Selection of studies

Two review authors will independently check the titles and abstracts resulting from the searches. Using a form developed to document the process, we will divide the titles and abstracts into three groups: ‘exclude’, ‘unsure’ or ‘include’. We will mark all those titles classified as ‘unsure’ or ‘include’ and retrieve their full-text versions for definitive assessment of eligibility. At this stage, we will only exclude those papers classified by both review authors as ‘exclude’.

Using another form developed to document the process, we will classify the full-text copies into three groups (‘exclude’, ‘unsure’ or ‘include’), according to pre-stated criteria (see 'Criteria for considering studies for this review'). We will resolve any disagreements through discussion. If we cannot reach a consensus, we will consult a third review author or the editorial base. If relevant, we will document the disagreements. If there is insufficient information to determine the eligibility of a study (full text classified as ‘unsure’), we will ask the authors for clarification. We will detail in the table of excluded studies all relevant studies labelled as ‘excluded’ after the assessment of the full text, with the reasons for their exclusion.

Review authors will not be masked to trial results or publication details during the selection of the studies.

Data extraction and management

Two review authors will independently perform the extraction of data from trial reports, using pre-designed data extraction forms. 

The review authors will resolve discrepancies on the data extraction through discussion. If there is no consensus, a third review author or the editorial base will settle the discrepancies.

One review author will enter the data into Review Manager 5 (RevMan 2012) and another one will check the data entered manually.

Assessment of risk of bias in included studies

We will assess the risk of bias of each included study, according to the criteria of The Cochrane Collaboration’s tool for assessing risk of bias as described in Chapter 8 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011). We will consider the following domains:

  • Sequence generation: was the allocation sequence adequately generated?

  • Allocation concealment: was allocation adequately concealed?

  • Blinding (masking) of participants, personnel and outcome assessors: was knowledge of the allocated interventions adequately prevented during the study?

  • Incomplete outcome data: were incomplete outcome data adequately addressed?

  • Selective outcome reporting: are reports of the study free of suggestion of selective outcome reporting?

  • Bias due to non-randomised selection of some of the eyes analysed in bilateral conditions.

We will make assessments for the primary outcome of each study only. In studies with multiple primary outcomes, or in which the primary outcome is not stated, we will assess the risk of bias for each of these outcomes.

At least two review authors, not masked to the study details, will independently label each domain as ‘low risk of bias', ‘high risk of bias' or ‘unclear risk of bias'.

Disagreements will be resolved by discussion and consensus and, if necessary, with the involvement of a third author or the editorial base. We will document the inter-rater reliability in the 'Risk of bias' assessment using the kappa statistic (Higgins 2003) and we will report relevant discrepancies in the assessments.

We will summarise the overall risk of bias for the primary outcome in each study in two different manners (Higgins 2011):

  • within each study across domains: each primary outcome will be defined as having a ‘low risk of bias’ only if it meets all the domains; as ‘high risk of bias’ if it demonstrates high risk of bias for one or more of them; or an ‘unclear risk of bias’ if it demonstrates unclear risk of bias for at least one key domain without any of them described as ‘high risk of bias’;

  • across studies: each primary outcome will be defined as having a ‘low risk of bias’ if most information is from studies at low risk of bias; as ‘high risk of bias’ if the proportion of information from studies at high risk of bias is sufficient to affect the interpretation of the results; or an ‘unclear risk of bias’ if most information is from studies at low or unclear risk of bias.

We will try to obtain the information from the trial reports. If any domain is graded as ‘unclear’, JJGL will write to the authors for clarification. If clarification is not obtained in less than 30 days, we will assign a grading to the outcome, based on the consensus between the review authors. 

Analysis methods

Our primary outcome is dichotomous (rationale stated or not stated).

We will cross-tabulate our primary outcome against all secondary outcomes. We will then use multivariable logistic regression to analyse whether or not stating the rationale for one eye in a unilateral ophthalmic RCT is associated with study factors. For example, we would anticipate that studies at low risk of bias for the usual domains will be at low risk for not stating rationale. The selection of variables in the final model will be performed by a forward-conditional method, with significance levels of ≤ 0.05 for inclusion and ≥ 0.1 for exclusion. We will assess overall model fit by the Nagelkerke R2 and the C-index. We will assess the validity of the model by the bootstrap technique (Efron 1993). For each bootstrap sample, we will refit the primary model and obtain the average odds ratio. We will use Stata SE 12.0 for Unix for the bootstrap analysis of the logistic regression model.

Acknowledgements

Julio J González-López is a Ph.D. candidate at the Surgery Department, Universidad de Alcalá, Madrid, Spain. He has received two study grants from 'Fundación Mutua Madrileña' and 'Fundación Jesús de Gangoiti Barrera' for the completion of this review.

We thank Anupa Shah (Managing Editor) and Iris Gordon (Trials Search Co-ordinator) for the Cochrane Eyes and Vision Group (CEVG) for their assistance with the protocol. We also thank Roberta Scherer, Barbara Hawkins, Gabriela Czanner and Ian Murdoch for their comments on this protocol.

Richard Wormald (Co-ordinating Editor for CEVG) acknowledges financial support for his CEVG research sessions from the Department of Health through the award made by the National Institute for Health Research to Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology for a Specialist Biomedical Research Centre for Ophthalmology.

The views expressed in this publication are those of the authors and not necessarily those of the NIHR, NHS or the Department of Health.

Contributions of authors

Julio J González-López: conceiving, designing, drafting and writing the protocol; searching the references collection for the background of the protocol; organising retrieval of papers; providing a methodological, clinical and policy perspective; final approval of the document to be sent to CEVG.

Catey Bunce: conceiving, designing and drafting the protocol; providing a methodological and policy perspective; final approval of the document to be sent to CEVG.

Fernando Rodríguez-Artalejo: support in designing, drafting and writing the protocol; final approval of the document to be sent to CEVG.

Declarations of interest

None of the authors declare any potential conflict of interest relevant to this protocol.

Sources of support

Internal sources

  • NIHR, UK.

    CB acknowledges financial support from the Department of Health through the award made by the National Institute for Health Research to Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology for a Specialist Biomedical Research Centre for Ophthalmology. The views expressed in this publication are those of the authors and not necessarily those of the NIHR, NHS or the Department of Health.

External sources

  • No sources of support supplied

Ancillary