To the Editor:

A recent HEPATOLOGY article by Lindor and Lindor1 suggests that we need to reconsider the status of randomized controlled clinical trials as the gold standard of evidence for evaluating new medical treatments. They suggest that observational studies for the evaluation of medical therapy are faster and cheaper and are more easily performed, and they ultimately lead to faster approval, a reduction of medication costs, and better patient care. However, Lindor and Lindor ignore the well-known fallibility of observational studies that have supported the therapeutic value of a treatment without evidence from well-developed randomized trials. There are many notable contemporary examples of widely used treatments that were accepted on the basis on observational studies and were subsequently proved to be ineffective or harmful when controlled clinical trials were performed. These treatments include the following:

  • 1
    High-dose chemotherapy with bone marrow transplantation for metastatic breast cancer. This treatment produced high rates of response in phase 2 trials but was proved to be no more effective than standard chemotherapy and was more toxic.2–4
  • 2
    Hormone replacement in menopausal females, which was eventually shown to be not beneficial to cardiovascular health and to be associated with multiple serious adverse outcomes.5, 6
  • 3
    Erythropoietin for the treatment of anemia due to chronic kidney disease. Observational studies had shown increased survival and higher hemoglobin levels; however, randomized trials were unable to show such benefits and found an association with increased rates of death, nonfatal heart attacks, strokes, and heart failure.7
  • 4
    Early initiation of hemodialysis versus late initiation. A number of observational studies suggested that starting dialysis early could improve the survival and quality of life of patients. Although this practice was widely employed by nephrologists throughout the United States with substantial financial advantages, a recent randomized controlled trial has found that the early initiation of dialysis is not associated with improvements in survival or clinical outcomes.8

In each of these contemporary examples, observational studies suggested a beneficial effect that was widely promoted in medicine but was proven false by a randomized controlled trial.

Even more bothersome is the recent report by Zuckerman et al.,9 who reported that the US Food and Drug Administration (FDA) recalled 113 medical devices between 2005 and 2009 because they were found to pose a high risk to patients and were not rigorously studied before they were cleared for sale by the FDA. Most of these devices were approved only on the basis of observational studies and were not subjected to sound scientific methodology.

Although observational studies often are cheaper, quicker, and less difficult to perform, we should not lose sight of the simple fact noted by Pocock and Elbourne10: “ignorance calls for careful experimentation.” This means high-quality randomized controlled trials and not observations that reflect personal choices or beliefs that often rely on bogus assumptions and mathematical arguments to prove what is not true to be true. In this era of money-based medicine and FDA laxity, when the ethics of pharmaceutical companies are continually being questioned and the costs of treatment continue to skyrocket, let us continue to evaluate health care interventions by the most scientifically sound and rigorous methods available.


  1. Top of page
  • 1
    Lindor RA, Lindor KD. The value of observational research in liver disease. HEPATOLOGY 2011; 53: 1-3.
  • 2
    Echt DS, Liebson PR, Mitchell LB, Peters RW, Obias-Manno D, Barker AH, et al. Mortality and morbidity in patients receiving encainide, flecainide, or placebo. The Cardiac Arrhythmia Suppression Trial. N Engl J Med 1991; 324: 781-788.
  • 3
    Stadtmauer EA, O'Neill A, Goldstein LJ, Crilley PA, Mangan KF, Ingle JN, et al. Conventional-dose chemotherapy compared with high-dose chemotherapy plus autologous hematopoietic stem-cell transplantation for metastatic breast cancer. Philadelphia Bone Marrow Transplant Group. N Engl J Med 2000; 342: 1069-1076.
  • 4
    Rettig RA, Jacobson PD, Farquhar CM, Aubry WM. False Hope: Bone Marrow Transplantation for Breast Cancer. Oxford, United Kingdom: Oxford University Press; 2007.
  • 5
    Moseley JB, O'Malley K, Petersen NJ, Menke TJ, Brody BA, Kuykendall DH, et al. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med 2002; 347: 81-88.
  • 6
    Rossouw JE, Anderson GL, Prentice RL, LaCroix AZ, Kooperberg C, Stefanick ML, et al. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women's Health Initiative randomized controlled trial. JAMA 2002; 288: 321-333.
  • 7
    Food and Drug Administration. Public advisory: erythropoiesis-stimulating agents including Procrit, Epogen, and Aranesp.
  • 8
    Cooper BA, Branley P, Bulfone L, Collins JF, Craig JC, Fraenkel MB, et al. A randomized controlled trial of early versus late initiation of dialysis. N Engl J Med 2010; 363: 609-619.
  • 9
    Zuckerman DM, Brown P, Nissen SE. Medical device recalls and the FDA approval process. Arch Intern Med; doi:10.1001/archinternmed. 2011.30.
  • 10
    Pocock SJ, Elbourne DR. Randomized trials or observational tribulations? N Engl J Med 2000; 342: 1907-1909.

Russell H. Wiesner M.D.*, * Mayo Clinic, Rochester, MN.