Science or audit?

The philosophy of the BJOG is the publication of the highest quality scientific research in obstetrics, gynaecology and women's health, in order to understand better the pathophysiology of disease affecting women and to search for effective treatments. For this reason we rarely publish papers on audit of clinical practice, for in a scientific sense audit is inherently flawed, relying as it does on comparison with historical controls and the consequent intractable problems of confounding variables. Audit therefore finds its way infrequently into scientific journals, being more at home in journals of health services research; so why do we publish a paper on audit in this issue?

We justify it where the subject is of great importance to obstetricians and gynaecologists, where publication of an audit will illuminate controversies which are fundamental to the provision of a service, and where the audit is large enough that its results may be applied to a nation and not just to a region. In the United Kingdom the fundamental controversy in treating gynaecological cancer is the concentration of therapy in cancer centres, which is part of a national strategy but which has not so far been shown to be effective in improving the outcome. In 1997 John Murdoch and his colleagues conducted a retrospective survey of the treatment of carcinoma of the cervix in South West England which discovered several apparent deficiencies; the paper in this issue (pages 308–315) complements that study, by describing a prospective survey of the treatment of carcinoma of the cervix in 1997, to measure any improvement by completing the audit cycle. And there were differences between the two surveys: in 1997 preoperative evaluation was more thorough and formal assessment of the stage of the disease was performed more frequently; radical hysterectomy was carried out more often in cancer centres; pelvic lymphadenectomy was carried out more completely; the greater the number of operations undertaken by the surgeon the greater the number of lymph nodes removed; and fewer women underwent inappropriate conservative surgery.

These are glowing results, but we should remind ourselves of the scientific limitations of a study of audit. There is an important contrast in the two surveys, the first being retrospective where much information was absent from the case notes, the second prospective where the information was gathered by use of a form. The improvement in performance may therefore be more apparent than real. The comparison of 1997 was with historical controls in 1989 and 1993, and so any differences may be due more to occult factors beyond the control of the investigators and less to the process of the audit. Above all, it was too early to measure the association of the standards introduced in this audit and the survival of the women. Treatment of cancer is by unsophisticated methods, and it is possible that the biological aggression of a cancer may defeat any apparent improvement in standards of treatment brought about by audit and guidelines. The treatment of cancer today is at a stage similar to the treatment of tuberculosis before the Second World War. Standard treatments of tuberculosis before 1948 resulted in modest improvements in outcome, or were harmful, until basic scientific research brought about the manufacture of a cure, streptomycin; so too standard treatments of cancer in 2000 result in modest improvements in outcome, or are harmful, until basic scientific research brings about the manufacture of a cure.

Until we find this cure, research will be directed towards randomised trials and observational studies. It is unrealistic to perform a randomised trial to compare treatment in cancer centres with controls, and so we rely upon surveys of treatment to direct us. Sometimes prospective surveys give convincing evidence of beneficial interventions: thus in several surveys there is such a clear relationship between the introduction of cervical screening and declining mortality from carcinoma of the cervix that we are convinced of the benefits of this screening test. So too we may be convinced of the value of cancer centres from the results of several observational studies, if they agree, where the outcome is mortality. We therefore await with interest the third survey of John Murdoch and his colleagues.