Design and statistical analysis of observational studies



In reference to the study by Awwad et al.[1] published in the October issue of BJOG, I would like to raise a few pertinent issues concerning the methodology.

The authors state that the design was that of a prospective cohort study. It is also stated that the participant recruitment involved ‘a rigid process of patient matching by recruiting the controls who matched study women at every level of known confounding variables including age, period of gestation, parity and BMI’ and the controls were recruited within the same or on the following day. However, data on the leftover cohort—those not matched with the study participants—are not accounted for. Therefore, to be precise, the design does not fulfil the characteristics of a conventional prospective cohort. However, the sequence here was from cause (fasting) to effect (preterm delivery) unlike that of a conventional ‘case–control study’ or nested case–control design (retrospective) with a portion of underlying cohort data missing. The current study presumably employed, a prospectively ‘matched’ case–cohort design with the exposure identified before outcome though, conceptually, controls are to be selected randomly in this design.

However, the main reason for concern in this study is the statistical method employed by the authors, which ignores the matched-controls design. Matching is employed in observational studies to reduce or eliminate the effects of confounding factors, i.e. to control variation due to extraneous variables at sampling stage; pairing reduces bias and increases precision of the statistical inference, especially when the sample size is small. There are two methods of matching: individual matching and frequency matching. When controls are matched to cases on one or more attributes like age, BMI, parity, etc. it amounts to individual matching. Whenever two groups are closely matched on potential confounders, the two samples are considered dependent (‘independent’ when they are not matched) because they are paired by design. To analyse the results, a procedure that accounts for the sample match is therefore required. Odds ratio is estimated as the ratio of case-positive to case-negative matched pairs. To test the statistical significance of the difference between paired means, a matched-pairs t-test is applied. McNemar's test is used as a normal approximation to the binomial test with correction for discontinuity. Whereas in the current study Student's t-test, Wilcoxon rank-sum test and a multivariate logistic regression were applied, thereby ignoring the matched design of the study; this may result in odds ratios biased toward conservation.

The ‘confusion’ in the biomedical literature about case–control study nomenclature has been previously highlighted.[2] Niven et al.,[3] in a structured review, analysed 37 matched case–control studies published in peer-reviewed journals and found that statistical methods were inappropriate in more than half of the studies they reviewed. Though one cannot be sure that use of proper statistics would change the conclusions reached by the studies that employed improper statistical methods, the observation made by the authors remains cogent.

To conclude, appropriateness of statistical procedures for the study design needs to be validated by stringent statistical methodology reviews by the authors, reviewers and editors. In this context, it is noteworthy that a recent article suggested an association of the strength of study design and use of recommended analytic techniques with the journal's impact factor.[4]