The paper1 has serious weaknesses that should not allow the assertion that Menevit has been clinically proven to double pregnancy rates in couples. The trial was conducted to determine if Menevit would increase the average number of good or excellent quality embryos with intracytoplasmic sperm injection from an expected two in the placebo group to three in the Menevit group. A double-blind random 2:1 allocation to verum and placebo was used with a total sample size of 60 patients selected for evidence of DNA damage in sperm. Although the data are not given or analysed explicitly, the proportions of excellent, good and poor embryos (Table 3) are similar in the placebo and Menevit groups. The conclusion should have been that the trial result provided no evidence that Menevit treatment improved embryo quality.
The so-called secondary endpoint of viable pregnancy is actually an implantation rate (number of fetal hearts continuing past 13 weeks per embryo transferred) and is only just significant, P < 0.05. This was then used to claim that Menevit doubles the pregnancy rate. This can be criticised from several angles. Statistically it was not the stated primary endpoint of the trial and it looks as if it has been developed during analysis and reported because the P-value is < 0.05. In fact, the authors acknowledged ‘preliminary power calculations using pregnancy rates revealed a sample size too large for a single site trial’. Thus, they should have been aware that this was an unusual result. The results for the placebo group seem low and those for the Menevit are similar to those reported for the clinic on the website (http://www.repromed.com.au/content.php?action=6). There is a multiple testing problem that increases the risk of a type 1 error: inappropriate rejection of the null hypothesis. In table 4, three related implantation or pregnancy outcomes are analysed and the result should not be considered significant unless at least P < 0.015 to allow for the multiple testing. As the number of embryos transferred varied between patients and some conceived twins, there are repeated measures and the χ2 test is not appropriate. The continuing pregnancy rate (singleton or twin) per patient for which a χ2 test is appropriate is not significantly different between the two arms (17 of 36, 47%, and four of 16, 25%). There is no information or adjustment for important covariates of implantation rate such as female age, previous assisted reproduction technology births, and number of embryos transferred. Also embryos at various stages were transferred. Such variability could bias implantation rates. The 2:1 randomisation and the dropout of four subjects in the placebo arm resulted in a very small placebo group in the trial; also, two of the six clinical pregnancies in this group miscarried. In both groups, embryos were cryopreserved but the outcomes of transfers of these embryos are not given. Plausibility is also lacking. If antioxidants worked, there should be generally positive results from trials with improvement particularly in sperm motility yet as indicated in the discussion that the reports are inconsistent. It is peculiar that no data are reported on change in semen analysis or TUNEL results for the trial patients. Was there no change?
There are other more minor issues. The cut-off values are not given for patient selection: poor morphology, motility or viability. The sample size calculation seems too small to detect an increase in the number of good embryos by 1. It seems strange the males are claimed to have severe male factor infertility yet the mean sperm concentration is above 20 million/mL. There is a statement that one placebo subject had no fertilisation which causes an inconsistency in the numbers (15 not 16). It is not clear in table 2 how many subjects were in the groups and, particularly, if all 20 placebo and 40 verum subjects had all the tests. There is a trend to lower oocyte numbers, less mature oocytes and lower fertilisation rates (table 3), and fewer multiple pregnancies (table 4) that contribute to the lower implantation rate in the placebo arm, yet these are mainly influenced by female factors.
Large multicentre double-blind placebo-controlled trials by independent investigators should be performed to confirm if this agent is beneficial before it is sold to patients.