A Hungarian example for handsearching specialized national healthcare journals of small countries for controlled trials. Is it worth the trouble?
Dr Dániel Bereczki, Department of Neurology, University of Debrecen, Medical School, Nagyerdei krt. 98., H-4012 Debrecen, Hungary. E-mail: email@example.com
The objective of this study was to determine whether a Hungarian language journal of clinical neuroscience, which is not indexed in medline, contains any reports of randomized or quasi-randomized clinical trials that are not also reported in medline-indexed publications. A cover-to-cover handsearch was performed of the journal Clinical Neuroscience/Ideggyógyászati Szemle, the official journal of Hungarian societies of neurology, neurosurgery, psychiatry and related clinical neurosciences, from its first volume in 1950 to the end of 1998; and a search of medline for other reports of the trials identified. Clinical trials in which patients were allocated to interventions by randomized, quasi-randomized or possibly randomized means were identified. Controlled trials were tabulated and coded as randomized controlled trials or controlled clinical trials according to the definitions of the Handsearch Manual of the Cochrane Collaboration. We identified three randomized and eight quasi-randomized trials. Six of the 11 trials were not reported in medline indexed publications.
In conclusion, to obtain a comprehensive controlled trials register, specialized healthcare journals printed in small numbers in the national languages of small countries should be searched.
Systematic reviews of the effects of health care interventions should include all available evidence. Because randomized controlled trials (RCT) provide the most reliable evidence, the Cochrane Collaboration collects randomized, quasi-randomized and possibly randomized clinical trials, which are referred to as controlled clinical trials (CCT). These are made available in the Cochrane Controlled Trials Register. 1 In order to identify the largest possible number of controlled trials a systematic cover-to-cover handsearch of healthcare journals is recommended by the Collaboration to be performed back to the first volume of the journals, or to 1948 or before. 2
Excluding non-English language publications from systematic reviews might result in neglecting a considerable amount of evidence. 3 It may also lead to a bias in favour of including more studies with positive results, because authors have been shown to be more likely to publish RCTs in English language journals if the results are statistically significant. 4 Therefore, it is important to try to include trials published in non-English languages in systematic reviews. 5 In addition to finding reports of trials in the languages of large nations, handsearching of medline-indexed journals from smaller countries has also identified a number of controlled trials. 6 It has not been reported whether handsearching non- medline-indexed specialized journals printed in small numbers in national languages of small countries does or does not yield a reasonable number of reports of RCTs and CCTs not published in medline-indexed journals also. In the present study we performed a cover-to-cover handsearch for RCTs and CCTs of all volumes of a Hungarian language journal Clinical Neuroscience/Ideggyógyászati Szemle, which is the official journal of Hungarian societies of neurology, neurosurgery, psychiatry and related clinical neurosciences.
A cover-to-cover handsearch was performed of the Hungarian journal Clinical Neuroscience/Ideggyógyászati Szemle from its first volume in 1950 to the end of 1998. Because it was a cover-to-cover search, not only full-length papers, but letters, society reports, abstracts of meetings, etc. were also checked. Trials were identified and coded as randomized controlled trials (RCT), or controlled clinical trials, according to the definitions of the Cochrane Handsearch Manual.
To check whether the definitely randomized or quasi-randomized trials identified by handsearch were also reported in medline-indexed articles, we searched medline for the names of all authors on the trials using the ‘Advanced medline search’ option of PubMed. 7 The search was performed twice. The first search was run in July 1999 using the family name and the first initial of all authors of each publication. Separate searches were performed one-by-one for each author. Not only the titles but also the text of all abstracts was read, because it is not always possible to decide the content of the paper only from the titles. The second search was run in February 2000. This search was performed in two steps. In the first step the family names without initials of all authors for each identified randomized or quasi-randomized trial was combined with OR, and the disease and the intervention was added to the query using AND after the authors’ names and using OR between the disease condition and the intervention. This search was run separately for each publication. If this search identified a medline-indexed corresponding publication for any of the reports identified by handsearch, then no further search was done for that study. For those reports where the above search yielded no result, in the second step we combined the family names without initials of all authors by OR, and ran the search separately for each report. The results were then checked against the original report identified by handsearch.
The handsearch results of all pages from the first issue in 1950 to the last one in 1998 of the Hungarian-language journal of clinical neuroscience Clinical Neuroscience/Ideggyógyászati Szemle is summarized in Table 1. There were more than 2400 full length articles plus numerous abstracts and other reports. We found three definitely randomized trials, 8–10 eight trials in which a method of quasi-randomization was described, 11–18 and a further 16 trials that were possibly randomized or quasi-randomized. Bibliographic data of these trials are now available in the Cochrane Controlled Trials Register. 1 The 11 trials where patient allocation was explicitly stated to be random or quasi-random ranged in size from seven to 120 participants (median: 33). Two of the three RCTs were published as full length papers. 8,9 The other was a conference abstract. 10
Table 1. The number of various types of controlled trials in Clinical Neuroscience/Ideggyógyászati Szemle.
Two medline searches were performed to investigate whether the 11 definitely randomized or quasi-randomized trials (three RCTs and eight CCTs) had been published in medline indexed journals. Two of the three randomized trials 9,10 also appeared in medline. 19,20 The only RCT not found elsewhere was a small, Latin square cross-over trial in seven healthy volunteers. 8 In the first search (July 1999) one of the eight quasi-randomized trials 12 could be identified in medline as a part of a more complex paper, 21 and another one with 22 patients 16 was a pilot study for a large multicentre trial involving 255 patients. 22 The second search (February 2000) yielded one more result, 23 which was a paper published in 1999 and could not be identified by the first search because at that time the report was not indexed on medline. The handsearch-identified version of that report was an abstract, 17 a preliminary report with 20 patients, whereas the final medline-indexed report involved 40 cases.
The number of RCTs and CCTs compared to the total number of papers published in Clinical Neuroscience/Ideggyógyászati Szemle was relatively small, but this is comparable to the proportion of RCTs and CCTs found in at least one specialized English language journal that is indexed in medline. 24 The Hungarian trials were usually small. Two of the three RCTs and one quasi-randomized trial could also be identified in English language papers published in medline-indexed journals. In addition, two more quasi-randomized trials found by handsearch were preliminary results of larger trials, reports of which appeared in medline 1 and 4 years later. Because five of the eight quasi-randomized trials and one of the RCTs could not have been identified by medline search, we suggest that to obtain a comprehensive controlled trials register, specialized medical journals printed in small numbers in the national languages of small countries should be searched.
D. Bereczki was supported by the Széchenyi Grant from the Ministry of Education, Republic of Hungary. The handsearch activity was partly supported by the Soros Foundation, Hungary. The authors thank Dr Mike Clarke (UK Cochrane Centre, Oxford) for his comments on a previous version of this manuscript.