SEARCH

SEARCH BY CITATION

Keywords:

  • Appraisal instruments;
  • endometriosis;
  • guideline;
  • review

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. References

Background  Guidelines exist for the management of endometriosis. Validated and reliable appraisal tools exist to assess the quality of guidelines.

Objectives  To systematically appraise the quality of guidelines for the management of pelvic pain associated with endometriosis.

Search strategy  Guidelines were identified using a prospective protocol through a systematic search of MEDLINE (1951–2005), EMBASE (1974–2005), the Cochrane Library (2005, issue 2), known guideline websites and the World Wide Web.

Selection criteria  Type of document: guideline, consensus statement, care protocol or healthcare technology assessment produced by national or international professional organisations and societies or governmental agencies; subject: managemant of pelvic pain associated with endometriosis.

Data collection and analysis  Two validated appraisal tools, Cluzeau and The Appraisal of Guidelines and Research and Evaluation (AGREE), were used to quantitatively assess the quality of guidelines. Areas evaluated included ‘rigour of development’, ‘context and content’ and ‘application.’

Main results  Eight of 596 potentially relevant citations identified met our inclusion criteria. The Cluzeau instrument quality score were the following: rigour of development, 53% (range 5–65%); context and content, 69% (range 29–79%) and application 20% (range 0–20%). The aplication dimension achieved significantly lower quality scores (P = 0.026 versus rigour of development and P = 0.017 versus context and content). The AGREE instrument quality scores were the following: scope and purpose, 68% (range 17–89%); stakeholder involvement, 33% (range 13–63%); rigour of development, 49% (range 10–81%); clarity of presentation, 55% (range 42–67%); applicability, 14% (range 0–28%) and editorial independence, 28% (range 8–67%). The applicability domain achieved significantly lower quality scores (P = 0.001 versus scope and purpose and P = 0.009 versus rigour of development).

Author’s conclusions  Guidelines for the management of pelvic pain associated with endometriosis do not comply with the recommendations for high-quality standards.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. References

Endometriosis is a common gynaecological condition among women of reproductive age that results in an array of problems including pelvic pain, dysmenorrhoea, dyspareunia and subfertility. There is a lack of consensus about its best management, particularly in dealing with symptoms of chronic pelvic pain. Guidelines are systematically developed statements to assist practitioner and patients’ decisions about appropriate health care for specific clinical circumstances.1 Their specific purpose is to make explicit recommendations with a definite intent to influence what clinicians do.2 It is believed that guidelines can improve the quality, appropriateness and cost-effectiveness of health care in endometriosis.

The methodology for development of high-quality guidelines requires that the recommendations should be evidence based. Validated and reliable appraisal tools3,4 can be used to assess the quality of a guideline, which may provide an assessment of its predicted validity, i.e. the likelihood that it will achieve its intended outcome. However, previous studies of guideline quality have shown that they are often poor in compliance with criteria concerning applicability, i.e. the likely organisational, behavioural and cost implications of applying the guideline. We identified existing guidelines for the management of pain associated with endometriosis and assessed their quality using two such tools, the Cluzeau3 and the Appraisal of Guidelines and Research and Evaluation4 (AGREE) instruments. We also examined if their compliance with quality criteria concerning scope or content and rigour of development was better than that with the criteria concerning applicability of recommendations.

Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. References

Our study was carried out using a prospective protocol based on recommended methods for search and assessment of guidelines.5

Identification of guidelines

General bibliographic medical databases, MEDLINE (1951–2005), EMBASE (1974–2005) and the Cochrane Library (2005, issue 2), were systematically searched. The search term combination captured the concept ‘practice-guidelines AND endometriosis’ using a wide range of indexing terms, free text words and word variants. In addition, relevant databases of guidelines including the National Guideline Clearing House, NeLH Guidelines Website and the Geneva Foundation for Medical Education and Research were searched. We also searched the World Wide Web using metasearch engines for national and international guidelines, which often resulted in being directed to the same databases. The electronic search strategy targeted relevant citations of guidelines concerned with any aspect of endometriosis management. From these, guidelines addressing the management of pelvic pain were identified after examination of all the retrieved manuscripts. Reference lists of all retrieved articles were checked and direct contact with experts and international organisations/colleges in various countries was made to identify guidelines not captured by electronic searches.

Citations and manuscripts were selected if they met the following criteria—type of document: guideline, consensus statement, care protocol or healthcare technology assessment produced by national or international professional organisations and societies or governmental agencies; subject: management of pelvic pain associated with endometriosis. There were no language restrictions in our search or selection.

The guidelines were identified in a two-stage process. The titles and abstracts identified as being potentially relevant from the computer-based searches, inspection of bibliographies and direct contact with organisations and experts were included unless they could definitely be excluded as not addressing the management of pelvic pain associated with endometriosis. The full texts of all the selected citations were retrieved. Final inclusion/exclusion decisions were made with reference to a checklist by two reviewers (T.-L.A. and C.H.M.). Disagreements about inclusion/exclusion were initially resolved by consensus between the two reviewers (T.-L.A. and C.H.M.) and, where this was not possible, by arbitration by a third reviewer (K.S.K.).

Quality assessment

All the guidelines which met the inclusion criteria were evaluated using two validated appraisal tools,3,4 which quantitatively assess the quality defined as the confidence that potential biases of guideline development had been addressed adequately and that the recommendations were both internally and externally valid. Cluzeau et al.3 developed a validated quality appraisal instrument using 37 items divided into three dimensions: rigour of development (items 1–20), context and content (items 21–32) and applicability (items 33–37). Details of the individual item questions are shown in Table 1, using a low-quality guideline as an example. Rigour of development included assessment of the composition of the guideline development group to evaluate if all key stakeholders are represented and assessment of the methodology for identification and interpretation of published evidence, formulation of recommendations and linking of evidence. The context and content dimension addressed the aims and objectives, the target groups, the clinical context in which the guideline is applicable, possible alternative management options, presentation of the guideline and an estimation of benefits, harm and costs. The application dimension considered dissemination and implementation strategies, standards and targets and advice on monitoring. Responses to each item were ‘yes’, ‘no’, ‘not sure’ or ‘not applicable’. Dimension scores were calculated by summation of items, where a value of 1 is given for a ‘yes’ and all other responses were given a value of 0, and standardised as a percentage of the maximum possible score for the dimension. It is thus possible to achieve a quality score ranging between 0 and 100%.

Table 1.  Application of the Cluzeau instrument3 to a low-quality guideline6
ItemAnswer
  1. N/A, not applicable

Rigour of development
 1. Is the agency responsible for the development of the guidelines clearly identified?Yes
 2. Was external funding or other support received for developing the guidelines?Not sure
 3. If external funding/support was received, is there evidence that the potential biases of the funding body were taken into account?N/A
 4. Is there a description of the individuals (e.g. professionals, interest groups—including patients) who were involved in the guidelines?No
 5. If so, did the group contain representatives of all key disciplines?N/A
 6. Is there a description of the sources of information used to select the evidence on which the recommendations are based?Yes
 7. If so, are the sources of information adequate?No
 8. Is there a description of the method(s) used to interpret and assess the strength of the evidence?No
 9. If so, is/are the method(s) for rating the evidence satisfactory?N/A
10. Is there a description of the methods used to formulate the recommendations?No
11. If so, are the methods satisfactory?N/A
12. Is there an indication of how the views of interested parties not on the panel were taken into account?No
13. Is there an explicit link between the major recommendations and the level of supporting evidence?No
14. Were the guidelines independently reviewed prior to their publication/release?Not sure
15. If so, is explicit information given about methods and how comments were addressed?N/A
16. Were the guidelines piloted?Not sure
17. If the guidelines were piloted, is explicit information given about the methods and the results adopted?N/A
18. Is there a mention of a date for reviewing or updating the guidelines?No
19. Is the body responsible for the reviewing and updating clearly identified?N/A
20. Overall, have the potential biases of guideline development been adequately dealt with?No (2/20)
Context and content
21. Are the reasons for developing the guidelines clearly stated?Not sure
22. Are the objectives of the guidelines clearly defined?No
23. Is there a satisfactory description of the patients to whom the guidelines are meant to apply?No
24. Is there a description of the circumstances (clinical or nonclinical) in which exceptions might be made in using the guidelines?No
25. Is there an explicit statement of how the patient’s preferences should be taken into account in applying the guidelines?No
26. Do the guidelines describe the condition to be detected, treated or prevented in unambiguous terms?Yes
27. Are the different possible options for management of the condition clearly stated in the guidelines?Yes
28. Are the recommendations clearly presented?Yes
29. Is there an adequate description of the health benefits that are likely to be gained from the recommended management?Not sure
30. Is there an adequate description of the potential harms or risks that may occur as a result of the recommended management?No
31. Is there an estimate of the costs or expenditure likely to incur from the recommended management?No
32. Are the recommendations supported by the estimated benefits, harms and costs of the interventions?No (3/12)
Application
33. Does the guideline document suggests possible methods for dissemination and implementation?No
34. Does the guideline document specify criteria for monitoring compliance?No
35. Does the guideline document identify clear standards or targets?No
36. Does the guideline document define measurable outcomes that can be monitored?No
37. Does the guideline document identify key elements that need to be considered by local guideline groups?No (1/5)

The AGREE instrument was developed and tested internationally.4 It emerged following discussions between researchers with extensive experience and knowledge of guidelines from several countries. It assesses both the quality of the reporting and various aspects of recommendations. It contains 23 items grouped into six quality domains (see below) with a 4-point Likert scale score ranging from 4 (strongly agree) to 1 (strongly disagree) for each item. Each appraiser allocates a score between 1 and 4 for each item, and a total domain score was calculated. Details of the individual item questions are shown in Table 2, using a high-quality guideline as an example. Scope and purpose (items 1–3) is concerned with the overall aim of the guideline, the specific clinical questions and the target population. Stakeholder involvement (items 4–7) focuses on the extent to which the guideline represents the views of its intended users. Rigour of development (items 8–14) relates to the process used to gather and synthesise the evidence, the methods to formulate the recommendations and to update them. Clarity and presentation (items 15–18) deals with the language and format of the guideline. Applicability (items 19–21) pertains to the likely organisational, behavioural and cost implications of applying the guideline. Editorial independence (items 22–23) is concerned with the independence of the recommendations and acknowledgement of possible conflict of interest from the guideline development group. A total domain score was calculated by summation of its items and standardised using the prescribed equation: [(obtained score−minimum possible score)/(maximum possible score−minimum possible score)] × 100, where maximum possible score was 4 (strongly agree) × number of items × number of appraisers, and minimum possible score was 1 (strongly disagree) × number of items × number of appraisers. It is thus possible to obtain a quality score ranging between 0 and 100%.

Table 2.  Application of the AGREE instrument4 to a high-quality guideline7
ItemScore
4321
Scope and purpose
 1. The overall objective(s) of the guideline is (are) specifically described  
 2. The clinical question(s) covered by the guideline is (are) specifically described 
 3. The patients to whom the guideline is meant to apply are specifically described  
 10/12
Stakeholder involvement
 4. The guideline development group includes individuals from all the relevant professional groups  
 5. The patients’ views and preferences have been sought  
 6. The target users of the guideline are clearly defined 
 7. The guideline has been piloted among target users 
 10/16
Rigour of development
 8. Systematic methods were used to search for evidence  
 9. The criteria for selecting the evidence are clearly described  
10. The methods used for formulating the recommendations are clearly described 
11. The health benefits, adverse effects and risks have been considered in formulating the recommendations  
12. There is an explicit link between the recommendations and the supporting evidence  
13. The guideline has been externally reviewed by experts prior to its publication  
14. A procedure for updating the guideline is provided 
 18/28
Clarity and presentation
15. The recommendations are specific and unambiguous  
16. The different options for management of the condition are clearly presented  
17. Key recommendations are easily identifiable  
18. The guideline is supported with tools for application  
 12/16
Applicability
19. The potential organisational barriers in applying the recommendations have been discussed  
20. The potential cost implications of applying the recommendations have been considered  
21. The guideline presents key review criteria for monitoring and/or audit purposes 
 5/12
Editorial independence
22. The guideline is editorially independent from the funding body  
23. Conflicts of interest of guideline development members have been reported  
 6/8

The assessments were performed independently by two reviewers (T.-L.A. and C.H.M.) using piloted data extraction forms with accompanying explanatory notes. The concordance between the two reviewers was computed using percentage of agreement and weighted kappa statistics.8,9 Weighted kappa statistics allow for measuring observer agreement in rank scales, taking into account chance agreement. The level of agreement between the reviewers was excellent (weighted kappa 0.84).

Data synthesis

The mean scores for the various dimensions and domains were computed for each selected guideline. A composite score was not produced as there was no consensus about the weighting to be used for such a combination. Comparison of median scores for dimensions and domains among appraised guidelines employed the Mann–Whitney U test as data were not normally distributed.

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. References

Guideline search and selection

Figure 1 shows the search and selection process. Electronic and guideline databases generated 596 citations, but only 32, 34 and 31 from MEDLINE, EMBASE and the guideline databases, respectively, were suitable for more detailed evaluation. Contacting experts directly identified three citations, of which one was a duplicate from the electronic searches. Searching World Wide Web identified 27 potentially relevant records of guidelines. Review of references did not identify any further guidelines. A total of 95 manuscripts were retrieved or downloaded for detailed evaluation. Exclusions were made because the development group was inappropriate, the manuscript was not a guideline or they were in duplicate. In total, eight guidelines were identified that met the inclusion criteria.6,7,10–16 Six were written in English and two required translation from Dutch6 and Finnish.15 Publication dates ranged from 1999 to 2005 as shown in Table 3.

image

Figure 1. Study selection process for systematic appraisal of guidelines for management of pain associated with endometriosis.

Download figure to PowerPoint

Table 3.  Details of selected guidelines on the management of endometriosis
NameCountryYearOrganisationTypeTitle
RCOG10UK2000Royal College of Obstetricians and GynaecologistsGuidelineEndometriosis—investigation and management (24)
PROD11UK2003ProdigyGuidelineProdigy guidance—endometriosis
USA7USA2002Chronic Pelvic Pain/Endometriosis Working GroupConsensus statementConsensus statement for the management of chronic pelvic pain and endometriosis: proceedings of an expert panel consensus process
ACOG12USA1999American College of Obstetricians and GynecologistsGuidelineACOG practice bulletin. Medical management of endometriosis
SOGC13,14Canada1999Society of Obstetricians and Gynaecologists of CanadaConsensus statementThe Canadian Consensus Conference on Endometriosis
NVOG6Netherlands2001Nederlandse Vereniging voor Obstetrie en GynaecologieGuidelineDiagnosis and management of endometriosis (no. 33)
FIN15Finland2001Finnish Office for Healthcare Technology AssessmentHealthcare technology assessmentTreatment of pain caused by endometriosis (Fin OHTA report 19)
ESHRE16Europe2005European Society for Human Reproduction and EmbryologyGuidelineESHRE guideline for the diagnosis and treatment of endometriosis

Appraisal of guidelines by the Cluzeau instrument

The median quality scores for the three dimensions were 52.5, 68.8 and 20% (range of quality scores 0–79.2%) as shown in Figure 2 and Table 4. The highest quality scores were seen in the context and content dimension. The quality scores for the application dimension were significantly poorer compared with those for rigour of development (P= 0.0172) and context and content (P= 0.0002).

image

Figure 2. Quality of guidelines for the management of pain associated with endometriosis using the Cluzeau instrument. (See Methods for details of dimension score results. See Table 1 for details of guidelines.)

Download figure to PowerPoint

Table 4.  A comparison of quality scores from review studies of the quality of guidelines using the Cluzeau instrument
Quality dimensions*Median (range) quality scores
Our study (endometriosis guidelines [2005])Cluzeau et al.3 (53 varied guidelines [1999])Coomarasamy et al.17 (RCOG Green Top Guidelines [2003])McDonagh et al.18 (SOGC produced guidelines [2002])Cranney et al.19 (osteoporosis guidelines [2002])
  • Current study results are represented by bold values. Last four columns contain scores reproduced from other published studies using the same instrument (see Results for details).

  • *

    See Methods for details.

Dimension 1: Rigour of development53 (5–65)31 (0.8–85)5015 (5–45)23 (5–80)
Dimension 2: Content and context69 (29–79)48 (9.7–78.3)6747 (14–81)58 (17–83)
Dimension 3: Application20 (0–20)24 (0–95)2027 (7–67)0 (0–47)

Appraisal of guidelines by the AGREE instrument

The median quality scores for the six domains were 72.5, 25, 50, 54, 11 and 24% (range of quality scores 0–89%) as shown in Figure 3 and Table 5. Overall, the applicability domain achieved the lowest quality scores, whereas scope and purpose domain achieved the highest quality scores. These results were significant when compared with scope and purpose (P= 0.0009), stakeholder involvement (P= 0.0457), rigour of development (P= 0.0092), clarity and presentation (P= 0.0002).

image

Figure 3. Quality of guidelines for the management of pain associated with endometriosis using the AGREE instrument. (See Methods for details of domain score results. See Table 1 for details of guidelines.)

Download figure to PowerPoint

Table 5.  A comparison of quality scores from review studies of the quality of guidelines using the AGREE instrument
Quality domains*Mean quality scores
Our study (endometriosis guidelines [2005])Burgers et al.20 (nononcology guidelines [2004])Burgers et al.20 (oncology guidelines [2004])
  • Current study results are represented by bold values. Last two columns contain scores reproduced from other published studies using the same instrument (see Results for details).

  • *

    See Methods for details.

Domain 1: Scope and purpose686663
Domain 2: Stakeholder involvement333034
Domain 3: Rigour of development493042
Domain 4: Clarity and presentation555257
Domain 5: Applicability143026
Domain 6: Editorial independence284147

Comparison of the results of our study was made between other published guideline reviews. Table 4 compares the median dimension quality scores for our study with the scores of other guideline reviews available in the literature using the Cluzeau instrument.3,17–19Table 5 compares the mean quality scores for each domain of the AGREE instrument of our study with the results of a study by Burgers et al.,20 who assessed the quality of guidelines produced for both oncology and nononcology subjects. Overall, our study performed favourably in almost all areas using both appraisal instruments.

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. References

This study is the first to evaluate the methodological quality of guidelines for the management of pain associated with endometriosis. Our findings indicate that the quality of these guidelines fall below the expected high standards, particularly in the application dimension and domain, thereby limiting the likelihood of implementation of the recommendations made.

The validity of our findings depends on the rigour of our methods. We used a prospective protocol, an exhaustive search and validated quality assessments. The use of two appraisal instruments was chosen since there was no evidence to support the exclusive use of any one instrument.21 The Cluzeau appraisal instrument has been in use since 1999 and is well recognised. The AGREE instrument is relatively new and the only instrument validated for use in appraising guidelines internationally.20 The Cluzeau instrument has received the greatest evaluation and is used by large guideline development groups. The use of both appraisal instruments allowed a comparison of our data with a wide range of other published guideline reviews (see below).3,17–20,22,23

Although the methodological quality of current endometriosis guidelines was low, the clinical advice contained in these documents was similar, although it was not always supported by evidence possibly due to lack of quality primary research in this field. Only three of eight guidelines met at least half of the quality standards in the rigour of development domain, with low scores attributable to a lack of external expert review, details of future updates and that risks have been considered in formulation of the recommendations. Moreover, the application dimension and domain concerned with dissemination and implementation strategies, standards, targets and advice on monitoring uniformly received the lowest quality scores. Little or no mention was made of these items, so the low scores may have been due to a lack of reporting. Some feel, however, that a lack of reporting can be an indicator for poor methodology.24 Dimension and domain scores allow comparison of guidelines and help inform decisions about subsequent use or recommendations. They do not assess the impact of a guideline on patients’ outcomes.

The increasingly important role of clinical practice guidelines as a support to clinical decision making is shown by the growing number of publications in the literature. Guidelines serve several objectives. They are the following: improvements in health care (increasing good and decreasing bad practices), greater availability of and access to medical knowledge and promotion of cost-effective strategies and can serve as valuable educational tools. They are also used as shields in medico-legal defence. They need to be free of biases, which could not only result in the denial of best care but also lead to harmful practices.2 Attempts to evaluate the quality of guidelines have generally shown poor adherence to the quality standards3,17–20 (Tables 4 and 5). This is of particular concern for guidelines produced by professional bodies as they reach a targeted audience that is generally expected to accept the recommendations as ‘the standard to be achieved’.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. References
  • 1
    Committee to Advise the Public Health Service on Clinical Practice Guidelines, Institute of Medicine. In: FieldMJ, LohrKN, editors. Clinical Practice Guidelines: Directions of a New Program. Washington, DC: National Academy Press; 1990. p. 38.
  • 2
    Hayward RS, Wilson MC, Tunis SR, Bass EB, Guyatt G. Users’ guides to the medical literature. VIII. How to use clinical practice guidelines. A. Are the recommendations valid? The Evidence-Based Medicine Working Group. JAMA 1995;274:5704.
  • 3
    Cluzeau FA, Littlejohns P, Grimshaw JM, Feder G, Moran SE. Development and application of a generic methodology to assess the quality of clinical guidelines. Int J Qual Health Care 1999;11:218.
  • 4
    AGREE collaboration Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care 2003;12:1823.
  • 5
    Centre for Reviews and Dissemination. CRD report 4—undertaking systematic reviews of research on effectiveness: CRD’s guidance for carrying out or commissioning reviews, 2nd edn. CRD reports. York, UK: The University of York, 2001.
  • 6
    Dutch Society of Obstetricians and Gynaecologists (NVOG). Diagnostiek en behandeling van endometriose (33). Richtlijn: Nederlandse Vereniging voor Obstetrie en Gynaecologie (NVOG) 2001.
  • 7
    Gambone J, Mittman B, Munro M, Scialli A, Winkel C. Consensus statement for the management of chronic pelvic pain and endometriosis: proceedings of an expert-panel consensus process. Fertil Steril 2002;78:96172.
  • 8
    Landis JR, Koch GG. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics 1977;33:36374.
  • 9
    Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:15974.
  • 10
    Royal College of Obstetricians and Gynaecologists. The investigation and management of endometriosis (24). Clinical Green Top Guidelines. 2000 [www.rcog.org.uk/guidelines.asp?PageID=106&Type=guidelines&GuidelineID=10]. Accessed July 2005.
  • 11
    PRODIGY PRODIGY Guidance—endometriosis. 2003 [www.prodigy.nhs.uk/guidance.asp?gt=endometriosis]. Accessed July 2005.
  • 12
    ACOG Committee on Practice Bulletins–Gynecology ACOG practice bulletin. Medical management of endometriosis. Number 11, December 1999 (replaces Technical Bulletin Number 184, September 1993). Clinical management guidelines for obstetrician-gynecologists. Int J Gynaecol Obstet 2000;71:18396.
  • 13
    Canadian Consensus Committee Society of Obstetricians and Gynaecologists of Canada. The Canadian Consensus Conference on Endometriosis. J Obstet Gynaecol Can 1999;21:574610.
  • 14
    Canadian Consensus Committee Society of Obstetricians and Gynaecologists of Canada (1998) The Canadian Consensus Conference on Endometriosis. J Obstet Gynaecol Can 1999;21:46898.
  • 15
    Setälä M, Hurskainen R, Kauko M, Kujansuu E, Tiitnen A, Vuorma S, et al. Endometrioosin aiheuttaman kivun hoito (Treatment of pain caused by endometriosis). FinOHTAn raportti 19. 2001 [www.stakes.fi/finohta/raportit/019/r019f.html]. Accessed July 2005.
  • 16
    Kennedy S, Bergqvist A, Chapron C, Hooghe T, Dunselman G, Greb R, et al. ESHRE guideline for the diagnosis and treatment of endometriosis. Hum Reprod 2005;20:2698704.
  • 17
    Coomarasamy A, Ola B, Gee H, Khan KS. Quality of brief guidelines produced by professional bodies: a study of the ‘green-top’ guidelines by the UK Royal College of Obstetricians and Gynaecologists. J Obstet Gynaecol 2003;23:47983.
  • 18
    McDonagh R, White C, Singh G, Mohide P. The quality of obstetrical clinical practice guidelines promulgated by a specialty society. J Obstet Gynaecol Can 2002;24:95762.
  • 19
    Cranney A, Waldegger L, Graham ID, Man-Son-Hing M, Byszewski A, Ooi DS. Systematic assessment of the quality of osteoporosis guidelines. BMC Musculoskelet Disord 2002;3:20.
  • 20
    Burgers JS, Fervers B, Haugh M, Brouwers M, Browman G, Philip T, et al. International assessment of the quality of clinical practice guidelines in oncology using the Appraisal of Guidelines and Research and Evaluation Instrument. J Clin Oncol 2004;22:20007.
  • 21
    Graham ID, Calder LA, Hebert PC, Carter AO, Tetroe JM. A comparison of clinical practice guideline appraisal instruments. Int J Technol Assess Health Care 2000;16:102438.
  • 22
    Lacasse Y, Ferreira I, Brooks D, Newman T, Goldstein RS. Critical appraisal of clinical practice guidelines targeting chronic obstructive pulmonary disease. Arch Intern Med 2001;161:6974.
  • 23
    Irani J, Brown CT, Van Der Meulen J, Emberton M. A review of guidelines on benign prostatic hyperplasia and lower urinary tract symptoms: are all guidelines the same? BJU Int 2003;92:93742.
  • 24
    Liberati A, Himel HN, Chalmers TC. A quality assessment of randomized control trials of primary treatment of breast cancer. J Clin Oncol 1986;4:94251.