The quality of diagnostic guidelines for children in primary care: A meta‐epidemiological study

To determine the quality of paediatric guidelines relevant to diagnosis of three of the most common conditions in primary care: fever, gastroenteritis and constipation.

1 General practitioners (GPs) face diagnostic uncertainty in children due to communication difficulties, undifferentiated symptoms and differing physiology across the ages. 2 Clinical practice guidelines (CPGs) aim to reduce uncertainty and standardise care pathways for diagnoses and management.3 A series of Australian population-based studies reported adherence to clinical guidelines 60% of the time for 17 common childhood conditions.Adherence was generally lowest in the primary care setting compared with inpatient and emergency care.

What this paper adds
1 Paediatric CPGs related to the diagnosis of fever, constipation and gastroenteritis are of varying quality (ranging from 3.8 to 6 out of 7 using the AGREE II tool). 2 Specific areas of weakness across the guidelines include limited consideration of guideline applicability, failure to involve parent representatives in guideline development and not adequately declaring or addressing competing interests.3 Poor-quality clinical guidance may contribute to poor guideline adherence and unwarranted variation in care in general practice where diagnostic uncertainty is high, and symptoms are more undifferentiated.
In Australia, approximately one fifth of general practice consultations are for children and young people under the age of 19. 1 Two thirds of children aged 0-14 present to primary care in the UK for their health needs. 2 Making a diagnosis in children and young people poses unique challenges compared with adults, due to their differing and evolving physiology, communication needs and functional reserve.Additionally, primary care practitioners may have limited dedicated training in paediatrics; for example, the UK training curriculum does not mandate paediatric experience for general practitioners (GPs), 3 which can exacerbate uncertainty for diagnosing paediatric patients with undifferentiated presentations.Given the critical role of primary care, improved diagnostic guidance can standardise care according to the best available evidence, lead to better children's' health outcomes and prevent unnecessary hospital presentations and referrals.
Poor quality diagnostic guidelines can lead to unwarranted variation in care, including underuse and overuse of diagnostic tests.The consequences of failing to perform a necessary test (underuse) may lead to an incorrect diagnosis with deleterious consequences.Performing unnecessary tests (overuse) can initiate a testing cascade, causing children and their families' pain and anxiety, and exacerbate the strain on health-care resources.
Adherence to clinical practice guidelines (CPGs) is used as a metric of the quality of health service provision. 4A series of population studies in Australia reported adherence to national and international CPG treatment guidance approximately 60% of the time for 17 common childhood conditions; this ranged from 43% for preventative care, 5 54% for fever 6 and 60% for acute gastroenteritis. 7CPG adherence was generally lowest in the primary care setting when compared with inpatient and emergency care.
One study assessed the quality and reporting of diagnostic testing guidelines for adults presenting to primary care; however, no similar study exists in children. 8A systematic review determined that the quality of 159 paediatric guidelines from 1990 to 2020 improved over time but remained sub-optimal. 9Another review examining the quality of 216 paediatric guidelines published from 2017 to 2019 found that quality was influenced by whether the guidelines were under the responsibility of organisations or groups, if evidence-based methods were used to develop guidelines, and the socio-economic development index of a country (i.e.countries from less economically developed countries had lower quality scores in four of the six quality domains). 10e aimed to determine the methodological quality and reporting of diagnostic guidelines from high-income countries for three of the most highly prevalent and important childhood conditions in primary care: fever, gastroenteritis and constipation.These conditions were selected by screening NICE guidelines and undertaking a feasibility count using the Clinical Practice Research Datalink, as described in the extended methods (Supplementary File: Data S1).

Study design
We performed a meta-epidemiological study of paediatric CPGs published between February 2011 and September 2022 using a systematic approach.There are no formal reporting standards for meta-epidemiological studies, therefore this study was conducted and reported in line with a modified Preferred Reporting Items for Systematic Reviews and Meta-analyses checklist. 11Ethical approval was not required.The study protocol was registered and published on the Open Science Framework (www.osf.io/hcnwr).This study reports the first step of the two-step protocol.
We searched for international guidelines relating to fever, gastroenteritis and constipation.We assessed the quality and reporting of these international guidelines using the Appraisal of Guidelines for Research and Evaluation (AGREE II) tool.

Search strategy and eligibility criteria
We searched Medline and Embase for paediatric CPGs relating to fever, gastroenteritis and constipation.We also searched the following guideline repositories: Trip Database, Guidelines International Network (GIN), National Guideline Clearinghouse and the WHO guidelines (see Supplementary File: Data S1 for search strategy).We included guidelines from high-income countries with similar health systems to the UK (i.e., Australia, Canada, Ireland, Norway, Denmark and the Netherlands) whereby GPs are the focal point for coordinating referrals to specialist paediatric care. 12We also included guidelines from the USA to avoid missing important guidance that may be used by practitioners in other countries.
Guidelines published or updated between February 2011 and September 2022 were included.If multiple versions of a guideline existed within the specified time period, the most recently updated guideline was used.Guidelines were eligible if they included (i) children and adolescents; (ii) diagnostic testing recommendations and (iii) relevant to primary care practice.Non-English language publications were excluded unless English versions were available.
Two reviewers independently assessed all abstracts for potential inclusion, with discrepancies resolved by consensus and a third reviewer if required.

Assessment of guideline quality and reporting using AGREE II
Two reviewers independently assessed the methodological quality and reporting of eligible guidelines using the AGREE II tool across six domains: scope and purpose, stakeholder involvement, the rigour of development, clarity of presentation, applicability and editorial independence. 13An overall quality score was then determined (ranging from a score of 1 for lowest to 7 for highest possible quality).Domain scores were calculated by adding item scores given by both reviewers within a domain and then calculating this as a percentage of the maximum possible domain score.Disagreements were resolved by discussion and a third reviewer if required.

Statistical analysis
AGREE II scores were reported as medians with corresponding ranges.The results were presented as grouped box-and-whiskers plot representing the variation in scores across domains for each condition.Cohen's kappa was calculated as a measure of interrater reliability.

Study selection
The search was conducted on 2nd September 2022 and returned 360 results (see Data S1 for search strategy).After removing 39 duplicates, the titles and abstracts of 312 records were screened for eligibility.Full texts of 50 guidelines were reviewed and 16 guidelines were deemed eligible for inclusion.Figure 1 shows the guideline selection flow diagram.

Characteristics of included studies
Of the 16 included guidelines, seven were related to fever, four for constipation and five for gastroenteritis (Table 1).All guidelines were published after or updated since 2013.Of the fever guidelines, three were published in the USA (American Academy of Paediatrics (AAP), American College of Radiology (ACR) and Cincinnati Children's (CC)), two were published in Australia (Royal Children's Hospital (RCH) and South Australia Ministry of Health (SA)), one was published in the Netherlands (Dutch College of General Practitioners (DCGP)) and one was published in the UK (National Institute for Health and Care Excellence (NICE)).Of the constipation guidelines, one was published in Australia (SA), one was published in the Netherlands (Federatie Medisch Specialisten (FMS)), one was published the UK (NICE) and the guideline published in Switzerland (European Society for Paediatric Gastroenterology Hepatology and Nutrition/North American Society for Paediatric Gastroenterology Hepatology and Nutrition (ESPGHAN/NASPGHAN)) was intended for world-wide use.Of the gastroenteritis guidelines, two were published in the USA (Infectious Diseases Society of America (IDSA) and World Gastroenterology Organization (WGO)), one was published in Australia (SA), and one was published in the UK (NICE).One of the guidelines published in the USA (World Gastroenterology Organization (WGO)) and the guideline published in Switzerland (European Society for Paediatric Gastroenterology, Hepatology, and Nutrition/European Society for Paediatric Infectious Diseases (ESPGHAN/ESPID)) were intended for world-wide use.

Evaluation of the methodological quality of guidelines using AGREE II
The AGREE scores for each domain are shown in Table 2 (fever), Table 3  statistic for inter-rater reliability was 67%, showing substantial agreement.The quality of CPG reporting was highly variable across all three conditions.The overall quality of all the included guidelines was moderate (median 4.8 out of a maximum of 7, range 2.5-6.5).The guidelines for constipation had the highest overall rating (median 5.3, range 2.5-6.0), and the guidelines for gastroenteritis had the lowest overall rating (median 4.0, range 2.5-6.0).NICE guidelines were rated the highest quality for all  13 AAP: evaluation and management of well-appearing febrile Infants 8 to 60 days old 2021 -USA RCH 14 Febrile child -2022 Australia SA 15 Fever in children aged 1-2 months 2014 Australia DCGP 16 Children with fever 2016 2020 Netherlands ACR 17 ACR appropriateness criteria fever without source or unknown origin-child 2016 -USA CC 18 Evidence-based care guideline for management of infants 0 to 60 days with fever of unknown source Guideline 10 three conditions due to their methodological rigour, high standards of reporting and transparency.Figure 2 presents the variation in the AGREE II domain scores.Most guidelines demonstrated excellent clarity in the presentation of their recommendations.The greatest variation in guideline quality for the three conditions was in the rigour of development (Domain 3), specifically, their transparency in reporting search strategies, reasons for selecting the evidence, consideration of health benefits, side effects and risks in giving recommendations and the process for monitoring, auditing and updating guidelines.Across all conditions, the main areas of weakness included whether the guideline development committees considered the applicability of their guidelines in the clinical setting, particularly facilitators and barriers of guideline implementation.For example, 44% (7/16) of guidelines did not factor in the cost implications of their recommendations.

(constipation) and Table 4 (gastroenteritis). The kappa
Eight guidelines (fever (2), constipation (2) and gastroenteritis (4)) guidelines did not explicitly state whether parents or public representatives were involved in the recommendation development process.Editorial independence was also highlighted as a major area of concern for 56% (9/16) of guidelines, with funding sources and conflicts of interest either not declared, or committee members failing to provide adequate explanations of how competing interests were addressed.

Discussion
For three of the most commonly encountered problems in paediatric primary care, the quality of guidelines was highly variable.The major areas of weakness in guideline development were related to guideline applicability and editorial independence.Guidelines for fever were the most variable in quality with overall scores ranging from 2.5 to 6.5 out of 7.For all three conditions, NICE guidelines performed the best in most quality domains compared to the other guidelines.Variable guideline quality and more importantly, limited consideration of how guidelines can be implemented in clinical settings is a major barrier to their uptake in clinical practice, which likely further contributes to variation in care delivery and health-care outcomes.We previously performed a meta-epidemiological study of childhood asthma guidelines. 28We found that the quality of evidence supporting diagnostic recommendations for asthma was poor, suggesting that guideline bodies need to advocate for better quality evidence to improve the delivery of diagnostic care for children.
To better understand the perceived barriers to guideline implementation and reasons for poor adherence to guideline recommendations, a group of researchers in the Netherlands conducted a qualitative study of Dutch GPs analysing the barriers to guideline implementation.They reported lack of agreement with the recommendations due to poor applicability, unclear recommendations, and lack of evidence as the most common reasons for poor guideline uptake. 29Other factors included organisational constraints, lack of awareness of guideline recommendations as well as patient preferences and behaviours not aligning with recommendations.
To our knowledge, this is the first review to systematically search and appraise guidelines for the most common childhood presentations in primary care.We specifically focused on guidelines that included diagnostic testing recommendations, as GPs tend to encounter more diagnostic uncertainty than other clinicians. 30eciding whether to undertake a test for a child can be difficult, and there are harms associated with both under-testing and over-testing.Good quality guidance is therefore required to aid practitioners.
This review included guidelines that are applicable for use in similar settings; we deliberately focused on guidelines from highincome countries.We also included guidelines for specific conditions, rather than grouping guidelines by organ system.This was to ensure fair comparisons between guidelines.It also allows clinicians to identify the highest quality guideline for a particular condition.
Our findings are limited to guidelines relevant to a primary care setting to better understand how GPs diagnose children, and we only focused on three of the most common conditions affecting children.Other reviews have sought to look at the issue more broadly across a wide range of conditions. 9,10Their findings are consistent with this review's findings suggesting that guidance for paediatric conditions in primary care is generally of poor quality.
Good quality guidelines are a valuable resource for synthesising evidence in the context of a growing body of primary research.However, to effectively inform health-care decisionmaking and reduce unwarranted variation, they should adhere to reporting standards, be developed using robust methods and synthesised using the best available evidence.Guidelines should also promote transparency and ensure that contributors declare and address conflicts of interests.To improve applicability, they should consider the impact of children's presentations with undifferentiated symptoms that certain tests are not readily available in primary care, and thus factor in the feasibility of any recommendation.
Our study also highlights the need for international collaboration for improving the accessibility of CPGs.There are currently multiple sources to search for guidelines including the BIGG International database of GRADE guidelines; Dynamed; ECRI Guidelines Trust; MAGIC authoring and publication platform; NICE; TRIP database; and the US Preventive Services Task Force.

1058
The GIN was established to promote collaboration to improve the efficiency of guideline development, adaptation, dissemination and implementation.However, it is not a comprehensive source of all paediatric guidelinesa search of all guidelines in GIN related to children returned 77 results of which only 20 were from the UK.Yet, there are 85 UK NICE guidelines related to children and young people.To improve the retrieval and accessibility of paediatric guidelines a separate repository is required, which collates all the available guidance for children by condition.Furthermore, guidelines should be open-access and include quality ratings, to enable GPs to identify and evaluate the best diagnostic and management options for children.
Guidelines only play a part when it comes to standardising care to improve health outcomes for children presenting to a general practice.An absence of high-quality guidance could necessitate the lengthening of the duration of GP specialist training to increase paediatric clinical exposure.Some have called for flexible GP models to improve paediatric care delivery, such as child health GP hubs, GPs with extended roles or increasing GP access to specialist opinions. 3In Australia, where paediatric experience is already a core requirement of general practice training, GP registrars have reported lower confidence in managing mental health and behavioural presentations but more confidence in managing acute presentations, suggesting the need for increased exposure to outpatient paediatric medicine in GP training programmes. 31The practice of evidence-based medicine integrates clinical experience with the best available evidence; taken together they can improve the targeting of diagnostics to improve outcomes.

Conclusions
Good quality guidelines could inform health-care delivery and reduce unwarranted health-care variation.There is substantial variation in quality of paediatric guidelines for primary care presentations.To improve guidelines, developers should adhere to strict methodological standards of reporting and appraise their own guidelines prior to publication.Streamlining guidelines into one resource would improve GPs' access to the best quality guidance to improve diagnosis in children.

Table 1
Characteristics of included guidelines

Table 2
AGREE II scores by domain for childhood fever guidelines AAP, American Academy of Paediatrics; ACR, American College of Radiology; CC, Cincinnati Children's Guidelines; DCGP, Dutch College of General Practitioners; NICE, National Institute for Care and Clinical Excellence; RCH, Royal Children's Hospital Melbourne; SA, South Australia Ministry of Health.

Table 3
AGREE II scores by domain for childhood constipation guidelines European Society for Paediatric Gastroenterology Hepatology and Nutrition/North American Society for Paediatric Gastroenterology Hepatology and Nutrition; FMS, Federatie Medisch Specialisten; NICE, National Institute for Health and Care Excellence; SA, South Australia Ministry of Health.

Table 4
AGREE II scores by domain for childhood gastroenteritis guidelines ESPGHAN/ESPID, European Society for Paediatric Gastroenterology, Hepatology, and Nutrition/European Society for Paediatric Infectious Diseases; ESPGHAN/NASPGHAN, European Society for Paediatric Gastroenterology Hepatology and Nutrition/North American Society for Paediatric Gastroenterology Hepatology and Nutrition; FMS, Federatie Medisch Specialisten; IDSA, Infectious Diseases Society of America; WGO, World Gastroenterology Organization.