Reducing epilepsy diagnostic and treatment gaps: Standardized paediatric epilepsy training courses for health care professionals

To evaluate improvement in knowledge and clinical behaviour among healthcare professionals after attendance at paediatric epilepsy training (PET) courses.

Epilepsy is the most common serious chronic childhood neurological disorder.It affects children across the world, with 80% of people living with epilepsy in low-and middleincome countries (LMICs). 1 Globally, over 320 000 children (1-14 years old) died in relation to epilepsy between 2007 and 2017; over 10 million years of life are estimated to have been lost because of epilepsy in the same period. 2,3 onvulsive status epilepticus is one of the most common childhood neurological emergencies worldwide, requiring urgent medical intervention to avoid death or disability. 46][7][8] This is markedly higher than the mortality rates (2.7%-5.2%)reported by high-income countries (HICs). 7,9 strategic objective of the Intersectional Global Action Plan on Epilepsy and Other Neurological Disorders is to 'provide effective, timely and responsive diagnosis, treatment and care'.It highlighted the need for health care worker training to achieve this. 10 The concept of the epilepsy 'treatment gap' is well recognized.11 Since 2005, paediatric epilepsy training (PET) courses have been taught by the British Paediatric Neurology Association (BPNA), a charitable organization of neurologists, paediatricians, and allied health care professionals.The aim of the courses is to improve the diagnosis and management of children with epilepsy.The courses have been taught internationally since 2012 in 17 countries across five continents.From 2005 and up to September 2021, more than 14 000 participants attended PET courses.
PET1 is an entry level course for all health care professionals who look after children with suspected epilepsy.It is included in the World Health Organization technical brief as a health care workforce case study. 12Course participants are primarily paediatricians, medical officers, or specialist nurses.It is a 1-day course with a standardized curriculum that teaches evidence-based best practice.Delivering important epilepsy care guidelines to key providers responsible for the main entry point to clinical care for affected patients is critical for their widespread application.The intended learning outcomes include recognition, investigation, management, and treatment of children with epilepsy (Appendix S1).The curriculum is developed collaboratively, referenced against international, peer-reviewed, evidence-based guidelines, [13][14][15] and revised regularly.At times of revision, course leaders from participating countries come from low-, middle-, and high-income settings to capture variations in the availability of medical resources, including investigations and treatment, and surveyed parental views.The structure is based around established models of adult learning, with lectures and small-group workshop teaching, incorporating simulated case scenarios.The courses are delivered by an unpaid, volunteer faculty of trained health care professionals.The volunteers must participate in a 1-day instructor training course and then perform regular teaching on PET courses, initially mentored, as trained faculty.
Course evaluation has been an integral component of PET1.First, we wish to ensure that the course is fit for purpose and meets the needs of the participants, and that it improves knowledge and changes in clinical practice.Second, the evaluation is used to inform a rolling programme of curriculum updates, thus promoting the longer-term sustainability of the educational intervention.
Ours was a cohort study that gathered data from participants immediately after the course and then 6 months later.The aim of the study was to document a series of evaluations performed on participants' feedback about the course, improvements in knowledge, and self-reported changes in practice.We aimed to establish whether these outcomes differed depending on country resource settings and skill mixes of participants by comparing improvements in self-reported clinical practice between HICs and LMICs.We present the findings from cohorts of over 7000 participants across 252 courses, 17 countries, and five continents over 15 years (2005-2020).

M ET HOD
The BPNA uses the Kirkpatrick evaluation model, 16 a wellestablished tool, to measure course effectiveness through outcome evaluation.The evaluation uses four domains: reaction, learning, behaviour, and results.This model has been used previously to evaluate other international medical teaching courses. 17,18 e methods of course evaluation evolved over time.Initially, participants completed a course reaction survey; later, a knowledge quiz before and after the course was introduced, followed by an after-course survey of behavioural change.A total of 7528 participants completed a participant reaction form.Subsets of these participants also undertook the knowledge quiz or the 'change in behaviour' survey.Findings were compared between LMICs (n = 12) and HICs (n = 5).All data were collected anonymously.

Level one: reaction
A course reaction form was given to every participant immediately on completion of the course.The form was given to participants of all 252 PET1 courses held between January 2005 and March 2020 in a total of 17 countries (Figure 1).
The form captured information on the participants' reaction (e.g.satisfaction) to the course.It collected information on the whole course, individual lectures, and workshops, as well as other aspects of the course, including venue, facilities, and booking procedures.The form used both Likert scales and open questions.These data also informed the regular updates of the courses every 3 years.

Level two: learning
A knowledge quiz was provided to participants before and immediately after the course in 56 courses in 13 countries between March 2016 and March 2020 (n = 1856 participants).

What this paper adds
• Simple and consistently applied evaluation of training can demonstrate improvements with direct relevance to epilepsy care.• Ninety-eight per cent of participants reported that the paediatric epilepsy training (PET) course improved their clinical practice.• Sixty-four per cent of participants reported that the PET course prompted improvements in their clinical epilepsy service.• Improvements in knowledge and clinical practice were most notable in low-resource settings.
The quiz examined the knowledge gained (learning).The quiz was undertaken by participants attending courses in both HICs (22 courses in New Zealand and the UK) and LMICs (34 courses across Angola, Brazil, Ghana, Kenya, Mauritius, Myanmar, Namibia, South Africa, Sudan, Tanzania, and Uganda).The quiz described seven clinical scenarios.For each scenario, the quiz presented a series of questions (using true and false statements) based on the appropriate clinical management taught during the course.Thirty-four questions were included (Appendix S1).Categorical data in the tables are presented as counts (%).Pearson's χ 2 tests were used to assess statistical significance between groups.The magnitude of differences between groups is presented as the odds ratio (OR) with its corresponding 95% confidence interval (CI).

F I G U R E 2
Overall course satisfaction as a percentage of total responses on a Likert scale (percentages have been rounded to nearest whole number).
Overall, participants demonstrated a significant gain in knowledge, with an increase in the proportion of correctly answered questions from 75% before the course to 88% after the course (p < 0.001; OR = 2.6, 95% CI = 2.5-2.7;Table 1).Before the course, the proportion of questions correctly answered was lower among participants from LMICs compared to participants from HICs (p < 0.001).Improvement in the proportion of questions correctly answered after compared to before the course was higher among participants from LMICs.
Improvement in knowledge occurred across all seven clinical scenarios presented in the knowledge quiz.For example, a question about requesting an electrocardiogram (ECG) to help exclude a cardiogenic syncopal seizure (Q1D, Table 2) showed a 35% and 17% improvement (LMICs and HICs respectively).Similarly, a question on epilepsy diagnosis based on history (Q1A) showed a 29% and 13% improvement respectively.A question on emergency management during a prolonged seizure showed a 12% and 6% improvement respectively.
Over 70% of responders reported improvements in both their history taking (323 of 432) and their ability to distinguish between epileptic and non-epileptic seizures (309 of 432; Table 4).Twenty-nine per cent reported now using standard treatment guidelines when managing prolonged seizures; 27% reported increased requests for an ECG as part of their patients' diagnostic work-up, if indicated.A higher proportion of participants from LMICs reported improvements in the delivery of their clinical care compared to participants from HICs across multiple practices.
As part of the survey, participants were asked whether the content of the PET1 course included epilepsy management concepts that were new to them.Multiple aspects a Proportion of correctly answered knowledge questions before and after the course and improvement for all countries, high-income countries (HICs), and low-and middleincome countries (LMICs).Percentages (total correct answers/total questions).
were regarded as new concepts.For example, knowing when to request an ECG was a new concept for 78% (153 of 197) in LMICs and 40% (74 of 185) in HICs.Course content was regarded as a new concept more frequently among participants from LMICs, with the exception of choosing the appropriate antiepileptic drug for prolonged convulsive seizures.

DISCUS SION
Professional medical training courses are a key educational tool in disseminating evidence-based practice to clinicians to ensure optimal management of, in this case, children with epilepsy.However, evaluations of whether such courses are effective and help support patient care are limited.We have been able to undertake this study because, in large part, our evaluation framework is pragmatic and inexpensive to implement.
Our study demonstrates a universally highly positive response among health care professionals (residing in high-, middle-, and low-income settings) to the PET course.Participants conveyed high course satisfaction, demonstrated a significant improvement in knowledge after the course, and reported positive changes in clinical practice 6 months later.The course narrowed the gap in epilepsy knowledge between participants from HICs and LMICs.
To our knowledge, this study represents the largest evaluation of a medical professional epilepsy training course, both in terms of geographical reach and participant numbers; it is the first standardized multilevel evaluation of epilepsy training across multiple countries.Mental health training (which includes an epilepsy module) as part of the World Health Organization Mental Health Gap Action Programme was reviewed. 15The review identified 15 published training evaluations from nine countries.Fourteen evaluated a single country and one study evaluated two countries.The authors used different assessments across each study. 20A single-country evaluation of a paediatric epilepsy course from Zambia was published. 21In 2012, the World Health Organization initiated an epilepsy training programme for non-specialist health care providers, but with only single-country evaluations published. 22,23  study of paediatric resuscitation training reported high participant numbers (n > 13 000).Again, evaluation occurred in only one country. 24hether knowledge gained on medical training courses translates into practice is a key issue.The fact that the overwhelming majority of survey respondents (98%) reported changes in their clinical practice 6 months after the course indicates that the courses may positively influence clinical care.Our study identified several examples of translation from knowledge gain to change in practice, including the use of an ECG for suspected epilepsy (to identify a potentially life-threatening cardiac syncope) and in prolonged seizure management.Participants later reported incorporating ECG testing into their practice.In turn, this stimulates appropriate cardiac management and has clear potential to save lives. 25mproved knowledge on prolonged seizure management was also demonstrated.Participants have since reported introducing standard guidelines for managing prolonged seizures into their local hospitals (Table 4).The management of prolonged seizures is linked to high mortality, particularly in LMIC settings. 7Again, empowerment of local hospitals to use standardized acute seizure management guidelines will encourage sustainable improvements in childhood epilepsy management among staff and help save more lives.The higher levels of change in practice reported by participants from LMICs after the PET course in part probably reflects the fact that PET has been running in the UK since 2005 but internationally only from 2012.This correlates with the lower reported 'new concepts' in HICs as participants from the earlier UK courses may already have experienced some influence on epilepsy care in their local setting.
Our study has several limitations.First, the response rate for the 'change in behaviour' survey was low (21%).Similar low response rates have been reported by other medical training evaluations. 18,26,27 Tus, our results are at risk of selection bias.The low response rate for the 'change in behaviour' survey was probably partly due to the survey being carried out 6 months after course attendance; however, it was important to evaluate after this interval to allow participants the opportunity to make changes in their clinical practice.There was also no active follow-up to ensure that the survey was completed; more responses may have been obtained had this been in place.
Second, there are no direct measures of the results from the Kirkpatrick evaluation model in these data and our methods are prioritizing a pragmatic, inexpensive, widespread evaluation of process outcomes in LMICs rather than a rigorous methodology, such as a cluster randomized controlled trial.More generally, in a critique of the Kirkpatrick model, some suggested that measuring impact or outcome should not be the criterion standard for programme evaluation, but rather to better understand how and why the intervention changes the outcomes. 28Notwithstanding, it is notoriously difficult to measure the impact of a single educational initiative in a health care setting, particularly so in LMIC settings.
Nonetheless, it is plausible to suggest that an educational intervention to improve care, particularly in relation to convulsive status epilepticus, will have an impact on the risk of death.Epilepsy-related mortality data are the most easily collatable data across a range of health care settings, although this still has considerable challenges in the lowestresourced countries.In those countries where this has been measured, epilepsy-related mortality has fallen.Epilepsyrelated mortality has shown a steep decline in the UK since the 1950s; similar and more recent declines have occurred in other countries. 17In an age-period-cohort analysis, O'Callaghan et al. described a strong birth cohort effect that has probably contributed to this decline. 29It is highly likely that declines in epilepsy mortality are a result of multiple, interlinking factors, of which professionals' epilepsy education may be only one.
In future evaluations, we will actively follow up the behaviour survey data to ensure we obtain responses from most participants.This will be done through multiple routes, from e-mail reminders through to social messaging.Results will be compared to the current data to check for selection bias in these latter responses.The BPNA is working with behavioural change specialists to develop more in-depth qualitative interviewing of participants.This will help to identify barriers to improvements in practice, some of which will be open to mitigation.We will also undertake an audit of key clinical practices before and after the courses at local health care facilities in countries where PET1 courses have not yet been established.These audits will assess whether self-reported changes correlate with objective changes in clinical practice.
Access to specialists in LMICs is lacking.Clinical care, especially for newly presenting children with seizures, is delivered by non-specialists, typically at the primary health care level.With the roll out of PET courses in LMICs, the principal attendees are primary health care workers, nurses, and general practitioners.This group would have had minimal access to training in the area of children with epilepsy or seizures.As such, it was not surprising that the 1-day teaching course, with content that was viewed as understandable and adaptable to local practice, had such an impact in LMICs and more so than in HICs where access to training support was more established.The regions of the world carrying the greatest burden for neurological diseases have the lowest ratio of paediatric neurologists per head of the population. 30,31 he specialist gap will take many years to address; in these settings, upskilling of existing health care workers is essential to empower them to be able to deliver effective care using teaching models that are acceptable and effective.
Complementing primary health care epilepsy training programmes and in line with the recent acknowledgement of PET courses by both the World Health Organization and the Education Council of the ILAE, 32 we believe that the PET1 course fills an important training gap for health care professionals in childhood epilepsy management. 12This evaluation lends further support to the effectiveness of this training.The PET curriculum is being used in the planning of other epilepsy interventions in LMICs. 33A network of collaborative teachers, using aligned, consistent, and evidence-based content, is growing as a result of course participants training to become PET faculty members in their own countries.In addition, most participants reported improving their own local epilepsy teaching since attending the course.
In conclusion, since 2005, BPNA PET courses have substantially grown in number across multiple countries.Evaluation of these courses demonstrates that they contribute to improvements in knowledge and behaviour change among health care professionals and support a global reduction in the epilepsy 'treatment gap'.

AC K NO W L E D GE M E N T S
The Buist Fund provided funding for this study via the University of Dundee.We thank Dr Neil Buist for generous financial support.The funder had no role in study design; the collection, analysis, and interpretation of data; the writing of the manuscript; and in the decision to submit the manuscript for publication.The researchers were independent from the funders.
The PET programme is run in partnership with and with financial support from the ILAE.MJG is supported for neuroscience research internationally by the Medical Research Council (MRC) Newton Fund (MR/S019960/1), MRC Developmental Pathway Funding Scheme (MR/R015406/1), and the National Institute for Health and Care Research (153195 17/60/67, 126156 17/63/11, and 200907).

C ON F L IC T OF I N T E R E S T S TAT E M E N T
MJG has research supported by Fast Track Diagnostics and Siemens Healthineers, payment for educational events by Siemens Healthineers, and patents pending or approved for a diagnostic for bacterial infection.PR was the Executive Director of the BPNA up to February 2022.JW has received payment for educational lectures on epilepsy from Sanofi, sits on the national advisory board for safe practice of valproate in women of childbearing age and cotreatment of antiseizure medications and antiretroviral agents, is the Chair of the ILAE African Commission and President of the International Child Neurology Association, and receives honoraria for editorial work for the Epilepsia and Frontiers in Neurology journals.JB receives a salary from an ILAE grant to the BPNA to support PET activities.CD is paid via the BPNA within his NHS job plan for his role as the PET Content Development Manager and via the Royal College of Paediatrics and Child Health for his role as the Epilepsy12 Clinical Lead.AM receives payment from UCB and GW Pharma for educational events and from GW Pharma for sitting on their advisory board, and holds an unpaid position as a trustee for the BPNA and Epilepsy Scotland.FO is the immediate past-President of the BPNA and has received honoraria from Novartis and GW Pharma/Jazz Pharmaceuticals for participation in advisory boards and speaking at educational events.MK is a trustee for international education within the BPNA.The authors are willing to share relevant data upon reasonable request.

DATA AVA I L A BI L I T Y S TAT E M E N T
The data that support the findings of this study are available from the corresponding author upon reasonable request.

A
'change in behaviour' survey was sent electronically to participants (n = 2035) at 58 courses in 15 countries (the countries listed earlier [n = 13] plus India and the United Arab Emirates) between April 2018 and March 2020.The survey was sent 6 months after course attendance.It collected demographic data on course participants including type or grade of health care professional.It recorded selfreported changes in personal practice (behaviour), clinical services, and their confidence in undertaking specific epilepsy-related tasks, such as managing prolonged seizures (Appendix S1).Statistical analysis was performed in R v4.0.3 (R Foundation for Statistical Computing, Vienna, Austria).

F
I G U R E 1 Total of feedback forms from 2011 to 2020 according to country (Mauritius became a high-income country after the course took place).
Results of the knowledge quiz before and after the course a .
T A B L E 1

mother takes her toddler to the doctor after a first 'collapse' at nursery school after he had bumped into another child. The nursery staff explained that the child went stiff, jerked, and was pale. He was sleepy for 30 minutes after the event.
Example of a specific knowledge question from clinical scenarios in the quiz showing the percentage of correctly answered questions before and after the course for highincome countries (HICs) and low-and middle-income countries (LMICs).Abbreviations: CT, computed tomography; ECG, electrocardiogram; EEG, electroencephalogram. Job type in the 'changes in behaviour' survey.a T A B L E 2 Clinical scenario knowledge question.a Q1: A a T A B L E 3 a Responses to question in the 'changes in behaviour' survey asking the question 'What is your current job?' for high-income countries (HICs) and low-and middle-income countries (LMICs) and total, shown as the percentage of responders.T A B L E 4 Changes in personal clinical practice.a In

you been able to introduce changes to the services for children with epilepsy? (Yes/No)
Changes in clinical services a .