What adults with ADHD want to know: A Delphi consensus study on the psychoeducational needs of experts by experience

Abstract Introduction A lack of knowledge about attention‐deficit/hyperactivity disorder (ADHD) can contribute to feelings of distress and difficulty in seeking and accepting an ADHD diagnosis. The present study uses a Delphi consensus design to investigate the psychoeducational needs of adults with ADHD and the information about ADHD they would like included in digital health interventions for adults with ADHD. Inclusion of perspectives of service users in developing such interventions ensures that they are evidence based and addresses the risks of engagement barriers. Methods The expert panel consisted of 43 adults with ADHD (age range: 23–67 years). Panel members were asked to rate the importance of the proposed topics and provide additional suggestions. Suggested topics and topics that did not achieve consensus were included for ranking in the second round. Results Interquartile ratings were used to determine consensus. A high consensus was achieved in both rounds, with an agreement on 94% of topics in the first round and 98% in the second round. Most topics were rated as important or essential. Conclusions The findings highlighted that adults with ADHD want to learn about many different aspects of ADHD and the importance of considering their perspectives when developing psychosocial interventions. Findings can be applied when creating psychoeducational content for adult ADHD. Patient or Public Contribution Adults with ADHD were recruited to the Delphi panel to use an experts‐by‐experience approach. In doing so, we are engaging service users in the development of a psychoeducational smartphone app. The evaluation of the app will involve interviews with app users. Additionally, the present study was developed and conducted with ADHD Ireland, a charity based in Ireland that advocates for people with ADHD.


| INTRODUCTION
Attention-deficit/hyperactivity disorder (ADHD) has been increasingly recognized as a lifespan disorder, with 21%-38.8% of those diagnosed as children fulfilling the diagnostic criteria as adults. 1,2 Psychoeducation is a standard component of psychosocial treatments for ADHD. 3 Multiple studies have shown the benefits of group-based psychoeducation for adults with ADHD, demonstrating improvements in knowledge of ADHD, self-esteem, psychological well-being and ADHD symptoms. [4][5][6] The research has focused mainly on the effectiveness of group-based psychoeducation. As clinics for ADHD are often busy services with high demand, 7-9 psychoeducation provided by digital health interventions may be helpful and an opportunity for patients to engage with psychoeducational content outside of the clinic, reducing the burden on services.
Digital health interventions, such as smartphone apps, have risen in popularity in recent years, 10 with research demonstrating their effectiveness for various mental health conditions. [11][12][13] The guidelines for ADHD provided by the National Institute for Health and Care Excellence advise clinicians to recommend helpful supports, like apps, to patients to facilitate adherence to treatment. 14 Apps can be useful for adults with ADHD, providing them with psychoeducation, 15 digital diaries and reminder notes at low or no cost. 16 Preliminary randomized studies demonstrate the potential effectiveness of apps for ADHD. An intervention using mobile apps to support organizational skills has been shown to significantly decrease inattention and hyperactivity symptoms and depression scores in adults with ADHD. 17 Additionally, a chatbot mobile app appeared to improve symptoms of attention deficit and emotional lability. Greater engagement with psychoeducational materials within the chatbot was associated with lower ratings of symptoms, 18 which may suggest a supportive role of psychoeducational digital health interventions for adults with ADHD. In children and young people with ADHD, digital health technologies may help support self-management. 19 A digital health intervention aimed at improving attention and related cognitive control processes significantly improved objective scores of attention, ratings of ADHD symptoms and functional impairment. 20 Feasibility and pilot studies demonstrated the utility of an iPad app for monitoring the behaviour of children with ADHD in their classrooms. 21 In addition to effectiveness studies, qualitative research appears to demonstrate a willingness to engage with digital health interventions for ADHD. Many adults with ADHD learn to rely on external resources like apps to help support their organizational skills. 22 People with ADHD and healthcare clinicians also frequently use websites for information on ADHD but want access to trustworthy and reputable resources. They recognize the potential supportive role of remote monitoring technology. The ideal app would be reliable and trustworthy and would include features for monitoring and tracking side effects and symptoms. 23 Furthermore, Flobak et al. 24 used a participatory design process to include the input of adults with ADHD in the development of their psychoeducational online therapeutic content. Some participants in their clinical trial described the codesigned videos as the most positive aspect of the intervention. Through these videos, participants recognized and related to the depicted challenges, characters and situations and viewed the video protagonists as companions and role models for change. A systematic review of ADHD apps downloadable from the app store found that 109 apps were available catering to a wide variety of users, with 30% targeted towards adults with ADHD. However, very few of the apps described their evidence-based support, and only two of the apps were being subjected to empirical evaluation. 25 The challenge in delivering digital health interventions is the high attrition rate of users despite those users' initial willingness to utilize such interventions. For example, Hidalgo-Mazzei et al. 26  There were no significant differences between the chatbot and control conditions in acceptability, potentially because of the chatbot's interface. Additionally, a qualitative study with clinicians and children and young people with ADHD found that the characteristics of the top 10 apps in the marketplace did not match the participants' views of what was important and helpful to include. 27 It may be that while there is an initial interest in the digital health interventions offered, the actual experience of them is negative due to lack of input from people with ADHD at a developmental stage. A review of technology research for ADHD conducted by researchers with ADHD found that technology development and research teams neglected the views of people with ADHD, which can lead to resistance towards the intervention. The focus of technologies has often been on disciplining or suppressing ADHD traits. Researchers and developers in digital health interventions for ADHD should aim to centre ADHD voices. 28 Engaging service users in the design of digital health interventions is also essential to meeting their needs and to ensuring that the technologies are more engaging, feasible, acceptable and effective. 29,30 While service user perspectives are crucial for the successful implementation and adoption of digital health interventions, there is no research on the specific educational needs of adults with ADHD to inform their development. Such research offers valuable insight into the target users' concerns and desires, which can help to address barriers to use and centre the ADHD voice. One way to do so is to understand the priorities of adults with ADHD in the content of digital health interventions. Delphi methods can be used to aggregate ideas, make future predictions, determine expert opinions and achieve consensus by conducting a series of 'rounds' of questionnaires with experts. 31,32 Achievement of consensus is valuable for the development of digital health interventions as ascertainment of a breadth of needs allows for the intervention to potentially resonate with a wider range of service users. The Delphi method has been commonly used in health research, [33][34][35][36][37] with a select few in the ADHD literature, 16,[38][39][40] and to identify the best forms of personalization and monitoring indicators in a digital health intervention for grief. 41 While studies traditionally include a panel of professional experts, with many including subpanels with patients, the Delphi method with experts by experience may potentially inform the development of interventions. They are also beneficial as service users' priorities may differ from those of professionals. 42 For example, Novais et al. 43 conducted a modified Delphi method involving caregivers of people with neurocognitive diseases to identify a consensus of caregivers' common needs. This expert-byexperience methodology has also been applied with children to identify what they feel is essential in an intervention on selfconcept, 44 to explore adolescents' needs following nonconsensual sexual images 45 and to define autistic burnout and highlight the specific needs of autistic adults. 46 A consensus of expert-by-experience views allows researchers to understand the priorities that service users have for the features and content of interventions. Identifying the priorities of service users and hierarchizing them can help in the design of targeted interventions. Understanding of the consensus of priorities enables the intervention to address the needs of as many people as possible. 43 Identifying priorities for neurodivergent people is particularly pertinent as research has shown that there are gaps between available and desired interventions for neurodivergent people and that outcomes are not aligned with the needs of the community. 47 In ADHD digital interventions, the focus has traditionally been on disciplining ADHD traits 28 and adopting a medical model of disability, thus marginalizing neurodivergent users. 48 A Delphi method aimed at achieving consensus on educational topics and adaptions to interfaces between experts by experience could help identify priorities of adults with ADHD. By identifying and achieving consensus in priorities of adults with ADHD, researchers and developers can ensure that they are listening to ADHD voices and addressing the breadth of needs from digital health interventions.
Therefore, a Delphi method of achieving consensus between experts by experience may produce a helpful ranking of educational topics that can inform digital health and psychosocial interventions for adults with ADHD.

| The present study
The current study is the initial step in a project to develop smartphone-based psychoeducation and a virtual workshop series for adults with ADHD. The app will be specific to Ireland and developed for the Health Service Executive's (HSE's) National Clinical Programme for Adults with ADHD with ADHD Ireland. As digitalbased interventions have high rates of attrition, 26 the present study was conducted to include the perspectives of adults with ADHD in the development of the app. By using a Delphi consensus method with experts by experience, the specific aims of this study were (1)

| Topics development
The list of topics for the first round of the study was developed similarly to Ahmed et al. 38

| Thematic analysis
In line with Braun and Clarke's 51 recommendations, the first author familiarized herself with the data by initially reading through the organization's website content and rereading content when it was selected for inclusion. Initial codes were generated and regularly reviewed with the senior author. Following initial coding, the first author searched for themes in the codes before reviewing them.
Themes were defined and named. A thematic map was developed.
Overall, four main themes were identified: background information on adult ADHD, diagnosing ADHD, interventions for ADHD and living with ADHD. Five subthemes were observed within the theme of living with ADHD: relationships, laws and rights, occupational and educational settings, finances and driving. The themes formed the categories of topics that could be presented in the smartphone app.
Potential topics were derived from codes in the themes for 119 topics.

| Participants and recruitment
Delphi studies do not require large samples, as the reliability and validity of achieving consensus appear to be unaffected by sample size. 49 Previous Delphi studies on ADHD have had final sample sizes that ranged from 21 to 58. [38][39][40] Adults who self-identified as having Participants who provided their email addresses and consented to be contacted for the next round of the study were invited to complete the final survey. A total of 26 participants completed the second round of the survey (attrition rate of 40%), with 15 not following the link to the second round and two dropping out of the study before reading the information sheet.

| Data analysis
The consensus was determined by using interquartile ranges (IQRs).
IQRs are an objective measure of observing agreement between panel members by measuring variance. 52 On a Likert scale with four or five options, an IQR of 1 or less is considered a high level of consensus. 53,54 An IQR of 1 or less demonstrates that respondents showed slight variance in their responses, and as such, there is a strong consensus. IQRs are calculated by subtracting the lower median from the upper median of scores. Topics with an IQR of more than 1 were included in the second round.

| RESULTS
A total of 156 topics were presented to the panel members between the two rounds of the study, including the original 119 topics developed from the thematic analysis of ADHD organization websites and the additional 37 topics that participants suggested.
The complete list of topics can be found in the online Supporting Information Material.

| The first round
In the first round of the study, consensus was achieved for 94% of the 119 topics, with 112 having an IQR of 0-1. Topics with an IQR of 0 were as follows: the prevalence of ADHD (mostly rated as important), executive functioning (essential), inattention (essential), the prevalence of specific learning disabilities (important), available intervention options (essential), staying focused (essential), managing procrastination (essential), time and task management (essential) and improving organization (essential).
Topics that did not achieve consensus (IQR of 2) were the history of ADHD, the gender ratio of ADHD, transitioning from children and adolescent mental health services to adult services, occupational therapy, recognizing snake oil treatments, notifying the Road Safety Authority (RSA), driving insurance and weight management.
Most of the topics (75) were rated as essential. No topics were rated lower than 'unsure' (four topics in total). A bookmark feature was also recommended by another participant, as well as a timestamp feature for long videos or audio clips.

| The second round
The second round included five topics from the initial round that did not achieve consensus and topics suggested by participants.

| DISCUSSION
The present study aimed to identify and prioritize the psychoeducational content that adults with ADHD want to know and whether the content should be adapted in a particular way to best develop a digital health intervention that will meet the needs of its service users. To do so, 119 potential educational topics were developed by analysing ADHD organization websites and conducting a modified Delphi study comprising two rounds. Overall, agreement was achieved on 94% of the items in the first round, and only the 'gender ratio' did not reach a complete consensus by the end of data collection. To the best of the author's knowledge, this is the first study to provide insight into the priorities of adults with ADHD for informational needs from a digital health intervention and to use a Delphi methodology utilizing experts by experience of ADHD rather than professionals with academic or clinical knowledge of ADHD.
Both rounds observed a very high consensus in participants' ratings, with most topics being rated as important or essential on average. It may be that adults with ADHD value a breadth of information and education about many different aspects of their ADHD. After a diagnosis of ADHD in adulthood, Aoki et al. 55 found that many participants are highly interested in learning about their ADHD and seek out information in pamphlets, books and online. It may be that participants in the current study felt that a significant number of the topics would be of value in a digital health intervention and, therefore, rated many as important or essential. Alternatively, the high acceptance rate may reflect the methodology of the study.
In their Delphi study, Ahmed et al. 38  forgetting about or losing interest in the study. It may also be that participants felt that the first round of the study required too much time, which can lead to dropout in Delphi studies. 57 The number of items in Delphi rounds is also associated with dropout, with more items leading to increased rates of attrition. 58  Items rated as essential reflected participants' desire to understand their ADHD in-depth (e.g., its nature, executive functioning, impulsivity and inattention). Aoki et al. 55 observed that following a mixture of confusion and self-stigma after being diagnosed, Japanese adults started seeking out information on ADHD as a disorder to learn how to cope and share with their loved ones. Clinicians or digital health interventions for adults with ADHD should aim to provide adults with a comprehensive (but not overwhelming) overview of ADHD while emphasizing the advantages of ADHD.
Considering how to best adapt content might be particularly relevant when providing an overview of ADHD, as this will likely be the earliest discussion with someone who was recently diagnosed. The use of graphics was rated as essential for the app. Clinicians may also want to use pictures or visuals when explaining ADHD, particularly for complicated concepts like executive functioning.
Mobile apps or digital health interventions can help address gaps in service provision due to a global shortage of clinicians and access to care in rural areas. 59 Apps are more likely to reach vulnerable patient groups than other digital health interventions 60 and also have the potential to engage service users in their care, increase access to and use of evidence-based interventions and provide supports after formal treatment has concluded. 61 The anonymous nature of apps may also help to protect the user from experiences of stigma. 62,63 Additionally, from a service perspective, digital health interventions and apps may be a cost-effective resource, 64,65 although research is still limited. Results from the present study have been used to inform the content of a psychoeducational app. This app will offer evidence-based psychoeducation and self-help techniques to adults with ADHD in Ireland, many of whom are likely waiting on services as the HSE's National Clinical Programme for ADHD in Adults gradually launches ADHD services around the country. 66 The app will hopefully provide a helpful resource to adults with ADHD, with content addressing the topics ranked as essential in the present study.
By drawing on the present study's findings in the development of digital health interventions, researchers and clinicians can ensure that they are meeting the informational priorities of adults with ADHD.
This may help to reduce attrition rates and usability challenges 18,26 and provide apps that are actually meeting the users' needs. 27 Researchers and clinicians should also look to continue to draw on the perspectives of adults with ADHD during other stages of development to ensure that service users identify and resonate with the content 24 and that the interface is accessible. 18 Developers can draw on the present study's findings on how contents should be presented on the apps to help design the interface. Clinicians may also want to consider findings during postdiagnosis discussions or follow-up appointments with recently diagnosed patients. Results could also inform any psychoeducational content, such as booklets or information on websites, that clinicians direct patients towards.

| Limitations
Findings should be considered with awareness of the study's methodological limitations. The Delphi method aims to produce a consensus of experts' priorities. 31 The present research wished to identify and prioritize the educational needs of adults with ADHD from a digital health intervention, similar to other studies that have highlighted the priorities of adults with psychosis 42 and caregivers of people with neurocognitive diseases. 43 However, this methodology does not allow for exploring experience. Future research that uses traditional qualitative methods will be essential to investigating what adults with ADHD would like to learn about their ADHD and the materials or interventions they received after a diagnosis, similar to The study was advertised as recruiting adults with ADHD and required those who identified as such to self-select into the research.
However, participants did not provide evidence of a clinical diagnosis.
This was to facilitate participation from adults who self-identify as having ADHD but who may not yet have a diagnosis or opted not to pursue a formal diagnosis, due to significant waiting lists and barriers to ADHD care. [68][69][70] Research has indicated that college students who self-identify as having ADHD but are undiagnosed demonstrate more neuropsychological impairment than non-ADHD peers, 71 which may possibly support the validity of self-diagnosis. However, we did not collect data on which participants self-diagnosed and were formally diagnosed and could not analyse any significant differences between these groups or their frequencies. Not collecting these data or requiring evidence of clinical diagnosis may have led to a recruitment bias. Future research could explore differences and similarities in educational and psychological needs between adults with formally diagnosed and self-identified ADHD.
Another limitation of the research is a lack of a stability criterion.
While Delphi studies generally focus on consensus, stability between rounds is provided by the consistency of responses between rounds. 72 The findings of systematic reviews on Delphi techniques have shown that the stability of judgements does not typically play a central role in Delphi articles. 32 The 112 items that achieved consensus in the first round of the present study were not presented in the second round to reduce participant burden and to minimize attrition. However, had these been presented to participants, it would have been possible to assess the stability of responses.
Similarly, we did not analyse disagreement or diversity of needs and perspectives across users. As the aim of the study was to identify the priorities of most participants, and therefore hopefully meet the needs of many of the app users, the focus of the research was on achieving consensus. However, disagreement between panellists is often a valuable and insightful outcome. 73 The original 119 topics were derived from the thematic analysis of the content available on the websites of multiple ADHD organizations. The language of these topics was drawn directly from the websites. However, some of the language may have been overly medical (e.g., comorbidities, transitioning from CAMHS to adult services). This may have caused participants confusion and led to rating the topic as more important due to a lack of understanding.
Technical language may have also led to an educated participant sample with a high literacy level. As the level of education was not asked in the demographic questions, this potential effect cannot be fully considered.
Eight topics did not achieve consensus in the first round.
However, three were not included in the list presented during the second round. Two of these topics referred to notifying the RSA and driving insurers about ADHD. Information on these topics was provided on two ADHD organizations' websites. It was later discovered that informing the RSA is not necessary in Ireland, and to minimize confusion, the item around insurance was also removed. Weight management was not included to avoid redundancy, as nutrition and exercise were both rated as essential on average.

| CONCLUSION
The present modified Delphi study aimed to determine what adults with ADHD want to know about ADHD from a psychoeducational smartphone app. Our findings highlight that adults want to learn about many aspects of ADHD, ranging from the condition itself to how it can affect their daily lives and techniques to manage it. A high consensus was achieved in both rounds, with most topics rated as important or essential. The high rankings of topics also underline the importance of providing psychoeducation to adults with ADHD. Findings will be used to inform the development of a psychoeducational app and can also be applied to content for interventions or as a reference point when discussing ADHD with someone, mainly if they have been recently diagnosed.