Identifying eating disorders at the earliest opportunity: Testing the reliability of an online eating disorder screener (IOI‐S) in primary care and youth mental health settings

Eating disorders (EDs) are associated with significant disease burden and unacceptably high mortality rates. Early intervention significantly improves prognosis and can prevent chronic suffering; however, large numbers of people with the illness are not being identified or managed in primary healthcare. The current study aimed to test the reliability of the face‐to‐face, clinician delivery of a previously validated, co‐designed, online screening tool for eating disorders.

face-to-face in clinical practice.In conjunction with increased practitioner education and improved treatment referral pathways, broad implementation of the screener in early healthcare settings can support timely identification and intervention for those with EDs.
assessment, early identification, eating disorders, primary care, screening

| INTRODUCTION
Eating Disorders (EDs) are serious psychiatric illnesses associated with high mortality, frequent co-morbidity and significant personal and economic burden (Arcelus et al., 2011;Butterfly Foundation, 2012;Hambleton et al., 2022;Smink et al., 2013;Tannous et al., 2021;Udo & Grilo, 2019;Ulfvebrand et al., 2015;Westmoreland et al., 2017).Despite a conservative population prevalence of 4%-5%, comprising mostly of Binge Eating Disorder (BED) and subthreshold disorders otherwise specified (Butterfly Foundation, 2012;Hay et al., 2015), currently only 0.32% of the Australian population is managed in primary care for an ED (Ivancic et al., 2021).Indeed, only 1 in 4 people with EDs are accessing treatment of any kind for their disorder (Hart et al., 2011).Given early intervention drastically improves chances of remission and recovery (Currin & Schmidt, 2005;Errichiello et al., 2016;Le Grange & Loeb, 2007), the profound gap between population prevalence and recognition of EDs in primary care demonstrates a clear and vital opportunity to improve outcomes by educating primary health professionals about EDs, empowering the patient to seek help and developing more efficient psychometric tools for earlier detection and management in these settings.
Early intervention and treatment at this time can prevent the negative impacts of prolonged nutrition on the brain and long-term vocational and social impairments, as well as minimize mortality risk (Eddy et al., 2017;Le Grange & Loeb, 2007;Treasure et al., 2015).
People with EDs access primary care more often than others in the years leading up to diagnosis (Ivancic et al., 2021).They also commonly present to general mental health settings seeking support for conditions not necessarily related to their eating (Burton et al., 2022;Fursland & Watson, 2014).A 2022 study found that over one third of young people presenting to a major headspace youth mental health centre in metropolitan Sydney reported symptoms of disordered eating (Burton et al., 2022).As such, GPs and mental health primary care centres are ideally placed to detect, evaluate and treat EDs and to coordinate multidisciplinary team responses at the earliest possible opportunity (Aalmen et al., 2020;Rowe, 2017;Walsh et al., 2000).A skilled GP or mental health clinician can provide crucial support for both patients and their loved ones throughout the ED treatment journey, from prevention to rehabilitation.While many individuals may welcome non-judgmental questions regarding their eating during the early stages of an ED (Evans et al., 2011), unfortunately, routine screening is uncommon in primary care and general mental health settings (Burton et al., 2022;Currin et al., 2007;Eisner-Fellay et al., 2020;Fursland & Watson, 2014;Ivancic et al., 2021;Klein et al., 2021;Sim et al., 2010).Many GPs and mental health clinicians working in primary care settings lack ED education and confidence, which may contribute to underdiagnosis (Allen et al., 2011;Bryant et al., 2022;Gooding et al., 2017;Sim et al., 2010).Further barriers to identification include short appointments (Irving et al., 2017), as well as intrapersonal factors on behalf of the patient, such as perceived stigma, lack of motivation and shame leading to reluctance to seek care (Ali et al., 2017;BMJ, 2017;Evans et al., 2011).However, qualitative research shows provision of an acceptable, inviting and valid screening tool is likely to be welcomed by those patients who may be reluctant to initiate a conversation (Lazare et al., 2021).In suicide prevention training, practitioners are trained to ask directly, "Have you ever thought about ending your life?" which leads to risk assessment and management planning.Broad implementation of appropriate, validated screening tools for EDs, which employ gentle, non-stigmatizing language to begin risk assessment, should be part of a system-wide response to increased ED incidence and burden (Castellini et al., 2020;Hay et al., 2008).Physicians in healthcare settings such as gynaecology, assisted reproductive services, cardiology, gastroenterology and endocrinology, as well as in general nursing and dental settings, may also benefit from adopting a screening toolthese are settings in which individuals with ED frequently go undetected (Bryant et al., 2022).
Screening tools exist for EDs but are not well utilized.The most widely recognized ED screening tool, the SCOFF questionnaire (Morgan et al., 1999) has limited sensitivity to ED presentations beyond anorexia and bulimia nervosa and occurring in genders other than female, and may be unacceptable to patients and clinicians due to its use of confronting language (Hill et al., 2010;Mond et al., 2007).
A meta-analysis on published validations of the SCOFF questionnaire found it had low sensitivity levels in primary care and community settings and indicated the need for a new screening tool (Kutz et al., 2020).Other validated ED screening questionnaires include the Eating Disorder Screen for Primary Care (Cotton et al., 2003) (more sensitive but less specific than SCOFF) and several short forms of the EDE-Q: the EDE-QS (Gideon et al., 2016) (12 items), the EDE-Q7 (Grilo et al., 2013) and the EDE-Q8 (Kliem et al., 2017).The short forms have shown good sensitivity and internal consistency however are subject to the limitations of the original EDE-Q including that it was developed prior to DSM-5 and is not designed to capture Binge Eating Disorder (BED), Avoidant Restrictive Food Intake Disorder (ARFID), Other Specified Feeding or Eating Disorder (OSFED) and Atypical AN.Few screening tools have been developed since the 2013 revision of the DSM despite the changes to diagnostic categories.Further, existing diagnostic and screening measures are typically validated to discriminate between healthy and ill, rather than using a continuum to identify those showing early symptomatology/risk (Mond et al., 2004).Bryant et al. (2021) described the psychometric validation of a digital screening tool co-designed with lived experience experts and clinicians, which addresses existing barriers to early identification by focusing on stigma-reducing language and online delivery, increasing accessibility and empowering consumers (Bryant et al., 2021).It is designed to "start a conversation", focusing on an individual's relationship with food and their body to capture a broad range of eating disorder presentations and identify those who likely have an existing eating disorder as well as those at high risk.Despite a broad shift to digital healthcare, particularly since the COVD-19 pandemic, internet accessibility remains an issue for some people and others simply prefer to use hard copy questionnaires or ask questions face-to-face.
Given the frequency with which individuals with EDs present to primary care in the years preceding diagnosis and the existing barriers to screening in that setting, it is important that a validated screening questionnaire is at least agile to multiple modes of delivery dependent on the preference of the primary care facility.Strong interest in the adaptive use of the screener from primary care physicians without access to facilities that enable their patients to transmit online selfreport scores led to the necessary exploration of the internal validity of the IOI-S of its face-to-face use.This paper describes the validation of the delivery of the IOI-S face-to-face in primary care and headspace settings versus online self-report methods.

| Participants and recruitment
Individuals aged 14 and over who read English and were attending either primary care, (general practice) settings or headspace centres (youth mental health centres located Australia-wide) were invited to participate in the study.Presenting clients were informed of the study by the practice staff or their treating clinician and were invited to present to a separate consult room after their scheduled appointment for 5-10 min if they were interested in participating.

| Study design
Participants were provided with a Participant Information Statement detailing the study procedure and information regarding confidentiality and management of data and asked to sign a consent form.Individuals under the age of 16 were also required to have a parent/guardian sign a consent form.The study clinician researcher asked basic demographic questions (age, gender, ethnicity) and the six IOI-S questions verbally.Answers were then rated verbally by the clinician on the Likert scale (not shown to the participant).Afterwards, they handed the participant an iPad to answer the six questions online, providing their own ratings.In order to minimize demand characteristics, participants were ensured that clinicians would not be viewing their online answers and were encouraged to answer as truthfully as possible.

| The InsideOut institute screener (IOI-S)
The IOI-S is a six-item digital screening tool designed to identify individuals who may be at risk of or currently experiencing symptoms of an eating disorder.The screener was co-designed with consumers, clinicians and expert researchers to capture a broad range of eating disorder symptomatology.It uses sensitive language designed to 'start a conversation' and has been psychometric validated with over 1300 people aged 14-74 using a longitudinal repeated measures survey research design (Bryant et al., 2021).The IOI-S has excellent psychometric properties, including strong construct and criterion validity and high accuracy (sensitivity and specificity) for two levels of risk (moderate and high) and likely eating disorders.A summary of the screener's psychometric properties can be found in the original study (Bryant et al., 2021).The IOI-S is rated on a 5-point Likert scale, where 1 is 'never', and 5 is 'all the time', except for Question 1, where 1 is 'worry and stress free', and 5 is 'full of worry and stress' (Table 1).
Responses are summed to yield a score between 6 and 30 points total, where 6 points is the lowest degree of risk, and 30 points is the highest degree of risk.Identified risk thresholds are as follows: 13moderate risk; 16high risk; 19probable ED.

| Sample size and statistical analyses
The primary outcome measure was test-retest reliability between the two modes or stability of responses between verbal/face-to-face clinician delivery of the IOI-S and online self-report delivery of the IOI-S.
Test re-test reliability was calculated using a two-way mixed effects intraclass correlation coefficient (ICC) with absolute agreement.Pearson's correlation coefficient between IOI-screener items was reported to determine inter-item correlation within verbal and online delivery, respectively.Sample size estimation was calculated using R 4.1.1(R Core Team, 2021), the ICC.Sample.Size (v1; Rathbone, Shaw & Kumbhare, 2015) package.For an ICC of 0.7, which is considered to demonstrate good to excellent reliability (Cicchetti, 1994;Matheson, 2019;Shrout & Fleiss, 1979) with a power of 0.80, a minimum of 79 participants was required.All outcome analyses were conducted using SPSS (Version 28; IBM Corporation; Armonk, NY).

| Participant characteristics
A final sample of 83 participants completed the study nationally in primary care and headspace clinics between April and November 2022 (84 participants commenced the survey, but one participant was excluded following survey completion due to self-reported language comprehension difficulties).48% were recruited in primary care and 52% in headspace.Participants were aged between 14 and 81 with a mean age of 36.2 (SD 20.1).Approximately one quarter were male (24.1%) (see Table 2 for demographic information).The headspace cohort had a lower mean age of 23.8 versus 49.5 years in primary care.26 participants were enrolled in the study in Katherine, NT.

| Total/overall IOI-S score
All participants completed the digital re-test.Mean estimations and a 95% confidence interval were reported.The ICC between successive iterations of the test was significantly positive, being 0.980, with a 95% confidence interval from 0.971 to 0.988 (F (82, 82) = 52.28,p < .01(see Table 3).

| Distribution of IOI-S scores
Table 3 shows the distribution of IOI-S scores for each question.Of the 83 participants, 81.9% with verbal delivery and 77.1% with online delivery answered that weight body or shape makes them feel bad about themselves at least a little bit of the time (2 or greater).The total range of scores spanned from 6 to 27 both verbally and online.20.5% of participants scored in the moderate to high-risk categories and 21.7% scored as probable ED for both verbal and online delivery.
T-tests found no significant difference in individual item or overall scores between primary care and headspace participants.

| Inter-item correlation
Table 4 presents the pairwise correlation between IOI-Screener items delivered verbally versus delivered online.The inter-item correlation between verbal and online delivery was consistent.

| DISCUSSION
Early detection of eating disorder symptomatology is critical to a person's long-term outcome; however, this is not occurring in many T A B L E 1 The InsideOut Institute Screener (Bryant et al., 2021).healthcare settings (Lazare et al., 2021).A significant barrier to appropriate identification of both sub-threshold and threshold illness in primary and general mental healthcare is a lack of acceptable, adaptable ED screening tools designed to capture both risk and symptomatology of a range of DSM-5 presentations (Bryant et al., 2022).The current study tested the adapted use of a co-designed digital ED screening tool (IOI-S) which has been shown to examine the full spectrum of EDs and identify risk thresholds, with strong demonstrated psychometric properties (Bryant et al., 2021).This brief verbal screening measure can be easily and readily rolled out for use in intake assess- There was little difference in means and standard deviations on overall IOI-S score between the primary care and headspace populations.Many participants reported their relationship with food as at least 'a bit' problematic.Whilst agreement on all items was high, item 6, which refers to compensatory behaviour, had the highest agreement between the two modes, which is surprising given the shame that has traditionally been associated with this behaviour and research showing it is frequently under-reported when broached face-to-face (vs.online or when the person is anonymous) (Mond et al., 2007).The acceptable and gentle language of this item may have minimized social desirability bias, where respondents underreport behaviour seen as 'undesirable' on direct questioning (Latkin et al., 2017).There are some limitations to the study.These include a relatively small sample size due in part to face-to-face recruitment difficulties experienced throughout the COVID-19 pandemic, where healthcare services largely transitioned to online delivery.This impacted the ability to include participants from a broader demographic in both primary care and headspace settings.However, this was mitigated in the final sample by participants being recruited from both rural and outback Australia and central Sydney, the very large sample upon which the IOI-S was originally validated and the high rates of agreement between the online and face-to-face version tested here.It is possible that repeated exposure to the test (i.e., delivery of the online iteration of the screener directly after the verbal iteration) influenced scores through the effect of practice, however the information provided to the participant about the study, including that clinicians would not be viewing their online answers and were encouraged to answer as truthfully as possible, aimed to mitigate this.
The earlier an ED is detected, the more successfully it can be treated (Bryant et al., 2022).The need for early intervention is a compelling reason to arm primary care practitioners with acceptable screening tools for case finding, particularly for identifying pre-symptomatic and early illness.Screening tools should be costeffective, reliable and accurate, and ideally, will be linked with effective treatments, which improve health outcomes for a population at a reasonable cost (Iragorri & Spackman, 2018).
The IOI-S, co-designed by clinicians and individuals with lived experience, is a psychometrically valid online tool to detect ED risk and subthreshold illness in a self-referred population, which was also designed to be used online in clinical settings.The current study extends its validation to face-to-face delivery across a wide age range in relevant clinical settings in both urban and regional Australia.Combined with clinician education and system improvements, it can aid the detection of these serious and sometimes deadly conditions earlier in the illness course and bring patients into treatment at a stage where they are more likely to respond to evidence-based treatments.
ment and routine care as an efficient and effective method of screening for ED risk and indeed for opening up a conversation about eating and body image concerns between clinician and patient.The IOI-S can replace the SCOFF by providing a more comprehensive screen across the ED spectrum (capturing higher prevalence presentations) and in language that is more suitable for both patient and clinician.Study results support the use of the IOI-S for face-to-face verbal delivery, with strong positive correlations and very high test-re-test reliability between the verbal and online iterations of the screener in a primary care and specialist mental health population aged from adolescence to elderly adulthood.The IOI-S performs equally well delivered by a clinician verbally, face-to-face, as it does self-reported online by the individual.Results consolidate support for the strong internal validity of the original scale.
Participant demographic characteristics.Frequency distribution of IOI-S Scores (N = 83), inter-modal reliability and internal consistency analysis.
When you think you have eaten too much, do you do anything to make up for it?(For example: skipping the next meal, going light on the next meal, working it off with exercise, purging via vomiting or taking laxatives, diuretics or diet pills.)Pairwise correlations between verbal and online delivery of IOI-screener items.