Telehealth to Train Interventionists Teaching Functional Living Skills to Children with Autism Spectrum Disorder

The number of Board Certi ﬁ ed Behavior Analysts ® (BCBA ® s) available outside of the United States is signi ﬁ cantly lower than the number of individuals with a diagnosis of autism spectrum disorder (ASD) requiring services. Research on alternative delivery platforms, such as telehealth, to disseminate interventions to those in need is of paramount importance. The current study evaluated the effectiveness of training professionals working with individuals with ASD in behavior-analytic procedures through telehealth. Four professional – child dyads took part. Data were collected on the professionals ’ treatment ﬁ delity and number of independent steps performed by the children for 3 functional daily living skills. The study employed a multiple probe design across participants and included a cost analysis. All 4 professionals met the mastery criterion and all 4 children demonstrated increases in independence.

Autism spectrum disorder (ASD) is a developmental disorder identified on the basis of restrictions in behavioral interests, social interaction, and speech (American Psychiatric Association, 2020).Currently, ASD has a prevalence of one in 89 among European children (Autism Spectrum Disorders in the European Union, 2018), and one in every 54 children in the United States (Centers for Disease Control and Prevention, 2020).Therefore, the need for effective interventions to reach more individuals is continuously increasing.Interventions based on the science of applied behavior analysis (ABA) are considered evidence-based practice (EBP; Dillenburger, 2011;Reichow, 2012;Reichow et al., 2018;Steinbrenner et al., 2020;Wong et al., 2015).
Policy makers in the United States categorize ABA as a medical therapy, with all 50 states now funding behavior-analytic interventions in state-regulated health plans (Autism Speaks, 2020) based on the quality of research deeming ABA as an EBP (Hagopian et al., 2020).Although loopholes may still exist in regard to nonlicensed states, ABA provisions in the United States are much more accessible when compared to other countries.Unfortunately for countries outside of the United States, the accessibility of EBP for autism intervention is limited.To add to this, many misconceptions surrounding ABA exist.In the United States, many professionals trained in the science of ABA are able to correct misinformation (Keenan et al., 2015).However, countries in Europe and beyond do not have enough professionals trained extensively enough in behavior analysis to do so.As a result, estimates suggest that only a third of children with ASD in Europe receive behavioranalytic intervention, with others receiving an eclectic approach as a form of treatment (Salomone et al., 2016).
As European governments do not endorse ABA-based interventions, families must pay for services themselves (Dillenburger et al., 2006;Fennel & Dillenburger, 2018;Keenan et al., 2015).Although the Behavior Analyst Certification Board (BACB) has seen a rise in the number of individuals obtaining board certification, this is mostly limited to those residing in the United States (Tsami et al., 2019).To put things into a global perspective, in the United Kingdom, statistics suggest that 1% of the population has a diagnosis of ASD (National Autistic Society, 2019), but just 362 Board Certified Behavior Analysts ® (BCBA ® s) reside there (BACB, 2020).Further, in Pakistan, only four BCBA ® s serve an estimated 2.1 million people with ASD (estimate based on 1% prevalence) and Serbia has a population of 7.2 million with only one BCBA ® (BACB, 2020).The United Arab Emirates (UAE) has a population of 9.4 million and just 88 BCBA ® s and Greece has a population of 10.2 million and just 11 BCBA ® s (BACB, 2020).It is therefore unsurprising that many professionals working in the field of ABA have reported high burnout levels (Dounavi et al., 2019;Plantiveau et al., 2018).The limited number of professionals available to provide high-quality interventions to individuals with ASD and the number of disparities in access to quality services based on racial, ethnic, and socioeconomic status (see Smith et al., 2020 for review) elevate the need to investigate alternative delivery methods to disseminate behavior analysis.This may be combatted by providing high-quality training to those responsible for ABA-based programming through alternative delivery platforms.
One promising solution to this issue is telehealth, defined as "the delivery of healthcare services, where patients and health care providers are separated by distance" (World Health Organization, 2021).It makes use of videoconferencing software and equipment such as mobile devices, cameras, laptops, and tablets to provide health-related services digitally.Research has already shown positive outcomes when utilizing telehealth to provide behavioranalytic interventions (Craig et al., 2021;Ferguson et al., 2021;Neely et al., 2017Neely et al., , 2019;;Suess et al., 2014Suess et al., , 2016;;Vismara et al., 2009Vismara et al., , 2013)).For example, Wacker et al. (2013) demonstrated that functional communication training (FCT) reduced problem behaviors by an average of 93.5% when conducted via telehealth.Research has also examined the use of a virtual training program to train behavior technicians in discrete-trial and play-based procedures with results demonstrating that participants in the treatment group had statistically significant advances in the implementation of such procedures (Fisher et al., 2014).Furthermore, a systematic literature review examining telehealth as a method to deliver interventions to individuals with ASD summarized telehealth-based interventions as positive, while calling for further research to strengthen the position of telehealth as an EBP (Ferguson et al., 2019).The review found that just 9% (n = 21) of participants delivering interventions were professionals (teachers, speech and language pathologists, ABA therapists, etc.), highlighting the need to investigate further research with this population.Additionally, no studies focused on the efficacy of telehealth to promote the acquisition of functional living skills by individuals with ASD.Since then, and to the best of the current authors' knowledge, just one study has demonstrated the effectiveness of telehealth to teach functional living skills to individuals with ASD (Boutain et al., 2020).
Functional living skills include, but are not limited to, washing hands and face, getting dressed, cooking, managing money, accessing the local community and shopping (Domire & Wolfe, 2014).These skills are fundamental to navigating existing and forthcoming environments with independence (Hong et al., 2016).Previous research has suggested that only 12% of adults with ASD and with an IQ of 50 or more had very good or good outcomes in regards to functional living skills and independence with such activities (Howlin et al., 2004).Furthermore, research shows that individuals with ASD who are able to perform specific functional living skills independently Emma Craig et al.
are more likely to gain post-high school education or employment (Wagner et al., 2005).The ability to care for oneself increases dignity and a sense of independence and reduces the cost of caring for individuals with ASD and the burden on their family (Järbrink, 2007).Clinicians and researchers should therefore routinely target these fundamental skills within behavioranalytic interventions.When an individual has limited access to in-person care, telehealth can allow BCBA ® s to target functional living skills in the environment where they will be used.
The purpose of the present study was to evaluate the use of telehealth to train international professionals who resided outside of the United States and who worked with individuals with ASD.The professionals were taught the principles of behavior analysis and the implementation of interventions that target functional living skills.More specifically, we assessed the efficacy of online didactic training separate from live coaching sessions conducted via telehealth.The professionals (hereafter called interventionists) taught children three complex behavior chains and received live coaching sessions similar to those described by Barkaia et al. (2017).The study evaluated whether the interventionists conducted teaching sessions with high treatment fidelity in the absence of the experimenter (i.e., coach) and whether the children acquired the skills, along with the social validity and cost-effectiveness of the intervention.

Participants and Setting
We circulated a call for recruitment among professional contacts and social media.Four professional-child dyads took part in the study.Interventionists were eligible for the study if they (a) worked with a child with ASD, (b) were not related to the child other than through a professional relationship, (c) had no prior formal training in behavior analysis and, (d) were fluent in English.Participants conducted sessions in their native language if they spoke in Greek, Spanish, or French as the second author was able to translate these languages to English for data collection purposes.Children were eligible for the study if they (a) had a formal diagnosis of ASD as assessed by formal diagnostic tools such as the Autism Diagnostic Observation Schedule, Second Edition (ADOS™-2; Lord et al., 2012), (b) were between the ages of 24 and 96 months (2-8 years), and (c) displayed deficits in ageappropriate functional living skills assessed through behavioral tools, including the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP; Sundberg, 2008) and the Assessment of Functional Living Skills (AFLS; Partington & Mueller, 2012).Participants included two schoolteachers and one teaching assistant based in the same school, which taught a British curriculum in the English language, based in the UAE and one Englishspeaking speech and language pathologist working for a clinic in Serbia.Each participant worked individually with a child with ASD (see Table 1).Interventionists working within the same organization conducted sessions independently and were instructed not to discuss procedures with colleagues taking part in the study.Two of the schoolteachers were originally from the United Kingdom and the teaching assistant, who was fluent in English, was from the UAE.
Participants based in the school took part in sessions in a classroom that was equipped with preferred items, a table and chair for making a snack, and a sink for handwashing.The children were also taught how to get dressed in this room.In the speech and language clinic, sessions occurred in a room which included a table and two chairs, and handwashing took place in the bathroom.The behavior consultant (first author) was an English-speaking Board Certified Behavior Analyst ® (BCBA ® ) based in Belfast, Northern Ireland, conducting the study as part of her doctoral dissertation, and had ASD.The behavior consultant did not know participants prior to the study.The study was supervised by the second author, who was a doctoral level behavior analyst with 20 years of experience in the field, and who was fluent in English, Spanish, French, and Greek.

Materials
Nine didactic training PDF documents were designed using Canva (online design software) with pre-and posttests designed on Canvas Instructure (online teaching platform) and SurveyMonkey for weekly quizzes.A GoPro ® camera recorded all video models included within the PDF training document.The behavior consultant conducted training sessions using a laptop with a built-in microphone and camera within a secured locked office to ensure privacy.Participants utilized a laptop for videoconferencing training sessions with a built-in camera.The aforementioned equipment was employed for live coaching sessions with the addition of wireless bug-in-ear headphones for all interventionists to prevent the child from becoming distracted by the consultant's voice.Interventionists used a smart phone to record additional video footage of sessions conducted in the absence of the consultant.On occasions, when the internet connection was weak, participants switched to their cellular device to make use of mobile data.Participants without the necessary equipment were loaned a web camera and wireless bug-in-ear headphones.
Videoconferencing software included Zoom™ and Skype™, with QuickTime Player utilized for screen recording purposes.Dropbox™ and WeTransfer were employed for sending and receiving large video files.All software utilized was available free of charge with the exception of Dropbox™ (i.e., required a monthly subscription from the behavior consultant to obtain a larger storage space).Data were stored on a password-protected hard drive which was backed up to the university's secure network.

Design
A multiple probe design across participants was used to assess the effect of the independent variable on the dependent variables.

Interventionists
Interventionist behavior was scored against a treatment fidelity checklist (see Table 2).Steps completed on the checklist were scored as correct or incorrect.The percentage of steps Note.*SEN = Special Education Needs, SLP = Speech and Language Pathologist.
Emma Craig et al.
performed correctly was then calculated.It should be noted that interventionist performance was not scored incorrectly for the step, "teaches each step of the task analysis in order" when teaching dressing if a child preferred to put on their socks and shoes before putting on their t-shirt.However, interventionist behavior was scored as incorrect if a prompt was not provided when the child attempted to put on their shoes before their socks.

Children
Data were collected for the following three target behaviors: (a) getting dressed, (b) making a snack, and (c) handwashing.All four children showed deficits in these skills during baseline.
We assessed baseline skill level using the selfcare checklist from the VB-MAPP and the AFLS.All children were taught using a totaltask chaining procedure as some of the children already had some of the skills in their repertoire.The number of steps performed independently and the number of steps performed correctly with a prompt was recorded for each behavior on a task analysis.Target behaviors were considered mastered when the child performed 80% of the steps written in the task analyses independently for two sessions in a row.All task analyses were individualized for each child.For example, for the snack-making task analysis, Mohammed and Sasmita were taught to make a sandwich, Daniel was taught to spread Nutella onto a cracker, and Catherina was taught to mix breadcrumbs and Nutella together (see Supplemental Materials for task analyses).Due to the ethical concerns of a child undressing on camera, the children were only taught how to put on a t-shirt, socks, and shoes.When removing their t-shirt, children kept a vest on under their clothes to protect their privacy.Once interventionists were fluent in the teaching procedures, they were advised to apply the skills to teach the children to put on underwear and trousers off-camera.

Cost Analysis
We compared the cost of providing behavior-analytic treatment in-situ versus via telehealth to assess the cost effectiveness of telehealth-based interventions.We calculated the hourly rate of the BCBA ® (Dounavi et al., 2019), the cost of equipment, and the travel costs associated with providing treatment in-situ.The hourly rate was converted from Euros to United States dollars (€1.00 = $1.27 as of January 18 th , 2021) and multiplied by the total number of treatment hours provided.Travel expenses were added to the total.To Teaches each step in the order of the task analysis but follows child's lead when appropriate (e.g., if child puts on socks and shoes before t-shirt).Did the interventionist provide reinforcement for correct responding at the end of the chain?
Provides reinforcement only at the end of the chain.
Did the interventionist collect data for each step of the task analysis?
Scores an independent response or prompted response on the data sheet.Did the interventionist calculate percentage of independent steps and percentage of prompted steps?
Calculates the number of steps performed correctly by dividing the number of independent steps completed by the total number of steps in the task analysis.
compare a number of different contexts, we calculated costs across six hypothetical situations in which a behavior analyst typically works: 1. BCBA ® is on site with all participants in the same school/clinic, charging an hourly rate of $60.36 (converted from €50.00 per hour as described by Dounavi et al., 2019 as the average hourly rate for a BCBA ® .BCBA ® also charging hourly rate for travel time).2. BCBA ® is on site with all participants in the same school/clinic, charging milage allowance for travel time instead of an hourly rate (converted from €0.57 per mile to $0.69 per mile).
3. BCBA ® is on site with all participants in different schools/clinic, charging an hourly rate for travel time.4. BCBA ® is on site with all participants in different schools/clinic, charging milage allowance for travel time instead of an hourtly rate. 5. BCBA ® is on site with all participants in the same school/clinic, charging air fare, train fare, and accommodation costs (average cost €1,590 over 14 days of treatment divided by four participants = $479.84).6. BCBA ® is offsite with all sessions conducted via telehealth.
We incorporated a calculation of direct labor and indirect labor to allow time for the BCBA ® to score prerecorded videos and visually analyze progress on graphs by adding an extra 30 min of billing time for every direct hour worked.Indirect costs associated with the design and preparation of training materials were not included in the analysis.

Baseline
Interventionists were instructed to teach their child the three target behaviors as they normally would, to video-record the session, and to upload the footage to Dropbox™ or send the file through WeTransfer.The purpose of this phase was to identify deficits in functional living skills and to collect baseline data on interventionist and child behavior.

Didactic Training
The purpose of this condition was to evaluate the effectiveness of didactic training alone on interventionists' knowledge and teaching strategies.At the beginning of this phase, interventionists were sent the task analyses for each targeted skill.Table 3 displays the list of didactic training topics.Although this list may appear to be extensive, the goal was to disseminate the science of behavior analysis by providing widespread training beyond what is necessary to implement the teaching procedures due to the lack of access to EBP outside of the United States.
During the first week, interventionists were sent one training document.Once they read through the document, a suitable time was arranged to meet the experimenter on a one-toone basis using one of the videoconferencing platforms.The experimenter and interventionist worked through the training document together, giving the interventionist a chance to ask questions and allowing the experimenter to model skills while giving the interventionists an opportunity to practice the skill through role play.Following the videoconferencing session, interventionists completed a 10-question quiz with questions based on the session content.The mastery criterion was a minimum of 80% correct on the quiz.If the mastery criterion was not met, the experimenter provided an additional one-to-one videoconferencing session for further training.Interventionists then re-took the quiz.Once the mastery criterion was met, interventionists were instructed to teach their child the target behaviors, applying any of the new skills they had learned during the didactic training session, and to send the video-recorded footage to the experimenter for data collection.
Each training document provided a description of a particular skill.For example, the chaining document described how to implement forward, backward, and total task chaining procedures and included links to video footage of a behavior analyst teaching a skill using these procedures with a confederate.Once the training session ended, interventionists were instructed to use a total-task chaining procedure and most-to-least prompting to teach the three target behaviors to their child.There was no time limit set for the duration of video footage, but interventionists were required to record the entire duration of at least one teaching session per skill.
Upon receipt of the video footage, the experimenter sent the next training document and followed the aforementioned sequence.
Interventionists were not required to meet a set criterion on the treatment fidelity checklist prior to receiving a new didactic training session, but rather score at least 80% on the knowledge test.Interventionists received a total of nine training sessions in the didactic training phase, with each videoconferencing session lasting approximately 1 hr and interventionists receiving two sessions per week (topics displayed in Table 3).

Live Coaching
Interventionists began live coaching sessions once they completed all nine didactic training sessions and met mastery criterion for each quiz.Interventionists met the experimenter via telehealth, using one of the previously mentioned videoconferencing software packages.Live coaching sessions began with the experimenter providing feedback based on performance in previous video recordings (e.g., "nice job providing reinforcement for the behavior," "next time use the next least-intrusive prompt").Interventionists had the opportunity to ask questions and then implemented the teaching procedures with the child for the three target behaviors.The experimenter provided live performance feedback on each step in the treatment fidelity checklist as the interventionist taught each skill.Performance feedback included correcting any errors in prompting strategies, rearranging stimuli in the environment, correcting the positioning of the interventionist, and reinforcing correct responding.All live coaching sessions were recorded using QuickTime Player.Interventionists were instructed to video-record one teaching session per week with the child in the absence of the experimenter.This footage was sent to the experimenter for data collection using Dropbox™ or WeTransfer.These sessions provided additional opportunities for both the interventionist and child to practice the skills.The mastery criterion for the interventionist was performing 80% of the steps in the treatment fidelity checklist correctly over 2 consecutive days.

Follow-Up
Three-month follow-up probes were collected for Rosie and Majidah (after school closed for the summer holiday).Two-month follow-up probes were collected for Jana.No follow-up probes were collected for Eliza, as she moved to a different classroom after the summer holiday and no longer worked with Sasmita.Procedures were identical to those in baseline.

Interobserver Agreement
The first and third authors calculated interobserver agreement data on 33% of sessions across all experimental conditions on both interventionists' and children's behaviors (i.e., percentage of steps performed correctly on the treatment fidelity checklist and percentage of steps performed independently on the task analyses, for interventionist and child, respectively).The first author conducted the sessions and was therefore not blind to conditions; however, the third author was blind to each experimental condition.Agreement was calculated using a trial-by-trial procedure (i.e., number of trials in agreement divided by the number of trials in disagreement multiplied by 100; Cooper et al., 2020).Mean agreement for treatment fidelity was 99% for baseline (range, 88%-100%), 97% for didactic training (range, 88%-100%), 98% for live coaching (range, 97%-100%), and 100% for follow-up.Agreement for child behavior was 98% during baseline (range, 93%-100%), 96% during didactic training (range, 86%-100%), 98% during live coaching (range, 86%-100%), and 99% during follow-up probes (range, 93%-100%).

Treatment Integrity
The third author collected treatment integrity data on the experimenter's coaching procedures during 33% of the live coaching sessions using a checklist that included the following steps: (a) provides a positive statement regarding performance in the previous video footage, (b) states areas for improvement, (c) provides another positive statement regarding performance in previous video footage, (d) provides discriminative stimulus for interventionists to begin teaching, (e) provides immediate feedback for correct responding, (f) provides immediate feedback on errors, (g) asks if the interventionist has any questions, and (h) ends the session with a positive statement.Treatment integrity data were calculated by the percentage of steps performed correctly and was scored as 100%.

Social Validity
Interventionists completed a social validity questionnaire designed by the first author containing 11 questions (e.g., "I found the technology useful and did not encounter any problems," "I feel confident using the principles of ABA," "I would recommend the use of telehealth for support with behavioral interventions").Each statement was evaluated on a Likert scale of 1 to 5, with 1 representing strongly disagree and 5 representing strongly agree.

Dressing
Figure 1 shows the results of the interventionists' treatment fidelity for teaching the skill of dressing and the child outcomes.During baseline, all interventionists implemented few steps correctly on the treatment fidelity checklist and the children had minimal success with performing the target behavior independently.After the first didactic training session, performance for all interventionists, with the exception of Eliza, remained identical to that in baseline.An increasing trend was noted for all interventionists across the didactic training phase, with interventionists performing at least 50% of the steps correctly by seventh session.However, none of the interventionists met the mastery criterion after completing all of the didactic training topics, with the exception of Jana when teaching the skill of dressing (indicated by the horizontal line for expected performance).An increase in independent dressing was visible for all children by the fifth didactic training session, with the exception of Daniel, whose performance did not increase until the seventh session.Treatment fidelity increased further with the introduction of live coaching.All interventionists met the mastery criterion after the second live coaching session and maintained this performance during followup probes.Independent increased further for two children (Mohammed and Catherina) and maintained for all children during the follow-up probes.

Hand Washing
Figure 2 shows the results of the interventionists' treatment fidelity for teaching the skill of hand washing and child outcomes.Results are similar to those for dressing.During baseline, interventionists implemented few steps correctly on the treatment integrity checklist and the children completed few steps independently.All interventionists completed at least 50% of the fidelity steps correctly by the seventh didactic training session, but none met the mastery criterion after completing all of the didactic training topics (indicated by the horizontal line for expected performance).The children showed corresponding increases in independent hand washing.Similar to the skill of dressing, all interventionists met the mastery criterion during live coaching sessions and maintained those skills during the follow-up probes.The children's performance remained unchanged during live coaching, although Daniel's responding decreased somewhat, and all children maintained responding during the follow-up probes.

Making a Snack
Figure 3 displays the results for the interventionists and children for the skill of snack making.All participants exhibited 0% correct responding during baseline.During didactic training, all interventionists showed an increase in performance, but none met the mastery criterion after completing all of the didactic training topics (indicated by the horizontal line for expected performance).All children showed corresponding increases in correct responding during didactic training.With the introduction of live coaching, the interventionists' correct responding increased further to 100% by the fifth coaching session and remained high during follow-up probes.The children's responding either increased or remained at similar levels during the live coaching and follow-up probes.

Social Validity
All interventionists either strongly agreed or agreed that the intervention was effective, that they liked utilizing telehealth to make contact with a BCBA ® , and that they would recommend telehealth.One interventionist scored not applicable/uncertain for finding the technology useful and for not encountering any connection difficulties.

Time Spent on the Design and Implementation of Telehealth Sessions
The experimenter created a total of nine training documents for the didactic training sessions, which took approximately 2 months.Once ready, a further 2 weeks were spent editing the training documents according to feedback obtained by three independent testers.A total of 56 video conferencing sessions took place, which totaled 56 hr.Of these, a total of 36 hr were spent on video conferencing during didactic training sessions, approximately 1 hr Emma Craig et al.
per session per participant.After completion of training, a further 20 hr were spent during live coaching sessions, approximately 1 hr per session per participant.A total of 204 videos across all participants and conditions was sent to the behavior consultant for data collection.A total of 15 min was allocated to each video for scoring (approximately 51 hr

Figure 3
Interventionist and Child Outcomes for Making a Snack in total).There was a total of 12 disconnections during video conferencing sessions.This was due to poor internet connection or software not being set up correctly.A total time of 115 min was spent trying to fix connection difficulties which interrupted videoconferencing sessions.Three sessions had to be rescheduled due to connection problems on the part of the interventionist not being resolved.

Cost Analysis
Figure 4 presents a cost analysis of the treatment package with costs converted from Euro to US Dollars.The majority of participants who took part in this study had their own equipment and therefore the behavior consultant loaned equipment to just one participant.The cost of one set of headphones and one camera totaled $115.87 (€95.58) with an additional $27.28 (€22.50) for postage (tracked and signed for), making the total cost for purchasing and sending the equipment $143.15(€118.08).The average hourly rate for a behavior analyst in Europe is between €0-75 (Dounavi et al., 2019;$0-91.69) and due to the limited amount of BCBA ® s available in some European countries, families and specialist schools and clinics who can afford it often pay for the travel of a BCBA ® .Travel expenses have been incorporated into Figure 4.For the purpose of this study, we calculated the cost of an extra 2 hr per working day to allow the BCBA ® to reach the destination and return, as would have occurred had the study been conducted in-situ.Indirect costs associated with the time and expense of preparing training materials were not included in the analysis because similar training tools are available.Additionally, the training materials designed for the purpose of this research could be used in future research and clinical practice; as a result, time and costs dedicated to their design would no longer be applicable.We calculated costs over the 14 days of treatment and divided costs by the total number of participants for the categories in which children were based in the same school/clinic.

Discussion
Results suggest that telehealth can be an effective model for training international professionals residing outside of the United States who have no prior training in behavior analysis The study examined some relatively novel factors, including the teaching of complex behavior chains to children with ASD via telehealth, a cost analysis of telehealth-based interventions versus in-situ training, and an assessment of interventionist performance in the absence of the coach.Overall, the results suggest that interventionists' treatment fidelity can increase without live coaching sessions, but that performance may not meet the mastery criterion with didactic training alone.Previous research on the use of self-instructional packages to increase skills has shown positive results (Berkman et al., 2019;Graff & Karsten, 2012).However, the results of the current study indicate that live coaching sessions were required for interventionists to master the teaching procedures.It should be noted that didactic training may have had sequence effects on the results of live coaching, that is, we cannot conclude whether mastery would have been reached without the initial didactic training phase.Although Jana met the mastery criterion when teaching Catherina how to get dressed in the last two didactic training probes, she did not perform 100% of the steps correctly until she received feedback during live coaching.These findings are similar to the results from previous research that utilized hybrid telehealth packages (Ingersoll et al., 2016;Wainer & Ingersoll, 2015).
As the data show, on occasion, interventionists scored slightly above criterion during didactic training even though the experimenter had not yet introduced particular content.However, the experimenter covered behavior principles in every session, and each training document provided examples and included video models.Furthermore, interventionists were provided with the task analyses for each of the child target behaviors at the beginning of didactic training.It is possible that these training materials promoted generalization to unlearned skills.The one-to-one consultation following training, during which the interventionist could ask questions and role play specific procedures with the experimenter (e.g., hand-over-hand prompting) also may have facilitated treatment fidelity.In future research, a component analysis of the variables responsible for behavior change (e.g., written description of procedures, video models, or one-to-one consultations with the trainer) would provide further insight, saving time and resources.
Improvements in the interventionists' treatment fidelity corresponded with increases in the children's independent responding for all three skills.These findings extend previous literature linking high treatment fidelity with optimal child outcomes (Penn & Perry, 2007;Symes et al., 2006).During baseline, interventionists tended to complete the steps for the children rather than teaching the children to engage in any of the three skills independently.The interventionists also delivered a high number of verbal prompts during baseline, which were not effective and are considered the most difficult type of prompt to fade (Anson et al., 2008).
Although Sasmita was the only child whose performance reached mastery, decisions for moving forward were derived from the interventionists' performance.Judging from established trends in the children's data (see Figures 1, 2 and 3), child behavior may have reached mastery with more time.Future research may consider a more in-depth focus on child outcomes to assess the timeframe required for telehealth to enhance skill acquisition in children with autism and potentially compare this to in-situ training.Additionally, Emma Craig et al.
all children were taught each skill using a totaltask chaining procedure even though a different approach, such as forward or backward chaining, may have led to stronger child outcomes.Future research might focus on training interventionists to modify their teaching procedures to increase the effectiveness of programs for the child.
Another aim of the research was to assess the cost effectiveness of a telehealth-based intervention compared to providing treatment in-situ.The cost analysis showed clear differences in costs between the two delivery methods.This result extends previous research examining the cost-effectiveness of a telehealth platform (Lindgren et al., 2016).Due to the limited number of BCBA ® s in Europe, a behavior analyst is often required to travel to various countries to offer services.Providing treatment in-situ most often results in families and/or clinics paying for travel expenses, including airfare, train fare, and accommodation.Telehealth has no other associated costs except for equipment, if needed, which is inexpensive when compared to travel fees.Although such devices are now universal, future researchers may encounter settings that do not have access to such equipment (Lerman et al., 2020).We loaned equipment to participants who did not have access to the specific equipment and future research may consider this approach.The results suggest savings between $181.88 (€149.97)and $916.43 (€755.63)per participant across 14 days, when a telehealth platform is employed.If these savings are extrapolated to a full school year with 38 weeks of classes, savings would total between $3,455.70 (€2,849.43)and $17,412.13(€14,356.97)per participant, representing an enormous budget that schools and families could employ to increase the intensity or reach of the intervention or for other health needs.Although the cost analysis may provide valuable data, these results should be interpreted with caution.
Statistical analyses were not conducted as described by Lindgren et al. (2016) because the costs for in-situ treatment were hypothetical.In addition, the analysis did not include operational costs such as ensuring participants have adequate internet speed.Further, hours spent preparing training materials were not included, as such materials are only prepared once or can be accessed for a relatively low cost from existing book or digital resources.Further research should include cost-analyses when examining telehealth as a platform to deliver services.
The final aim of the study was to assess the social validity of providing behavior-analytic training and interventions via telehealth.Results suggested that telehealth was a wellliked, acceptable platform with all participants agreeing that telehealth was a good way to connect with a BCBA ® .All interventionists said they would recommend telehealth to those who may have difficulties accessing services.These results were consistent with previous findings (Bearss et al., 2017;Neely et al., 2016;Tsami et al., 2019).
A number of additional limitations should be noted.First, the experimenter did not conduct a sequence of consecutive baseline sessions prior to introducing the independent variable on each baseline, as typically employed in a multiple probe design (Horner & Baer, 1978).Second, Eliza did not provide follow-up probes for Sasmita due to changes in classrooms and assigned teachers.Third, on occasion, connection issues caused obstacles when providing treatment, which were generally resolved by switching to a smart phone data connection.Fourth, sessions were often disrupted due to interventionists mistakenly hitting the mute button or closing the application.Future research may consider delivering thorough training on the use of the technology equipment prior to other training to ensure technical difficulties can be resolved quickly and sessions can run smoothly.Additionally, other technical difficulties, such as the child walking out of view of the camera or the camera falling over, presented a challenge but were easily overcome.Interventionists fixed these issues quickly with verbal prompts from the trainer.Furthermore, quite often, the children would become distracted, notice themselves on the camera, or perceive the laptop as equivalent to a reinforcer.Future research should consider placing equipment in the classroom for a few days prior to the start of the intervention to reduce reactivity.
Future research should also consider the strength and stability of internet connections in developing countries to improve accessibility and assess the efficacy of telehealth across a wider and more culturally diverse population.Telehealth interventions can only be implemented with professionals who have a reliable internet connection, which poses barriers against those without these resources.Currently, the vast majority of BCBA ® s are located in what are considered developed countries; however, the few BCBA ® s residing in developing countries who could offer their services to remote and rural populations might face limitations related to the availability of technology and internet connectivity.
Overall, this study illustrated telehealth as feasible, well-liked, cost-effective, and an accessible alternative to connect professionals working with individuals with ASD residing outside of the United States with a behavior analyst.Clinicians with the availability to offer services to additional clients may wish to consider the utilization of telehealth.This way, behavior analysts can have a positive impact on individuals globally, rather than just locally or nationally.
Figure 1Interventionist and Child Outcomes for Dressing Figure 2Interventionist and Child Outcomes for Washing Hands Figure 4Cost Analysis

Table 2
Treatment Fidelity Checklist

Table 3
Didactic Training Topics