Validity of the multidimensional assessment profile of disruptive behavior in autistic toddlers

Abstract Objective Early measurement of atypical disruptive behavior within autistic children is critical for later referrals to behavioral screenings, diagnoses, and services. Disruptive behavior in autistic toddlers is often measured using a categorical approach and identifies the presence or absence of behavior. In contrast, dimensional approaches evaluate behavior on a spectrum of typical to atypical by measuring the clinical salience of disruptive behavior. We sought to assess the validity of the Infant/Toddler version of the multidimensional assessment profile of disruptive behavior (MAP‐DB‐IT), a dimensional approach measurement tool, in a sample of autistic toddlers. Methods Autistic toddlers (n = 82, M age = 33.2 months, SD = 6.28 months) and their mothers received 8 weeks of caregiver‐mediated social communication intervention. Mothers completed the MAP‐DB‐IT and the Infant Toddler Social Emotional Assessment (ITSEA) across three timepoints: before intervention, immediately after intervention, and at 3 months post‐intervention follow‐up. The MAP‐DB‐IT provided scores for three subdomains: temper loss, noncompliance, and aggression (generically or specifically with siblings). Ratings on the MAP‐DB‐IT were compared to the ITSEA using several analytic strategies such as evaluating (a) the internal consistency of the MAP‐DB‐IT domain scores; (b) the convergent validity between the two measures; and (c) its convergent change due to intervention and if this varied by child characteristics. Results The MAP‐DB‐IT demonstrated excellent internal consistency across all four subdomains. We evaluated convergent validity and found positive correlations between the (a) ITSEA externalizing and MAP‐DB‐IT aggression domain, (b) ITSEA externalizing and MAP‐DB‐IT aggression with siblings domain, and (c) ITSEA dysregulation and MAP‐DB‐IT temper loss domain. Conclusion The MAP‐DB‐IT is a valid measurement tool for disruptive behavior in autistic toddlers. Clinicians should consider the use of the MAP‐DB‐IT for young autistic clients presenting with disruptive behavior to (a) discriminate between early developmentally appropriate tantrums from clinically salient dysregulation, and (b) refer to additional behavioral evaluations and services.

Specifically, within a clinical sample 50% of autistic children were reported to display disruptive behaviors (Kanne & Mazurek, 2011).
Furthermore, a review of the literature found that 30% of autistic children met full diagnostic criteria for a disruptive behavior disorder (Kaat & Lecavalier, 2013).Disruptive behavior is common in toddlerhood, as tantrums and dysregulation may be due to children experiencing changes in autonomy as well as increasing environmental demands (Brownell et al., 2007).The display of disruptive behavior is often an external manifestation of a child's internal perspective (e.g., a child may scream to communicate that they are frustrated) (Liu, 2004).As children develop into school-age, they gain more sophisticated self-regulation and communication skills, which reduces the frequency of disruptive behavior used for a child to express their wants and needs (Brownell et al., 2007).However, if disruptive behavior persists from toddlerhood into later childhood it is associated with lower language, communication, and academic skills, and increases the risk for co-occurring disruptive behavior disorders (e.g., oppositional defiant disorder, conduct disorder [CD]) (Ashburner et al., 2008;Chow & Wehby, 2019;Kaat & Bearss, 2020).Therefore, early evaluation of clinically salient disruptive behavior is critical for timely referral to additional screenings, diagnostic evaluation, behavioral services, and facilitating optimal developmental outcomes (Krogh-Jespersen et al., 2021).Clinicians, that work with autistic children, should consider the use of assessment tools that adhere to a dimensional approach; that operationalizes a spectrum of typical (i.e., developmentally-expected) to atypical disruptive behavior, rather than categorical approaches that inform if the disruptive behavior is present or absent.Ultimately, dimensional assessment tools further inform characterization of early disruptive behaviors that may negatively impact later development (Wakschlag et al., 2014).However, few disruptive behavior assessment tools that adhere to a dimensional approach are available and have been validated in young autistic children (Wakschlag et al., 2014).Therefore, the current study aims to validate the psychometric properties of the Multidimensional Assessment of Preschool Disruptive Behavior Infant-Toddler (MAP-DB-IT) (Krogh-Jespersen et al., 2021;Zhang et al., 2023) version, a behavior rating scale following the dimensional approach for multiple types of disruptive behavior, in a sample of autistic toddlers.
The MAP-DB-IT is a parent report questionnaire that is comprised of three subdomains: temper loss, noncompliance, and aggression (Krogh-Jespersen et al., 2021).Temper loss measures the regulation of frustration (e.g., how often does your child suddenly have a temper tantrum that came out of nowhere), noncompliance evaluates a child's internalization of rules (e.g., how often does your child do something dangerous despite repeated warnings), and aggression assesses the capacity to modulate aggressive tendencies (e.g., how often does your child hit, shove, kick, bite, or pinch to try to get something they wanted) (Krogh-Jespersen et al., 2021;Wakschlag et al., 2014).Most of the aggression questions are relevant to all children, but a subset of questions relate specifically to aggression towards siblings and thus are only asked to those with siblings.The MAP-DB-IT was created based on a dimensional approach (Krogh-Jespersen et al., 2021;Wakschlag et al., 2014).The dimensional approach assesses normative disruptive behavior within childhood from clinically salient disruptive behavior by evaluating the behavior's frequency, quality, and context (Wakschlag et al., 2014).Within the MAP-DB-IT behavior is characterized by behavioral frequency (e.g., how often does your child have a temper tantrum that lasted longer than 5 min), quality (e.g., how often does your child have a temper tantrum that came out of nowhere), and context (e.g., how often does your child have a temper tantrum when you were out in public when him/her).Therefore, the specific items on the MAP-DB-IT provide a more nuanced distinction between developmentally appropriate and clinically concerning behaviors.For example, when distinguishing developmentally expected versus unexpected irritability (i.e., temper loss) using the MAP-DB-IT, during the transition to toddlerhood found that over 40% of toddlers "have a temper tantrum when s/he is hungry, tired, or sick," but only 3% "suddenly have a temper tantrum that came out of nowhere" (Krogh-Jespersen et al., 2021).This suggests that temper loss that is "out of nowhere" is exhibited at a lower rate and may be clinically concerning as  compared to similar behavior that may be due to environmental circumstances (e.g., sickness, fatigue, hunger).By using a measure informed by the dimensional approach to evaluate early childhood disruptive behavior, clinicians may be able to monitor clinical patterns, and later refer children for behavioral screenings, diagnostic evaluations, or services.
There are various disruptive behavior assessment tools that have been created for autistic children or validated in autistic samples (e.g., the Child Behavior Checklist [CBCL], the Infant Toddler Social Emotional Assessment [ITSEA], and the Aberrant Behavior Checklist [ABC], etc.) (First et al., 2013;Kaat et al., 2014;Offermans et al., 2022).Many of these measures were created based on attributes of both dimensional and categorical approaches.As such, many use diagnostic criteria to determine frequency and severity of disruptive behavior.However, the individual items within these measures may not capture the quality or the context of the disruptive behaviors that are present (Kaat et al., 2019).When accounting for frequency, quality, and context, as done in the MAP-DB-IT, individual subdomain scores encompass a variety of items that quantify the dimensionally of multiple facets of a child's disruptive behavior.
While this allows for the MAP-DB-IT to provide a more nuanced measure of disruptive behavior, the items were not originally developed for autistic toddlers.As such, the MAP-DB-IT needs to be evaluated for its ability to identify specific manifestations of behavioral psychopathology within autistic toddlers.By gathering validity evidence for the MAP-DB-IT for autistic toddlers, researchers and clinicians may be able to discriminate between early developmentally appropriate tantrums from problematic dysregulation across groups (i.e., the general population and autistic toddlers).
To date no disruptive behavior assessment tool based on the dimensional approach has been validated within a sample of autistic toddlers.Therefore, the primary purpose of the current study was to evaluate the validity of the MAP-DB-IT within autistic toddlers.With appropriate validity evidence, the MAP-DB-IT may be used to evaluate autistic Toddlers developmentally appropriate behavior from clinically salient behavior, and evaluate differences in disruptive behavior between toddlers from the general population and autistic toddlers.Specifically, we evaluated several types of validity evidence,

Study design
The current study was a secondary analysis of data from a randomized clinical trial (NCT02632773).Participants were recruited continuously between July 2015 and March 2020.This study included 82 mother-child dyads (M age = 33.2months, SD = 6.28 months).Previous work with the same sample reported 96 mother-child dyads (Jones et al., 2022); however, the MAP-DB-IT was added as an additional survey after study launch and 14 participants did not complete the survey.Participants were randomized to receive either directive or responsive communication interventions.Mothers in both intervention groups received weekly, 1-h intervention sessions for 8 weeks.

Participants
Enrolled participants were from the greater Chicagoland area and were recruited from pediatricians, autism diagnostic programs, and Part C Early Intervention providers (see Table 1).Children were eligible for the study if they were between 24 and 48 months old, had a diagnosis of autism or mothers had concerns about autism, which were later confirmed by an in-person autism assessment including the ADOS-2, and had no additional diagnosis that could impact language development (Lord et al., 2012).The original clinical trial evaluated maternal characteristics (i.e., broader autism phenotype) on intervention strategy use; therefore, caregivers were eligible if they were the biological mother to the participating child, spoke English at least 50% of the time with the participating child, and did not have a diagnosis of: Fragile X syndrome, cerebral palsy, schizophrenia, profound hearing loss, or brain or head injury where she lost consciousness.

MAP-DB-IT
The MAP-DB-IT version is a parent-report questionnaire that evaluates disruptive behavior in toddlers and was originally created for toddlers within the general population ages 12-24 months (Krogh-Jespersen et al., 2021;Zhang et al., 2023).Mothers were asked to complete the MAP-DB-IT across all time points (i.e., T0, T1, and T2).
The MAP-DB-IT consists of three subdomains: temper loss, noncompliance, and aggression.Participants completed additional questions if the study child had a sibling.As such, these items formed a 4th subdomain, hereinafter referred to as aggression with sibling items, that captures aggression with sibling and assesses the study child's capacity to modulate aggressive tendencies towards a sibling (e.g., "In the past month, how often does your child hit his/her brother or sister at the slightest provocations").Items within the temper loss domain (30 items) range from developmentally expected expressions (i.e., tantrums due to frustration) to intense expressions (i.e., dysregulated tantrums that "appear out of the blue").Items within the noncompliance domain (16 items) range from normative refusal to For the current study, the MAP-DB-IT was scored based on a mean scoring method.If the parent answered more than half the items within a domain, then the items answered were averaged and multiplied by the total number of items within the domain.If the parent answered less than half of the items within the domain, then the domain went unscored (i.e., the subdomain score was also missing).All parents across all time points met the minimum scoring criteria and no domains went unscored (see Table 2).

Infant toddler social emotional assessment
For the purposes of validating the MAP-DB-IT, we compared various psychometric properties to the ITSEA (Briggs-Gowan & Carter, 1998).
Autistic children were included in the normative sample and this measure has additionally been validated in an autistic sample (Briggs-Gowan & Carter, 1998;First et al., 2013;Mottes, 2015).Autistic children were not excluded from the ITSEAs norming sample (First et al., 2013), the ITSEA has been used to evaluate disruptive behavior in young autistic toddlers (Mottes, 2015), and researchers have used the ITSEA to screen young children for autism (Kruizinga et al., 2014).
Within the current study, mothers completed the ITSEA across all three time points.The ITSEA consists of four subdomains: externalizing, internalizing, dysregulation, and competence.The ITSEA was also scored based on the mean of the items answered within each subdomain (see Table 2) (Baxter, 2007).

Statistical methods
Following best practices in psychometric studies, we use an argumentative approach to evaluating validity evidence (American  Table S1 provides internal consistency estimates by each time point. To evaluate convergent validity of the MAP-DB-IT, with the ITSEA, we correlated scores from the T0 (i.e., baseline) time point.
We then correlated the MAP-DB-IT and ITSEA subdomains.We also ADOS-2 language level or the ADOS-2 severity level (i.e., the potential moderator factors), and their interaction.We used the default priors within the R brms package which are flat, uninformative priors for each of our regression parameters and student-t priors for the intercept and residual variance sigma (Bürkner, 2017).For each model, we used three chains with 100 burn in iterations and 3000 samples per chain from samples per chain from the posterior distribution of regression coefficients.Each model's convergence was evaluated via Rhat values, effective sample size values for each parameter, and trace plots.Finally, each model was evaluated for accurate data prediction via posterior predictive check plots.For all our Bayesian regressions, baseline data collected at the T0 time point were used.

Evidence supporting the internal consistency of the MAP-DB-IT
The most basic evidence for the reliability and validity of a measure

Convergent validity between the MAP-DB-IT and ITSEA
Next, we evaluated the convergent validity between the MAP-DB-IT and the ITSEA domain scores.Consistent with our hypotheses and expectations, we found that there was a positive correlation between  and severity score).We conducted Bayesian regression moderation analyses where the relationship between MAP-DB-IT subdomains and the ITSEA externalizing domain were allowed to vary by child characteristics.None of the models, were significant, and the magnitude of the interaction effects were quite small, suggesting that the convergent validity of the MAP-DB-IT does not vary by child characteristics.See Table 5 for these results.

Responsiveness of the MAP-DB-IT
We were greater than r = 0.6 (Table 3).When evaluating convergent change validity immediately after intervention and 3 months after intervention (T1-T2 change), there were no correlations between the ITSEA and MAP-DB-IT subdomains that were greater than r = 0.6 (Table 3).Some T1-T2 change correlations were within the

DISCUSSION
The current study evaluated the validity evidence for the MAP-DB-IT as a dimensional approach measurement for evaluating disruptive behavior in autistic toddlers.We found excellent internal consistency  Limitations to the current study include a small sample size, which restricted our analyses to evaluating cross-sectional convergent validity instead of item level factor analyses between the MAP-DB-IT and the ITSEA.Additionally, the small sample may have underpowered our moderation analyses and explain the lack of significant findings from regressing MAP-DB-IT subdomain on parent's concern about their child's disruptive behavior as moderated by child characteristics.
Furthermore, we see a small R 2 in our MAP-DB-IT temper loss and noncompliance Bayesian moderation models which may suggest that each of these models do a poor job of explaining the variance in our dependent variables.Previous research has found an association between child characteristics (i.e., ADOS-2 severity and language level) and child disruptive behavior; as such, more research investigating this association is needed (Chow & Wehby, 2019;Matson et al., 2008).
Additionally, our sample was predominately male (consistent with the sex distribution of autism in the population) and White (a common limitation in research samples).However, we had a wide range of language skills, cognitive abilities, and ADOS severity scores.Taken together, our sample lacks some aspects of diversity.

�
Disruptive behavior in autistic toddlers is typically measured using a categorical approach, which identifies the presence or absence of behavior.� Clinicians should consider the use of a dimensional approach measurement tool which assesses normative from clinically salient disruptive behavior by evaluating the behavior's frequency, quality, and context.� We evaluated the validity of the Infant/Toddler version of the Multidimensional Assessment Profile of Disruptive Behavior (MAP-DB-IT), a dimensional approach measurement tool, in a sample of autistic toddlers.� The MAP-DB-IT demonstrates appropriate validity evidence for disruptive behavior in autistic toddlers, including excellent internal consistency and convergent validity between subdomains of the MAP-DB-IT and the Infant Toddler Social Emotional Assessment.
including (a) the internal consistency of the MAP-DB-IT domain scores; (b) the convergent validity of the domains with a more generic measure of toddler behavior (i.e., the ITSEA); and (c) the responsiveness-operationalized as convergent change-of the MAP-DB to intervention.Further, we evaluated whether the convergent validity between the MAP-DB-IT and ITSEA varied based on child characteristics such as child language level or severity of autism (quantified by the autism diagnostic observation schedule [ADOS-2]).
MAP-DB-IT FOR AUTISTIC TODDLERS refusal due to uncooperative disobedience.Items within the aggression domain (20 items) include normal reactive aggression to developmentally unexpected display of intentional or hostile aggression.Items within aggression with sibling items were only provided to participants with siblings (n = 43, completed across any timepoint) and captured similar aggression items to the aggression subdomain but as they pertain to interactions with a sibling (five additional items).Within each of the subdomains, parent's reported behavior on a sixpoint frequency scale (0 = Never; 1 = Rarely [less than once per week]; 2 = Some [1-3] days of the week; 3 = Most [4-6] days of the week; 4 = Every day of the week; 5 = Many times each day).
Council on Measurement inEducation, 2014;Gagnier et al., 2021).As such, we evaluated multiple types of validity evidence, which when combined would support the use of the MAP-DB-IT among this population and context.We first examined the internal consistency reliability of the MAP-DB-IT domain scores using Cronbach's alpha for each of the four subdomains: temper loss, noncompliance, aggression, and aggression with sibling items.Participant's MAP-DB-IT domain scores were randomly selected from one of three time points (i.e., T0, T1, or T2).Randomly selected MAP-DB-IT time points were chosen to ensure that Cronbach's alpha was not confounded by assessment occasion and represented the full range of scores on the scale.
examined the correlation between the change scores for the MAP-DB-IT and ITSEA over time to index responsiveness via convergent change.The responsiveness via convergent change allows us to evaluate the validity of a change score.Therefore, change between the MAP-DB-IT and ITSEA was examined from (a) T0 to T1, and (b)T1 to T2.Based on the COSMIN guidelines for validity studies(Gagnier et al., 2021), a priori hypotheses are needed to establish convergent and responsive validity.We set an a priori hypothesis of correlations within the range of approximately 0.6 with a null offset of 0.4 (i.e., correlations with confidence intervals that contain values at or below 0.4 will not be considered supporting the validity of the MAP-DB-IT even if statistically significant).Finally, we calculated the magnitude of change across time for each of the MAP-DB-IT and ITSEA subdomains.To assess if the convergent validity of the MAP-DB-IT varies by child characteristics such as severity and language level as measured by the ADOS-2, we conducted moderation analyses using Bayesian regression models for the temper loss, noncompliance, and aggression MAP-DB-IT subdomains.Within our models, we regressed ITSEA externalizing domain raw score on MAP-DB-IT domain score, within a new population is evidence supporting its internal structure and internal consistency.As such, we calculated Cronbach's alpha for each of the MAP-DB-IT subdomains.The temper loss subdomain yielded a Cronbach's alpha of α = 0.95 [CI: 0.94, 0.97], the noncompliance subdomain was α = 0.90 [CI: 0.87, 0.93], the generic aggression subdomain was α = 0.93 [CI: 0.91, 0.94], and the aggression with sibling items (i.e., all 25 aggression items and an additional five aggression with siblings items) was α = 0.95 [CI: 0.93, 0.96].
the (a) ITSEA externalizing and MAP-DB-IT aggression domain (r = 0.62, p = 0.00 [CI: 0.47, 0.74]), (b) ITSEA externalizing and MAP-DB-IT aggression with siblings domain (r = 0.59, p = 0.00 [CI: 0.43, 0.72]), and (c) ITSEA dysregulation and MAP-DB-IT temper loss domain (r = 0.55, p = 0.00 [CI: 0.41, 0.67]).See evaluated the convergent change between the MAP-DB-IT and the ITSEA as evidence for the responsiveness of the MAP-DB-IT to change over time.We evaluated convergent change by correlating the subdomains from (a) T0 before intervention to T1 immediately after intervention, and (b) T1 immediately after intervention to T2 3 months after intervention.When evaluating convergent change before and after intervention (T0-T1 change), there were no correlations between the ITSEA and MAP-DB-IT subdomains that
across all three subdomains, suggesting the MAP-DB-IT measures subdomains of disruptive behavior reliably.We saw that there were positive correlations between the (a) ITSEA externalizing and MAP-DB-IT aggression domain, (b) ITSEA externalizing and MAP-DB-IT aggression with siblings domain, and (c) ITSEA dysregulation and MAP-DB-IT temper loss domain.Taken together, these findings suggest that the psychometric properties of the MAP-DB-IT have sufficient validity evidence to use as a measurement tool for autistic children with disruptive behaviors.However, our findings suggest that the responsiveness of the MAP-DB-IT is different from the responsiveness of the ITSEA; they evaluate change over time differently, reducing evidence for the responsiveness of the measure.It may be that the social communication interventions had an effect on children's disruptive behavior as conceptualized by only one of the measures but not both measures.Further research should consider the evaluation of additional child outcomes outside of intervention targets (Curtis et al., 2019).Therefore, the two measurement tools may be evaluating disruptive behavior differently across time due to the effects of the social communication intervention.Alternatively, it may be that the more nuanced quantification of disruptive behavior by the MAP-DB-IT can assess smaller changes than the more global assessment provided by the generic externalizing behavior subdomain on the ITSEA.Future research may consider how the specific dimensional or categorical characteristics of measurement tools affect change validity.Additionally, we did not find evidence that convergent validity of the MAP-DB-IT varied by child characteristics (i.e., ADOS-2 language level and severity score); suggesting that there were not significant differences between the ITSEA externalizing domain and MAP-DB-IT subdomain scores depending on subgroup membership for the child.The MAP-DB-IT was originally created for young children within the general population ages 12-18 months (Krogh-Jespersen et al., 2021).It is critical to evaluate measurement tools originally created for the general population as they may not generalize to subpopulations' individual characteristics or psychopathology.Therefore, our analyses aimed to evaluate the psychometric properties of the MAP-DB-IT within a specific clinical subgroup (e.g., autistic toddlers).As such, our study's findings have broad implications for clinicians working with young autistic children who display disruptive behaviors as they can (a) use the MAP-DB-IT to discriminate between early developmentally appropriate tantrums from clinically salient dysregulation, and (b) compare children's MAP-DB-IT scores (i.e., contrast scores of autistic children to children in the general population).
Future longitudinal research should consider how the MAP-DB-IT evaluates disruptive behavior over time for a large, diverse group of autistic children (e.g., race, ethnicity, biological sex, SES).Finally, without additional diagnostic information we were unable to evaluate the diagnostic validity of the MAP-DB-IT.As such, future research should consider how the MAP-DB-IT predicts children's additional disruptive behavior disorder diagnoses.Despite these limitations, our findings suggest the MAP-DB-IT is a valid dimensional approach disruptive behavior measurement tool for autistic children.Disruptive behavior is common in autistic toddlers, however clinically significant levels of disruptive behavior can lead to negative long-term outcomes (Ashburner et al., 2008; Chow & Wehby, 2019; Kaat & Bearss, 2020).Therefore, it is important clinicians use behavioral measurement tools with autistic toddlers to refer them to further behavioral services.Dimensional approaches, such as the MAP-DB-IT, provide a developmentally informed metric for characterizing disruptive behavior on a spectrum from developmentally expected typical behavior to unexpected (or atypical) clinically significant concerns.However, there are few dimensional approaches validated in a young autistic sample.The current study provides preliminary psychometric validation to extend the use of the MAP-DB-IT to autistic toddlers.As such, the MAP-DB-IT may be used by a variety of clinicians as a measurement tool for autistic toddlers who may benefit from additional behavioral services or further clinical evaluations.
Mother and child demographic characteristics.
T A B L E 1 Convergent validity and convergent change validity correlations between ITSEA and MAP-DB-IT subdomains.
Abbreviations: ITSEA, infant toddler social emotional assessment; MAP-DB-IT, multidimensional assessment profile of disruptive behavior infant/ toddler.T A B L E 3 Bayesian regression moderation models of ITSEA externalizing domain as regressed on MAP-DB-IT subdomains, including the ADOS-2 language ability, and ADOS-2 severity level.
T A B L E 5