This project constituted the master's thesis completed by the first author under the supervision of the second author to fulfill graduation requirements for the Applied Behavior Analysis program at Youngstown State University. We thank Cecelia Maderitz and Vilmary Placeres for their assistance with data collection.
A further evaluation of behavioral skills training for implementation of the picture exchange communication system
Article first published online: 16 JAN 2014
© Society for the Experimental Analysis of Behavior
Journal of Applied Behavior Analysis
Volume 47, Issue 1, pages 198–203, Spring 2014
How to Cite
Homlitas, C., Rosales, R. and Candel, L. (2014), A further evaluation of behavioral skills training for implementation of the picture exchange communication system. Jnl of Applied Behav Analysis, 47: 198–203. doi: 10.1002/jaba.99
- Issue published online: 7 MAR 2014
- Article first published online: 16 JAN 2014
- Manuscript Accepted: 6 OCT 2013
- Manuscript Received: 5 JAN 2013
- behavioral skills training;
- picture exchange communication system;
- staff training
We evaluated the effectiveness of a behavioral skills training package to teach implementation of Phases 1, 2, and 3A of the picture exchange communication system (PECS) to teachers employed at a therapeutic center for children with autism. Probes in the natural environment and follow-up were conducted with children who were assigned to work with the teachers in their own classrooms. Results provide additional support for the efficacy of behavioral skills training to teach implementation of PECS.
The picture exchange communication system (PECS; Frost & Bondy, 2002) has become an increasingly popular, easily implemented means to teach an alternative form of communication (Sulzer-Azaroff, Hoffman, Horton, Bondy, & Frost, 2009). This augmentative communication system consists of six phases: Phase 1 establishes physical exchange of a picture with a communicative partner; Phase 2 teaches distance and persistence; Phase 3 teaches picture discriminations; Phase 4 introduces sentence structure; Phase 5 incorporates attributes; and Phase 6 introduces commenting. To date, only a handful of studies have empirically evaluated training protocols to teach proper implementation of PECS to caregivers.
Although instructions and video alone are not sufficient for direct-care staff to demonstrate mastery of Phases 1 to 3B of the PECS protocol (Barnes, Dunning, & Rehfeldt, 2011), training packages that incorporate instructions, modeling, rehearsal, and feedback have successfully been used to teach Phase 1 of this protocol to direct-care staff (Wood, Luiselli, & Harchik, 2007), Phase 5 to parents of children with autism (Ben Chaabane, Alber-Morgan, & DeBar, 2009), and Phases 1 to 3A to university students (Rosales, Stone, & Rehfeldt, 2009). Rosales et al. (2009) used a training package that included a video, verbal and written instructions, modeling, rehearsal, and feedback using confederate learners, and later assessed the generalization and follow-up when adults with developmental disabilities served as learners. All participants required remedial training to demonstrate mastery of the skills acquired during training in the generalization phase, and only adult learners were recruited.
One focus of training the implementation of PECS involves ensuring that the teaching procedures are conducted with integrity. Moreover, the results of effective training must be assessed in the natural environment in order to evaluate the generalization and maintenance of any skills acquired by caregivers. Therefore, the purpose of the present study was to replicate the procedure outlined by Rosales et al. (2009) with teachers as participants and to evaluate the results of training when children with autism with no formal experience with PECS served as the learners in their typical classroom environment.
Participants, Setting, and Materials
Three teachers from a therapeutic center for children with autism volunteered to be participants. Each participant had a bachelor's degree and had worked at the center for 6 to 18 months at the start of the study. All participants had minimal (Teacher 2) or no (Teachers 1 and 3) experience implementing PECS. All participants attended a professional development training session on Phase 1 of PECS 6 months prior to the start of this study. The session included a didactic presentation, modeling by an instructor, and role-play with the participants and instructor. Feedback and evaluation of the skills that were acquired following this brief training session were not provided to the teachers at any time.
Confederates (the second author and a graduate student with previous training in the PECS protocol) served as learners during baseline and training. Each confederate received a written list of potential responses a learner with developmental disabilities might emit during each phase of PECS and was instructed to emit each response on the list at least once during each five-trial block (see Table 1). Nine students (2 to 7 years old) with a diagnosis of autism and a limited vocal verbal repertoire were recruited as learners for probes in the natural environment and follow-up sessions. The students had experience exchanging single pictures for food and highly preferred toys during specific times of the day (i.e., in the lunchroom and play area) but no formal training in PECS. Each teacher was assigned to work with one to three students from their own classrooms during these probes.
|1. Pick up the picture card and reach out towards the trainer.|
|2. Reach for the item without picking up the card first.|
|3. Pick up the card but do not reach towards the teacher, or play with the card.|
|4. Make no response; do not pick up the card or reach for the item.|
|5. Pick up the card and reach out to the trainer only after he or she provides an open hand prompt.|
|6. Pick up the card and reach out to the trainer only after he or she provides a physical or gestural prompt.|
|7. Pick up the card and throw it to the ground or at the trainer.|
|8. Get out of your seat.|
|9. Reject the item.|
|1. Pick up the card, walk over to the trainer, and reach out with the card in hand.|
|2. Pick up the card but don't move towards the trainer; play with the card.|
|3. Walk directly to the trainer without picking up the card first.|
|4. Hold out your hand or try to reach for the item without picking up the card first.|
|5. Pick up the card and walk towards the trainer only after he or she provides a gestural prompt.|
|6. Pick up the card and throw it at the trainer or on the ground.|
|7. Respond correctly and independently two consecutive times.|
|8. Wait until prompted two times in a row.|
|1. Select the preferred item card and reach out towards the trainer.|
|2. Select the distracter card and reach out towards the trainer.|
|3. Select the preferred item card and throw it at the trainer or on the ground.|
|4. Select the distracter card and throw it at the trainer or on the ground.|
|5. Select the preferred item card only after the trainer goes through the correction procedure.|
|6. Select the distracter card again after the trainer goes through the correction procedure.|
|7. Reach for the preferred item without picking up or exchanging the card.|
|8. Select the distracter card and reach out to the trainer to exchange the picture, then immediately reach for the preferred item picture after the trainer provides access to the distracter item.|
|9. Pick up both the preferred and distracter cards.|
Baseline and training sessions were conducted in an isolated room of the center. Natural environment probe sessions were conducted in the teacher and students' regular classroom in an area designated for one-on-one instruction. All sessions were recorded using a Digital Flip or Kodak Easy Share video camera to score interobserver agreement later. Other materials included handouts with information on the first three phases of PECS, several trinkets designated as preferred for confederates, preferred items for students (determined by a formal preference assessment), data sheets created for the purpose of the study, laminated digital pictures of preferred and neutral items (5.1 cm by 5.1 cm), and a 1-in. binder with strips of hook-and-loop tape. Training and natural environment probe sessions were conducted two to three times per week for 15 to 30 min. One five-trial block was conducted per session.
A multiple baseline design across participants was employed. Baseline sessions began concurrently across teachers and continued until stability was demonstrated. This was followed by implementation of the training package for Phases 1 to 3A of PECS. After participants met mastery criterion for each phase, natural environment and follow-up probes were conducted.
Dependent Measure, Interobserver Agreement, and Treatment Integrity
The primary dependent variable was the percentage of correct responses on checklists similar to those used by Rosales et al. (2009). The percentage of correct responses was calculated by summing the total number of correctly performed steps and dividing this number by the total possible responses in each five-trial block, then converting this number to a percentage. A correct response was defined as the teacher emitting a response as described on the checklist. Any deviation from the description on the checklist was scored as incorrect. Steps did not have to be performed in the order listed, with the exception of those that required completion of a previous step to be executed (e.g., wait 1 or 2 s before administering a prompt). Steps not applicable were marked N/A and omitted from calculation of responses.
The interobserver agreement data were collected across all sessions and teachers by secondary trained observers, and were calculated by summing the number of agreements from each observer's scored checklist, dividing by the total possible responses, and converting the result to a percentage. A total of 41% of all sessions were scored for Teacher 1 (M = 94%; range, 82% to 100%); 42% for Teacher 2 (M = 93%; range, 77% to 100%); and 44% for Teacher 3 (M = 95%; range, 88% to 99%).
Secondary trained observers also scored treatment integrity via in vivo observations using a checklist created for the purpose of this study (available from the second author). The percentage of correct responses was determined by summing the total number of correctly performed steps and dividing this number by the total possible responses. Treatment integrity data were collected for 32% of all sessions for Teacher 1 (M = 100%), 34% for Teacher 2 (M = 98%; range, 86% to 100%); and 33% for Teacher 3 (M = 100%).
Before baseline, a paired-stimulus preference assessment (Fisher et al., 1992) was conducted for all students. The two most frequently selected items were used during subsequent probes.
Teachers were given information on Phases 1 to 3A based on the training manual developed by Frost and Bondy (2002), were given the opportunity to read through this material before baseline probes, and had unlimited access to these materials for the duration of the study.
Teachers were given all materials necessary to conduct each phase of PECS and were instructed to conduct one five-trial block. For the purposes of data collection, a trial was terminated after a successful picture exchange. One trial block per phase of PECS was conducted at a time with no feedback from the experimenter. During each session, confederates referred to the menu of responses available and emitted each response on the list at least once per trial block. Several responses on the checklist could be emitted during a single trial. For example, during Phase 3, Responses 3, 7, and 8 could be performed in a single trial.
Teachers were first given verbal instructions and a corresponding response checklist of Phase 1 of PECS (identical to that used by the experimenter to score teacher responses). The experimenter then modeled one five-trial block with a confederate while the teacher observed. The teacher then rehearsed one five-trial block with a confederate. Teachers were asked to refrain from talking or asking questions until the completion of five trials but were given an opportunity to ask questions before or after the completion of each trial block. After the completion of each trial block, teachers were provided with both corrective and positive feedback by the experimenter. Modeling, rehearsal, and feedback continued until mastery criterion was attained for Phase 1 (90% correct independent responses across three consecutive five-trial blocks). Training was provided in a sequential order for Phase 2 and Phase 3A in an identical manner.
Natural environment probes
After teachers demonstrated mastery for Phases 1 to 3A with a confederate, probes were conducted with a student learner in the assigned classroom. The procedure was identical to baseline.
Follow-up probes were conducted in a manner identical to baseline, but with a student in their regularly assigned classroom 1 month after the last natural environment probe.
RESULTS AND DISCUSSION
Figure 1 represents the percentage of correct responses during all phases across teachers. Throughout baseline, no teachers demonstrated criterion performance for any phase of PECS. After implementation of BST, mastery criterion was met by all teachers after no more than four five-trial blocks per phase. During Phase 2, additional training sessions were conducted to give teachers an opportunity to practice all responses outlined on the checklist for this phase at least once (e.g., an opportunity to practice moving away from the learner and then also moving the binder away from the learner). During natural environment and follow-up probes, teacher responses remained at criterion levels (i.e., at or above 90% accuracy).
The results of the present study provide additional support for the use of BST to teach professionals to implement the first three phases of PECS (Barnes et al., 2011; Rosales et al., 2009). All three teachers performed at or above criterion levels during natural environment and follow-up probes when multiple children from their own classrooms served as learners, and no remedial training was required. This may be attributed in part to the stringent mastery criterion, which was set in an effort to ensure that critical steps of each phase were implemented accurately. Practitioners and researchers who teach multiple responses to caregivers may consider a similar approach for mastery criterion. A more stringent criterion may prevent completion of training when critical components of the skills to be acquired are missing from participants' repertoires.
Future research should address limitations of the present study. First, careful consideration should be given to confederate responding during training. In the present study, confederate learners were provided with a menu of responses and were specifically instructed to emit each response on the list at least once per five-trial block. However, a script was not provided for each session, and treatment integrity data were not collected on confederate responses. Scripts for confederate learners would ensure consistency in the level of difficulty across all training sessions and the opportunity for teachers to practice all of the relevant steps for each phase of the PECS protocol.
Second, the short latency between the end of training and Phase 1 natural environment probes may have resulted in a more accurate representation of the outcomes of training. In the current study, natural environment probes for Phases 2 and 3 were delayed until the student learners had completed Phase 1. Because of the delay that this procedure introduced between training and some natural environment probes, those later probes may also be considered early follow-up evaluations. Despite this delay, PECS skills were maintained across all natural environment probes for all participants without additional training or feedback.
Third, it may be recommended for future research to conduct natural environment probes during baseline similar to those conducted in the present study after training. These probes could help determine generalization effects of skills acquired in analogue training conditions to naturalistic teaching contexts.
Finally, future studies should investigate strategies to teach implementation of PECS to multiple participants over time without the continued presence of a behavioral consultant. The use of pyramidal training has demonstrated efficacy with other commonly used assessments, and may therefore be one avenue to accomplish this goal (Pence, St. Peter, & Tetreault, 2012).
- 2011). An evaluation of strategies for training staff to implement the picture exchange communication system. Research in Autism Spectrum Disorders, 5, 1574–1583. doi: 10.1016/j.rasd.2011.03.003 , , & (
- 2009). The effects of parent-implemented PECS training on improvisation of mands by children with autism. Journal of Applied Behavior Analysis, 42, 671–677. doi: 10.1901/jaba.2009.42-671 , , & (
- 1992). A comparison of two approaches for identifying reinforcers for persons with severe and profound disabilities. Journal of Applied Behavior Analysis, 25, 491–498. doi: 10.1901/jaba.1992.25-491 , , , , , & (
- 2002). The picture exchange communication system training manual. Newark, DE: Pyramid Educational Products. , & (
- 2012). Increasing accurate preference assessment implementation through pyramidal training. Journal of Applied Behavior Analysis, 45, 345–359. doi: 10.1901/jaba.2012.45-345 , , & (
- 2009). The effects of behavioral skills training on implementation of the picture exchange communication system. Journal of Applied Behavior Analysis, 42, 541–549. doi: 10.1901/jaba.2009.42-541 , , & (
- 2009). The picture exchange communication system (PECS): What do the data say? Focus on Autism and Other Developmental Disabilities, 24, 89–103. doi: 10.1177/1088357609332743 , , , , & (
- 2007). Training instructional skills with paraprofessional service providers at a community-based habilitation setting. Behavior Modification, 31, 847–855. doi: 10.1177/0145445507302893 , , & (
Action Editor, Claire St. Peter