Journal of Computer Assisted Learning

1.1 The learning target

Our learning target was derived from evidence concerning the earliest foundations of mathematics. Research with neurotypical children reveals that during the first 5 years (and beyond), children are able to make finer and finer discriminations between quantities that differ in the ratio of one to another. Children at 6 months, for example, are able to distinguish between arrays that differ in the ratio of 1:2; ratios of 2:3 at 9 months; 3:4 at 3 years; and 5:6 at 6 years (Brannon, Abbott, & Lutz, 2004; Halberda & Feigenson, 2008). Children with DS on the other hand appear to struggle in discriminating between quantities that vary by a ratio of 2:3 (Camos, 2009; Paterson, 2001; Paterson, Girelli, Butterworth, & Karmiloff‐Smith, 2006), and this may explain some of their difficulty with mathematical skills. Research with typically developing children suggests that these early visual skills of magnitude representation provide a foundation for later mathematical learning (Gilmore, McCarthy, & Spelke, 2010; Libertus, Feigenson, & Halberda, 2011). Our aim therefore was to develop an app that would strengthen these skills through a game mechanic that required the discrimination of quantities where the level of difficulty was determined by ratio rather than absolute number.

1.2 Learning and children with Down syndrome

Although research does not provide good evidence to suggest a distinct pedagogic approach for children with DS (Wishart, 2005), there are a number of potential areas of difficulty that impact on their learning. Of particular relevance here, are aspects of attentional control. For example, Borella, Carretti, and Lanfranchi (2013) illustrate their difficulty in controlling attention to irrelevant and distracting information and suppressing responses to it. J. G. Wishart and Duffy (1990) report fluctuating levels of engagement across a range of tasks and how this impedes both acquisition of new skills and understanding and the consolidation of previous learning. Children's disengagement occurs in the face of success as well as failure (Faragher et al., 2008), suggesting that issues of motivation need to be considered alongside task difficulty.

Children with DS are more likely than their peers to experience auditory impairment (Park et al., 2012), and the incidence of visual impairments is also higher with refractive errors increasing in childhood and near vision more likely to be consistently out of focus (Cregg et al., 2001). Both impact on the clarity of information to be processed. Visual processing appears to be stronger than auditory processing (Baddeley & Jarrold, 2007) although this does not hold true for every context; in part, it depends on the type of presentation (whether sequential or simultaneous) and the kind of visual spatial task that is presented (Yang, Conners, & Merril, 2014). There is some evidence that these learners do better at discrimination tasks when the information is located in a particular part of the screen rather than dispersed across the screen (Belacchi et al., 2014). Of relevance to their interaction with technology, children often experience a number of difficulties in making precise fine motor movements (Fidler & Rogers, 2006).

These characteristics have a number of design implications. There are important advantages to presenting material visually, but it must have a visually clear display, where the important elements are distinctive and easy to locate. It should not be reliant on auditory cues or require delicate motor responses. If it is to be accessible to all children with DS, it requires careful attention to the levels of difficulty and to the type of feedback and reward.

1.3 Math intervention and children with Down syndrome

Browder, Spooner, Ahlgrim‐Delzell, Wakeman, and Harris (2008) in a comprehensive review of mathematics instruction with children with severe and moderate intellectual disabilities reveal that two properties are characteristics of successful interventions; a systematic approach and “in vivo” learning. The majority of intervention studies target two areas: numbers and operations (adding and subtracting) and money. None target the area of magnitude representation. Lemons, Powell, King, and Davidson (2015), in a more recent review of the efficacy of math intervention programs focusing specifically on children with DS, found nine papers with a robust methodology that report learning outcomes. Only one paper provides clear evidence using computer‐assisted learning. Features of the other eight nondigital studies that met their inclusion criteria include direct instruction and modelling of responses, fading the support given, and using praise. The single paper using computer‐assisted learning that met their criteria compared the use of a multimedia software program with traditional paper and pencil methods to teach counting and cardinality (Ortega‐Tudela & Gómez‐Arizaw, 2006). It was a relatively small study with 10 children with DS in an experimental group and 8 in the control, but significant differences in favour of the multimedia group were found across all aspects of counting and cardinality. The authors argue that in part the nature of the feedback given reinforced children's understanding of the task. More recently, Agheana and Duta (2015) used a commercially available preschool app to successfully teach how to count, add, and subtract. Eight children with DS outperformed a matched group who received practice with concrete materials. The authors emphasize the possibilities of multimedia materials that utilize “dynamic and appealing” presentations (p. 45).

1.4 Evaluation of the app

The lack of robust studies may well reflect the problematic nature of evaluating an iPad game alongside its development. Typically, evaluations focus on outcomes—indeed much of the literature is reviewed with respect to whether specific learning outcomes are reported and then whether the methodology or research design has been sufficiently robust to draw conclusions that those outcomes were a result of that intervention. Stephenson and Limbrick (2015) in reviewing evidence on the use of tablets with people with developmental disabilities identified 251 refereed outputs of which 36 met their screening criteria. They do not report on qualitative data. Qualitative data provide the opportunity to understand the process of learning and how particular outcomes are reached and for designers to learn from it. Evidence of the way that children interact both with the tablet and with the game is particularly important in guiding new developments (Marcedo, Trevisan, Vasconcelos, & Clua, 2015), especially where it is placed alongside theories of how children learn and in this particular case an understanding of the challenges they may face. Focusing only on quantitative data may effectively screen out valuable insights, ones that guide and shape refinements. A flexible design can also offset the limitations of small‐scale studies through taking each iteration to new groups.

Given the wide range in ability of children with DS, some standard comparator is required to identify differences between groups. Although studies often use non‐verbal IQ or mental age measures, this is usually because the authors wish to compare the ability of children with DS with that of typically developing children. That was not the purpose here as we were interested in how children interacted with the game. In this study, we used children's performance on a nondigital game, one that also incorporated the choice of different quantities. In mathematics, changes in context can effectively change the nature of the activity, as children may utilize everyday understandings and procedures in one context but not in another (Lave, Murtaugh, & de la Rocha, 1984). Performance on a nondigital task serves to raise interesting questions about these contextual differences. Whereas the digital game was subject to changes and developments, the parallel game was a fixed entity throughout the period.

The following questions guided this study: Are children with DS engaged by the game? Are they able to discriminate arrays of 2:3 using a tablet game? Is their profile of attainment better or worse than with a parallel card game? How do they interact with features of the game? The project was research council funded as a rapid research and development activity, and therefore, there was no scope for evaluating the app in relation to progress in mathematics. However, addressing these questions was an important starting point for understanding the potential of the game.

2.1 Participants

The study included children aged 3.5–19 years with more boys than girls, reflecting the higher incidence of DS in males (Bishop, Huether, Torfs, Lorey, & Deddens, 1997). Children in the first two cycles were recruited through a DS organization and mainly attended mainstream schools. In the final cycle, we approached two special schools in another region to avoid a skewed sample. Table 1 sets out the details of our participants.

Ethical procedures followed the university guidelines with formal approval granted. Written consent was sought individually from parents and verbally from children. We described the purpose of our research activity in simple terms to children—that they were helping us in the development of a new game—and invited them to take part. They were seen individually in a quiet room accompanied by a parent or teaching assistant, or in a few instances, due to their own preferences, they took part in the activity alone. We were mindful of the need for an “ethical radar” (Skanfors, 2009), responding immediately to non‐verbal signs that they wished to discontinue the activity.

2.2 Procedures

Children in all three cycles received two tasks that were counterbalanced in order between children.

iPad game: The consistent feature of the game involved Millie, “an intrepid space explorer” whose path across the planets is strewn with obstacles and who requires the player's help. The central game mechanic hinges on the player being able to look at two groups of dots and correctly choose which group has more dots so that her path is made smooth. A demonstration and practice trial was given prior to their independent play. Responses of correct or incorrect choice were recorded. Observational field notes were kept of the child's level of interest in the game and their engagement with different features as they were introduced.

Card game: A series of card pairs were presented; on each was depicted a random array of dots. Each pair of cards utilized dots of the same colour and format, differing only in quantity. A large and a small set size were presented of each of the following ratios: 1:3 (1:3, 3:9); 1:2 (1:2, 4:8); 2:3 (2:3, 4:6); 3:4 (3:4, 6:8); 5:6 (5:6, 10:12); and 10:11. The child was invited to pick the card “which has more” and to turn it over to see if it had a star on the back, indicating that they were correct. Children were given a demonstration and practice trial prior to completing the set. Small and large sets were alternated. Responses were recorded of the correct or incorrect choice of “more.”

Data analysis: Quantitative data of children's correct responses in relation to ratio and set size were analysed using SPSS to provide descriptive statistics and boxplots and explore skewedness and kurtosis, which indicated the need for nonparametric statistics. Wilcoxon signed‐rank tests were used to compare differences between children's performance on the iPad and with the cards.

The data in Table 2 and Figure 1 reveal the range of responses, with performance on our target ratio of 2:3 being above chance (mean 68.9%; median 69%) and surprisingly slightly higher than ratio 1:2.

The boxplot shows the higher degree of dispersion and lower median scores of pupils' discriminations of ratio 1:2 (Mdn 58) compared with 2:3 (Mdn 69) but with no statistically significant difference between the two (N = 16, T = 103, p = .07, z = 1.8).

Ten children also played the card game that gave some indication of the extent to which their responses varied with the format.

As Table 3 reveals, this subgroup represented the range of responses on the iPad game. As with the larger group, they did very slightly better on the harder 2:3 discriminations with average responses above the level of chance. Their performance with the card game however showed, in accordance with the literature, greater difficulty with discriminations that varied in the ratio of 2:3. Two children performed better on the iPad game, and eight performed better with the cards, but there was no significant difference at a group level between their performance on the iPad and the card game for ratio 1:2 (T = 33, p + .56, z = 0.58) or for ratio 2:3 (T = 16.5, p = .48, z = −0.7).

Our observational field notes revealed that some parents and children perceived the game to require counting skills. For example, two parents spontaneously said when their child got correct responses that it was “pot luck” (they couldn't count that many) and that they were guessing. One child tried counting the dots to make her selection. Counting transformed the activity to one of absolute rather than relative number and therefore a harder and more advanced task. This had important implications for the format of the mechanic that, because of its structured appearance and likeness to a dice, unintentionally reinforced the message that this was a game around counting.

The children displayed rapid repeated tapping, their desire to make something happen outstripping their recognition of the rules of the game. The home button proved particularly attractive, its iconic features drawing their attention. Pressing this button led back to the beginning of the game, delaying the reward of Millie dancing. Thus, children were effectively distracted from the task. Some children found it tricky to look at both the mechanic and Millie and focused on one element to the exclusion of the other. This militated against recognizing the impact of their response. This was particularly problematic for those children who wore glasses whose field of vision was lessened. Some parents suggested that their child would find it more engaging if there was sound.

Data from 12 children who tested this version is set out in Table 4 and Figure 3. Children in this second iteration were slightly younger than the first, but the same broad range of responses were seen, varying from incorrect to near perfect scores. Their average performance was around the level of chance with the easier ratio of 1:2, slightly higher for 2:3 but, as shown in the boxplot, with little difference between the medians. A Wilcoxon signed‐rank test revealed that there was no statistically significant difference between performance on 1:2 ratio (median 53) than 2:3 (median 56), N = 11, T = 41, p = .45, z = 0.71.

A subsection also played the card game, and their responses are listed in Table 5. In both contexts, they did better with the 2:3 ratio, their higher group average performances being with the cards but with no significant difference between performance on the iPad and the card game for ratio 1:2 (T = 29.5, p = .4, z = 0.83) or ratio 2:3 (T = 21, p = .67, z = 0.42).

Individual data for the nine children revealed that all four girls and one boy performed better on the card game than with the iPad for both ratios, three boys performed better on the iPad than the cards, and one boy showed a mixed profile.

Our observational field notes suggest that a number of children appeared to be disadvantaged by the iPad game. Children continued to make repeated and frequent taps, and although these were more precisely located on the buttons, it could not be described as “minds on” activity. The change in appearance had brought the game mechanic closer to the learning goal but possibly made the game less accessible, although comparison between the children's responses in Cycles 1 and 2 is not straightforward given small group sizes, large age, and ability range, all of which preclude against hard and fast judgements. The focus of the next iteration therefore lay with the provision of supports as well as progression within the game, so that children at the earliest levels experienced shorter and easy games where they would learn the rules, and for those who already understood these, there would be further levels of difficulty.

In total, 36 children aged 5–19 years, 21 boys and 15 girls, tested the third iteration. Of these, 30 also did the card game. Children started with Water World (ratio 1:4) before going on to Native World (ratio 2:3). If they encountered difficulty, the ratio between the quantities changed to 1:3 gradually returning to 2:3 when they were successful.

Table 6 sets out the responses of this larger group. Average scores for both planets were high: 95% for players of Water World (ratio 1:4) and 85% for Native World (with ratios of 2:3 and 1:3). In contrast to the previous iterations, the boxplot in Figure 5 shows a skewing with median scores above 80% but with a long tail to the data for Native World and six outliers (see below) for Water World. There was a significant difference between pupils' scores on the two planets (N = 29, T = 21.5, z = −3.27, p = .001, effect size 0.43) with children's scores on the easier ratio 1:4 higher.

The change in ratios of the third iteration of the iPad game makes performance on the card game no longer directly comparable, but it is useful for indicating that the group, while spanning the usual range of ability, revealed higher mean scores than previous groups, with many scoring at ceiling level for both ratios (see Table 7).

This third version proved to be accessible for most players, with four exceptions. Three children had behavioural support needs that impeded their independent playing of the game. One, for example, required the assistant to tap the quantity and only intermittently did so himself. The fourth player had a significant visual impairment, so the display with fewer dots was more easily seen and therefore more salient. Additionally, due to technical problems, we did not record the specific responses of two children. Notably, however, on Water World, 26 of the 30 children had at least one game where they scored 80% or more, providing a clear indication that they both understood the game and were able to do it.

The new planet, Native World, with its smaller ratios proved more challenging. The games were longer as Millie's walk to the end pole involved more crevasses, and these were often deeper, requiring repeated “more” choices as brick by brick the path was restored. Only eight of the group managed to score the full 100% on one or more games. However, 23 children had scores above 80% and therefore received collectibles in reward for a high score. A few children went on to try other planets; seven tried Forest World, which required a choice of “less” for arrays of 2:3. This shift to choosing the opposite proved tricky, but three succeeded scoring 70–100%, and one young man tried the next planet, which required responses of both “more” or “less” depending on whether Millie was faced with a crevasse or a tower of bricks, a very challenging game requiring close attention, and he scored 86%.

There were some interesting patterns of responding on this iteration, revealing different player profiles. The motivation for some was clear, to get a high score and to receive a collectible. Often they played more slowly and carefully. A few gave self‐prompts signing “more” before the selection. Typically, these players did well. In contrast, the sound of a mistake was hugely amusing for some children, and we could not be sure whether it provided an alternative goal to a correct selection. Conversely, nine players did not like making errors or the noise and became reluctant to continue. These players often shifted from using a magnitude to a locational cue, selecting the other side on their next turn as if burnt by the previous selection. If this was also a mistake, they shifted sides again. Thus, errors often led to a string of wrong answers and the player withdrawing from the game. Unusually, one player returned to the Water Planet when he started making mistakes on Native World and returned to play the harder game when he got 100% right, confidence restored.

The inclusion of the “collectibles” and a short simple version of the game gave experience for those whose understanding was less well established. The activity of putting the collectibles in the rucksack meant that more or less time was spent on the harder quantity choice elements.

3.1 Limitations and future directions

One of the challenges of planning research around the design process is of not knowing what the outcome of the design will be. The unforeseen developments here meant that our standard comparator, the card game, was no longer completely fit for purpose, as it employed different ratios to the final version of the game. However, the card game did provide a stable indication of the differences between the groups in relation to their discrimination between ratios. This indicated that the children we included had a broad range of skills and suggested that our recruitment practices of including children from both special and mainstream education had been effective.

There was a trade‐off in the development process. The study had enabled us to achieve breadth with a wide range of children participating in the research process, allowing us to test issues of access across children with a variety of additional needs across three iterations of fine tuning the game; their levels of engagement indicated its potential for cognitive learning, but we did not formally evaluate whether children acquired new skills over time. We now need to examine whether their engagement survives the test of time when aspects of the game were no longer novel, whether it results in stable new learning. This forms the next step, drawing on the data here to identify children with different profiles of learning using a fixed rather than flexible research design with pretest and posttest measures and investigating how these skills were then generalized to different settings.

Working collaboratively across a team engaged in a range of activities produced some tensions and dilemmas. There were many benefits to bringing together the craft of game design with the researcher's experience of early mathematical skills, but this also brought competing agendas for creativity and exploration within the design team versus the research evidence base of the importance of structure and goal‐directed small steps. Unforeseen alterations to ratios arose during the design process that had consequences for the research, and some aspects rose more intuitively from the data with limited opportunity for testing out. For example, more specific data could be collected on children's responses to failure thereby informing the programming of the DDA.