## SEARCH BY CITATION

### Keywords:

• Teaching;
• Assessment;
• Variation;
• Distributions

### Summary

Several tasks used in research studies are presented with assessment rubrics and examples of the development of student understanding. The tasks focus on students’ appreciation of variation in several contexts and illustrate the need to discuss variation in the classroom and to ask students specifically about it during assessment.

### INTRODUCTION

Following the formal introduction of statistics and probability (or data and chance) into the mathematics curriculum in the early 1990s (e.g. Department of Education 1995; National Curriculum Council 1991), research into students’ understanding generally followed the ‘investigation steps’ model of the curriculum based on early work of Peter Holmes (1980). Early research of David Green (1983) in probability was followed by research that focused on school students’ understanding of average, sampling, data representation and inference. Although measures of both centre and spread were mentioned in the curriculum documents, most of the research at school level concerned centre and various measures of average rather than spread and measures such as the standard deviation. Speculation on the reason for this relates to the formula for the standard deviation being more complicated than that for the arithmetic mean, causing the topic to be postponed to senior secondary years. The disadvantage of this progression, be it true, is that postponing the standard deviation is also likely to postpone consideration of the fundamental concept of variation underlying the standard deviation. Although all statisticians know that the arithmetic mean is calculated to represent the centre of a data set with varying values, Mokros and Russell (1995) found that most students in their study had not gleaned this global appreciation from their educational experiences. Following the introduction of the data handling curriculum, Green (1993) and others began calling for research into the underlying concept of variation, which David Moore (1990) claimed to be the very heart of statistical investigations.

The consideration of variation as an isolated topic is a very difficult, if not impossible, undertaking. Variation takes place in a context, usually somehow associated with data collection in a statistical investigation, and in most cases is juxtaposed with an expectation. Statistically expectation can be expressed in terms of ‘the expected value’ of a distribution, which also has a ‘variance’, the square of the standard deviation. School students, however, do not have these concepts at hand and are building their intuitions on experiences with data in the classroom (e.g. tossing dice) or from the everyday world (e.g. observing cricket stars’ batting performances). Although curriculum documents introduce and stress early ideas associated with expectation, for example simple probability and the mean, before ideas associated with variation, for example range and standard deviation, Watson's (2005) analysis of interviews with students from ages 6 years to 14 years suggested that they develop an appreciation of variation in the world around them before they develop an appreciation of expectation. Expectation is then likely to be reinforced by the school curriculum in the middle school years with variation becoming more formal in the high school years.

One of the recommendations arising from much of the research into variation taking place today is that, even without formal measures, there needs to be an emphasis on variation from the very beginning of children's experiences with chance and data. Along with this, assessment tasks need to allow students the opportunity to display a range of levels of developing understanding. One of the difficulties with many traditional assessment tasks is that they ask for a numerical answer, which is then judged as right or wrong, often without very much indication of how the answer was obtained, or how ‘far away’ from the correct answer an inappropriate response might be. The tasks presented here include rubrics that indicate levels of performance indicating progress with respect to the structural complexity of the response and its statistical appropriateness. Having this information can inform the teacher as to the next steps in assisting students to move to an appropriate understanding. Such understanding in the case of variation is rarely associated with a single number or answer, but with a description, a range of numbers, or a sketch of a graph to tell a story.

Table 1.  Levels of response to the spinner task for six trials of 50 spins (adapted from Watson and Kelly 2004); level of variation determined following procedure adopted by Shaughnessy et al. (1999)
LevelDescriptionExample
3Appropriate variation30, 20, 32, 26, 24, 18
(SD = 1.3 to 5.0)22, 24, 21, 28, 20, 25
2Too small, too wide variation25, 26, 25, 24, 25, 25
(SD < 1.3 or > 5.0) OR5, 30, 26, 18, 45, 50
No variation (strict proportion)25, 25, 25, 25, 25, 25
1Lop-sided: all < 21 or > 301, 3, 7, 8, 9, 11
43, 45, 48, 44, 47, 46
0Inappropriate response25, 50, 75, 25, 50, 75
S, W, S, W, S, W

At Level 0, responses are inappropriate and display a general lack of understanding of what the task is asking. Common misinterpretations reflect a belief that the question is asking for the likely pattern of landings between shaded and white, or give values outside of the range of 50 for the six trials. At Level 1, responses display an inappropriate centre and therefore display a lack of appreciation for proportion. For 50 spins, all six suggestions are less than 25 or greater than 25. The degree of observed variation is not relevant at this level. Level 2 responses reflect a strict probability-based view of what the outcomes should be or show more than a reasonable degree of variation. At Level 3, responses display appropriate variation within the boundaries suggested for 50 spins.

An extension of this task is suggested for classrooms, one that also asks students to explain their predictions. Use of spinners with different proportions (e.g. 3/4 shaded, 1/4 white) may further challenge older students in their thinking about expectation and variation.

The task in figure 2, used by Watson and Kelly (2003), is preceded by another asking about the most likely outcome of a single toss of a six-sided die. Three alternatives are offered (1, 6 and equally likely) and an explanation is requested. This item assists in setting the context and appreciating students’ understanding of the operation of a fair die. Students are then asked to speculate on the outcomes for 60 tosses and the table is provided in order to scaffold the response. The total of 60 is included to assist younger students and the rubric recognizes when students are unable to account for this information. The rubric for the 60 tosses task is presented in table 2. The explanation requested as well as the numbers supplied are taken into account in the rubric; the rubric is hence more complex than that suggested for Task 1.

Table 2.  Levels of response to the 60 tosses task (adapted from Watson and Kelly 2003); level of variation determined following procedure adopted by Shaughnessy et al. (1999)
LevelDescriptionExample
4Appropriate variation and explanation‘8, 9, 11, 13, 10, 9 – because they all range around the same number because they all have an equal chance.’
3Conflict of probability and variation OR‘10, 10, 10, 10, 10, 10 – in theory all numbers should come up equally. They probably will not.’
Appropriate variation with inadequate explanation, OR‘9, 12, 10, 7, 6, 16 – I used these numbers based on what usually happens to me.’
Explanation but too much variation‘14, 6, 17, 6, 3, 14 – because there's a chance of rolling these digits.’
2Strict probabilistic prediction and explanation OR‘10, 10, 10, 10, 10, 10 – they all have the same chance of coming up.’
Explanation but too little variation‘10, 10, 9, 11, 10, 10 – these numbers are reasonable because there is a chance in six.’
1Results add to 60 without appropriate variation and explanation OR‘10, 20, 10, 5, 5, 10 – because it adds to 60.’
Do not add to 60 but describe some aspect of variation‘19, 18, 5, 7, 23, 10 – because any number can come up.’
0Do not add to 60 or have unrealistic values‘6, 3, 2, 1, 4, 5 – because the one might have a bigger chance of coming up more than the other numbers.’
‘2, 8, 45, 0, 3, 2 – because they sometimes come up when I roll a die.’

At Level 0, responses do not take into account the requirement to account for 60 tosses or suggest totally unrealistic outcomes. Some explanations reflect beliefs related to the previous question about a single die toss. At Level 1, responses display a single appropriate aspect of the task. On one hand, they may appreciate the need for numbers to sum to 60 but do not show appropriate variation or explain it. On the other hand they may not choose numbers adding to 60 or showing reasonable variation but they may suggest that some form of variation can occur in the explanation. Level 2 responses reflect a strict probability-based view of what the outcomes should be or show less than a reasonable degree of variation. At Level 3, there are three possibilities in dealing with the dilemma of variation and expectation. Responses may deal directly with the dilemma by stating a uniform outcome but saying it is not likely to occur. Responses may suggest reasonable values for the outcomes but offer no explanation as to why there is variation in them. Finally, responses may provide numbers with too much or too little variation but an explanation with appreciation of why it occurs. At Level 4, appropriate variation is displayed in both the suggested outcomes and in the explanation presented.

In the study where this task was used, the percentage of Level 0 responses decreased with increasing grade, the percentage of Level 1 and 3 responses remained relatively constant, but the percentage of Level 2 responses increased dramatically from 4% in grade 3 to 39% in grade 9. As was also found for Task 1, this outcome suggests that the teaching of probability from a theoretical perspective at the middle school is not balanced by a discussion of the potential variation to occur if trials are actually carried out.

Green's (1983) suggestion of challenging students to choose among three bar graphs to select the most likely outcome for 60 tosses could be used as a follow-up classroom activity for this task. Green's graphs included one uniform distribution, one peaked in the middle and one with appropriate ‘random’ variation.

### TASK 3: PROBABILITY SAMPLING LOLLIES

The task presented in figure 3 is a shortened version of an interview protocol used with students aged 6 to 15 by Kelly and Watson (2002). Although again students are asked to predict outcomes, this time students predict outcomes of individual sampling draws of 10 lollies from a container with 50 red, 20 yellow and 30 green lollies in it. The question of interest is how many red lollies are in each draw. Students then perform the collection of 10 lollies six times and are asked if they wish to change their initial predictions. Finally students are asked to draw a graph of the number of red lollies in 40 collections of 10 lollies. The last question is aimed at moving students away from suggesting a few estimates to a distributional view of outcomes.

The rubric for Q2 (a), the six suggested outcomes for drawing 10 lollies, is given in table 3, adapted from that used by Kelly and Watson (2002) based on the longer protocol. The rubric for Q2 (a), like the 60 tosses item, is based on the description of expectation and variation for the six possible outcomes as well as the justification given.

Table 3.  Levels of response to Q2 (a) of the lollies task; categorization of numerical responses determined by Shaughnessy et al. (1999)
LevelDescriptionExample
4Developed understanding of reasonable variation with an explicit acknowledgement of appropriate proportion and variation‘5, 4, 6, 3, 4, 6 – 4, 5, 6 are around the halfway mark. You might get 3 less than what you expect.’ (Five as centre, reasonable spread)
‘5, 4, 6, 5, 7, 6 – most around 5 because there's 50 reds in there.’ (Five as centre, reasonable spread)
3An understanding of reasonable variation with an implicit acknowledgement of appropriate proportion (more red)‘4, 6, 3, 5, 2, 7 – more reds than any other colour, because there is more reds than yellow or green.’ (Five as centre, reasonable spread)
‘5, 4, 6, 3, 4, 4 – around an average, hard to explain. Sometimes you get different things, just chance.’ (Five as centre, reasonable spread)
2Demonstration of reasonable variation about the centre without appropriate explanation OR‘4, 6, 5, 8, 2, 3 – all different numbers and it varies, lots of reds.’ (Five as centre, reasonable spread)
An implicit acknowledgement of proportion with an inappropriate centre‘6, 5, 4, 8, 7, 6 – lots of reds in there but not a lot will always come out.’ (High centre, reasonable spread)
1Transitional understanding of variation and probability with a variable spread (narrow or wide) and inconsistent centres (low or high) in suggested outcomes OR‘3, 9, 10, 3, 6, 8 – can't always be sure of what you would get. You could get something twice or a lower or higher number.’ (High centre, wide spread)
‘2, 3, 1, 4, 3, 4 – it could go anyway.’ (Low centre, reasonable spread)
Strictly proportional outcomes with no variation‘5, 5, 5, 5, 5, 5 – half the number is red, 50/50 chance.’ (Five as centre, strict probability)
0Idiosyncratic explanations with variable spread (narrow or wide) and inconsistent centres (low or high) in suggested outcomes‘2, 2, 2, 1, 2, 1 – some might get two and then some might get one’ (Low centre, narrow spread)
‘4, 5, 1, 3, 6, 8 – numbers that would fit in my hand.’ (Five as centre, wide spread)

At Level 0, responses usually do not take into account that ‘5’ should be the centre of the six suggested outcomes and have either a high or low centre with a wide or narrow spread. Whenever ‘5’ is the average of the six outcomes, then the spread is wide, demonstrating too much variation. Explanations are idiosyncratic. At Level 1, responses still do not often take into account that ‘5’ should be the centre of the six suggested outcomes, again with a low or high centre and a variable spread. Generally responses have an intuitive explanation focused on the idea of unpredictability and anything can happen but are inconsistent in choice of centre and variation. Also classified as Level 1 are responses that are strictly proportional in suggesting six outcomes (i.e. display no variation). These responses reflect appropriate expectation but no appreciation of variation.

There are two types of Level 2 responses. The first type is those that reflect an intuitive understanding of reasonable variation by giving values for the six suggested outcomes that are centred around ‘5’ with an appropriate amount of variation. Explanations, however, are either idiosyncratic or based on unpredictability and ideas that anything can happen. The second type of response gives a reasonable spread, but provides either a low or high centre for the six outcomes. Explanations are more sophisticated than the first type of response despite the low or high centre, implicitly acknowledging the proportions of colours in the container by noting ‘more’ red lollies. At Level 3, all responses to the six suggested outcomes have ‘5’ as the centre and a reasonable spread. The explanations, however, focus implicitly on only one of two separate ideas related to proportion and variation: ‘more’ reds than any other colour or outcomes ‘centred’ without an explicit recognition of what the centre is. At Level 4, reasonable variation with an explicit acknowledgement of appropriate proportion is displayed in both the six suggested outcomes and the explanations.

In the study where this task was used, overall performance increased with grade but peaked at grade 7. This may reflect the sample of students or again teaching of probability in the middle school years. The use of this task as an introductory individual activity could be followed by the drawing of lollies and recording of outcomes as a class exercise. Students could then assess their own initial responses in the light of whole class trials.

Question 5 of the lollies task in figure 3 is a graphing exercise. Part (a) asks students to show what the number of reds would look like for the 40 trials and a blank piece of paper is provided. Part (b) is essentially the same as part (a); however, a set of axes is supplied, with ‘Number of Lollies’ and numbers 1 to 10 on the horizontal axis and ‘Number of Students’ with multiples of 5 to 40 on the vertical axis. Students who are successful on part (a) are quite often not presented with part (b). The rubric for Q5 is based on the statistical appropriateness of the graph, including an appropriate distribution around the centre (‘5’). Students’ reliance on the scaffolding in part (b) is considered when determining levels. The rubric is presented in table 4.

Table 4.  Levels of response to Q5 of the lollies task (from Watson et al. 2007)
LevelDescription
4An appropriate distribution with a peak around ‘5’. Reference to the centre, acknowledgment of variation, and discussion of the shape of the distribution.
3Without axes, logical time series graphs that focus on the centre. With axes, data focused around ‘5’. Some reference to the centre and variation in discussion.
2Mixed performance including:
• Without axes, logical time series graphs that do not focus on proportion but do appreciate variation. With axes, no engagement.
• Without axes, no meaningful representations. With axes, data that are focused on the centre.
• Without axes, meaningful representation in the form of a list of reasonable numbers, a table or primitive graph. With axes, data that are focused on the centre.
1Various inappropriate representations including:
• Primitive graphs, tables or lists of possible numbers, but with no focus on the centre.
• Graphs without consideration for the context of the question or for proportion.
0Single numbers or pictures.

At Level 0, students write single numbers or draw pictures of lollies. They are not able to comprehend the task of graphing, even when presented axes in part (b). Figure 4 shows two examples of Level 0 responses.

At Level 1, responses convey a variety of inappropriate representations including primitive graphs and tables, or lists of numbers. Some students can comprehend the task of graphing, but do so without any consideration of proportion or context. Two examples of Level 1 responses are shown in figure 5.

Level 2 responses reflect a mixed performance on the graphing task. Without the presence of axes (part (a)) some students can produce logical time series graphs that do not focus on proportion, but appreciate variation. Other students cannot produce any meaningful representation without the axes and at most can generate a list of reasonable numbers or produce a table or primitive graph to show the data. With axes, however, these same students can plot data that are focused on the centre. Level 2 performances are at a transitional stage in relation to using the scaffolding presented in Part (b). Figure 6 shows two Level 2 responses, with part (a) on the left and part (b) on the right.

At Level 3, responses to Q5 (a) and (b) are similar in that students can produce a logical time series graph with a focus on the centre without being provided with axes, and when provided with axes are able to construct a graph with a focus on the centre, with some reference to variation around ‘5’ in the discussion. Figure 7 displays two examples of Level 3 graphs, one without the axes provided and one with the axes.

At Level 4, students produce graphs with an appropriate distribution with a peak around the centre (‘5’). This is done with or without the axes being presented. Students make reference to the centre, acknowledge variation, and freely discuss the shape of the distribution. Two Level 4 responses are shown in figure 8.

In the study where this graphing task was used, the higher level results were more closely associated with higher grade levels, most likely related to students in the middle school having learning experiences related to graphing. Only preparatory students and grade 3 students responded at Level 0 and Level 4 responses only occurred in grades 7 and 9. Part (a), asking for any representation, engaged students and many younger students displayed that they understood the context although were not able to draw graphs (cf. figure 4).

### SUMMARY

These three tasks have illustrated the progression from only asking students for numerical variation, to asking them for numbers plus an explanation, to asking them also to draw a graph. It is clear that more understanding of student thinking about variation is gained with each task. As well, there is an appreciation of the relationship of variation to expectation in each context.

### References

• Department of Education (England and Wales) (1995). Mathematics in the National Curriculum. London: Department of Education.
• (1993). Data analysis: what research do we need? In: (ed.) Introducing Data Analysis in the Schools: Who Should Teach It? pp. 219239. Voorburg, The Netherlands: International Statistical Institute.
• (1983). A survey of probability concepts in 3000 pupils aged 11–16 years. In: , , and (eds) Proceedings of the First International Conference on Teaching Statistics: Vol. 2, pp. 766783. Sheffield, UK: Teaching Statistics Trust.
• (1980). Teaching Statistics 11–16. Slough, UK: Schools Council and Foulsham Educational.
• and (2002). Variation in a chance sampling setting: The lollies task. In: , , and (eds) Mathematics Education in the South Pacific (Proceedings of the 26th Annual Conference of the Mathematics Education Research Group of Australasia, Auckland, Vol. 2, pp. 366373). Sydney, NSW: MERGA.
• and (1995). Children's concepts of average and representativeness. Journal for Research in Mathematics Education, 26, 2039.
• (1990). Uncertainty. In: (ed.) On the Shoulders of Giants: New Approaches to Numeracy, pp. 95137. Washington, DC: National Academy Press.
• National Curriculum Council (1991). National Curriculum Council Report: Mathematics. York, UK: National Curriculum Council.
• , , and (1999, April). School mathematics students’ acknowledgment of statistical variation. In: (Chair), There's More to Life Than Centers. Presession Research Symposium conducted at the 77th Annual National Council of Teachers of Mathematics Conference, San Francisco, CA.
• (2005). Variation and expectation as foundations for the chance and data curriculum. In: , , , , , and (eds) Building Connections: Theory, Research and Practice (Proceedings of the 28th Annual Conference of the Mathematics Education Research Group of Australasia, Melbourne, pp. 3542). Sydney, NSW: MERGA.
• and (2003). Predicting dice outcomes: The dilemma of expectation versus variation. In: , , and (eds) Mathematics Education Research: Innovation, Networking, Opportunity (Proceedings of the 26th Annual Conference of the Mathematics Education Research Group of Australasia, Geelong, pp. 728735). Sydney, NSW: MERGA.
• , and (2007). Students’ appreciation of expectation and variation as a foundation for statistical understanding. Mathematical Thinking and Learning, 9, 83130.
• and (2004). Statistical variation in a chance setting: A two-year study. Educational Studies in Mathematics, 57, 121144.