Discussions about formative assessment are fundamentally about teachers' awareness, in order that they can make adjustments that respond to students' reasoning and participation. To this point, we have raised concerns over how the literature has focused on strategies for assessment but taken disciplinary substance for granted. We have shown that prominent examples in the literature have tacitly treated subject matter as information, and the assessment of student thinking as a check on its correctness against that information.
Effective assessment in science education, we argue, should involve genuine, extended attention to the substance of student reasoning, on at least two levels. Teachers should elicit and pay “persistent attention” (Strike and Posner, 1992) to students' arguments. What reasons do students have for answering as they do? What evidence and logic are they using? In this, the teachers are not only becoming aware of student reasoning but modeling for students how they should focus their attention in science. In other words, they are assessing student reasoning in ways that are consistent with how students should learn to assess ideas as participants in science.
At another level, formative assessment should involve awareness of how students are engaging in disciplinary practices. Are students reasoning about the natural world, or are they focused on what they are “supposed to say,” playing the “classroom game” (Lemke, 1990) of telling the teacher what they think she wants to hear? We see these multiple and interacting goals highlighted in other areas of literature.
Practices of Formative Assessment Elsewhere in the Literature
We are arguing that formative assessment should be understood as a matter of attention to disciplinary substance, and in this sense it should be inherent throughout classroom activity, not restricted to specifically designate “assessment activities.” While this is often in the rhetoric of formative assessment, we have argued, it is often not achieved. There are, however, models of formative assessment in work that identifies itself in other ways.
A prime example is Ball's (1993) account of “dilemmas” teachers face in honoring children's reasoning while at the same time guiding them toward disciplinary understanding and practices. When the student Sean in her third grade class claimed that the number 6 could be both even and odd, Ball's response was to elicit his argument, pay it attention, and guide other students to pay it attention as well. This lead to the discovery and definition of a class of numbers they called “Sean numbers” that had the interesting property that they “have an odd number of groups of two,” which allowed further investigation into patterns and properties.
Ball never described what she or her students were doing in class as assessment, but that is precisely what it was: They were assessing Sean's idea, its validity, merits, weaknesses, and possibilities; they were working with the idea as nascent mathematicians. Ball was assessing at another level, as well, in her attention to the students' participation in mathematical inquiry, to the community they were forming and the understandings they were developing of what it means to engage with mathematical ideas.
There are many other accounts in the literature that, like Ball's, depict close attention and responsiveness to student thinking (Franke and Kazemi, 2004; Lampert, 2001; Warren, Ballenger, Ogonowski, Rosebery, & Hudicourt-Barnes, 2001; Warren and Rosebery, 1995). Hammer (1997) described the “tensions” among multiple objectives and multiple dimensions of awareness of student reasoning and participation in his high school physics class. In one episode, for example, a group of students devised their own experiment and collected their own evidence to arrive at the conclusion that Styrofoam™ conducts electricity. It was a group that included students who had generally been reluctant to pursue their own ideas, and Hammer wanted to support their initiative and nascent scientific stance, but this was in tension with their learning which materials do and do not conduct electricity. (Styrofoam does not.) Hammer described his own and others' accounts of their practices as “discovery teaching,” in which “successful instruction depends on teachers' often unanticipated perceptions and insights” into student thinking and participation.
In a study of middle school math teachers responsiveness to student ideas, Pierson (2008) defined “responsiveness” as “the extent to which teachers ‘take up’ students' thinking and focus on student ideas in their moment-to-moment interactions,” and in particular “High II” responsiveness in which the focus is on the students' meaning and logic for the immediate purpose of understanding their reasoning on its own terms. She distinguished High II from High I responsiveness, in which the teacher worked to identify student ideas with the purpose of correcting them. In Pierson's language, our critique of the formative assessment literature is that there is evidence of “High I” but not of “High II” responsiveness.
With data from 13 teachers, Pierson found a strong, significant correlation between High II responsiveness and student learning, but not with High I. Working to explain this finding, she discussed how “discussions high in responsiveness can act as formative assessments, which research indicates is positively related to student learning,” citing work from the formative assessment literature. She also cited work by Saxe, Gearhart, and Seltzer (1999) who found that “integrated assessment,” which they defined as “the extent of opportunity for students to reveal their understandings, to receive interpretations of their contributions, and to provide interpretations of others' contributions,” (p. 11) correlated with students' achievement in problem solving.
Not all of these accounts speak of formative assessment, but that is what they depict: teachers continuously attending to the students' understanding, reasoning, and participation. They do not focus on particular, discrete strategies; rather, they show teachers adopting stances of “respecting students as thinkers,” as Ball put it. They learn about what and how students are thinking and participating, and they use that information to guide their instruction.
While we expect readers are familiar with some of this work, at least Ball's (1993), it will be helpful to have an example of what we mean by this continuous attention to students' reasoning and participation. We have chosen one from the classroom of a teacher who, like the teachers in the examples above, was working to reform his practices.
An Example of Substantive Formative Assessment
The following example comes from a corpus of data collected during a 3-year study7 that examined what high school science teachers attended to in their classroom interactions with students and how that informed their instructional decision making and curricular modifications. Terry Grant, a co-author of this article, was one of 28 collaborating teachers. The participating high school science teachers worked in subject-matter cohorts of 8–10 others and university-based researchers. For 3 years, they met bi-weekly for 2 hours to share and discuss video and student work from their classroom teaching. In addition, all of the project teachers met together each summer for a 1–2 weeks workshop. The focus of the regular meetings was discussion of student ideas and reasoning, along with consideration of possible next moves. Teachers, often in collaboration with university-based researchers, wrote case studies about their student learning and their instructional decision-making, grounded in their video data, student work, and their personal notes.
The excerpt below comes from Terry's case study of a class he taught in the 2nd year in the project, and his 10th year as a classroom teacher.8 It was also his first year at a new school, and he decided to use that opportunity to remake his practices, specifically to become more aware and responsive to his students' thinking.
Terry's 9th grade biology class was scheduled to begin a unit on the chemistry of life, which assumed some basic understandings of matter, atoms and molecules. He had intended a 15-minute review of these concepts, starting with the textbook definition they had read: “Matter is anything that occupies space and has mass.” He began by asking students to say what it means to take up space and have mass. They did this with some hesitancy; as Terry pressed they gingerly offered that you “can see it” and “can touch it,” that it “weighs something.”
Terry thought to remind students about the difference between mass and weight, which they had studied.9
Teacher: What influences your weight? Do you weigh more on the earth or on the moon?
Teacher: Ohhh (quietly). And. So what's the difference between your weight and your mass. Standing right here. [silence for several seconds] Nothing.
Terry decided to set the topic aside, saying, “I don't think it's going to be that significant” for the topic at hand, and for the rest of the conversation, he let them say “weight.” Terry asked about a table, which the students all thought is matter, and then water. Barb answered, “No.”
Barb: I think it's composed of molecules.
Teacher: OK, which are?
Barb: Matter? [barely audible]
Teacher: Are they? Perhaps it doesn't matter, but you kind of went, “Matter?” [mimicing the tentative tone]
Barb: Yes. [clearly stated]
Terry guided them again to apply the definition of matter to water, and students quickly agreed: It has weight, and it takes up space, such as in oceans. One student offered, “doesn't matter have to do with a state? Like liquid and solid and gasses?” Since water is one of those, it must be matter. Terry then asked whether air is matter, and many voices in unison said “No.” Barb and Brianna said “You can't weight it.” and “You can't see it.” Several students were speaking at once, and then Barb had the floor.
Barb: It takes up space but you can't feel it, Like you can't bump into it… Cause air is everywhere, except for in water… well actually no there is air in water.
Terry: Adria. Shh. Adria, did you have your hand up?
Adria: I was gonna say you could feel it. Or you can't feel it.
Barb: Yeah, well you can feel wind. [Overlapping talk.]
Terry: Yeah, What's wind?
Brianna: Air… air blowing [Overlapping talk.]
Terry: So can we weigh it?
Students: No… no
Terry: Those are the issues we've got to resolve. Can we weigh it?
Maggie: No [Multiple students. Emphatic.]
Terry: How could I weigh it? What could you do to weigh it?
Barb: You could like put it in a balloon or something but there's the weight of the balloon so you couldn't weigh it.
Terry: I haven't got a scale with me today. (Walks over to his desk and pulls out a bag of balloons)… So I have balloons, right? (Tosses Barb a balloon.) Blow it up.
By this point, the class was past the 15 minutes Terry had planned. We pause here to consider what is happening in this snippet.
First, it is immediately obvious that the students showed hesitancy and confusion over the concept of matter: They had trouble articulating what it meant; they were not sure at first whether water is matter and they remain unsure about air.
Less obvious, but as important, is how the students approached the topic. Barb, the most outspoken student in the room, seemed to be trying to find the right terminology (“gravity,” “molecules”), and her questioning “Matter?” suggested that she was “playing the classroom game” (Lemke, 1990). Another student reasoned in terms of material they had covered. Her “doesn't matter have to do with a state” sounds like a comment based on the structure of a curriculum rather than on the substance of ideas.
Terry noticed and responded at both levels. For one, much like the examples we presented earlier from the assessment literature, he discovered that material he thought would need only a quick review was deeply problematic for the students, first the definition of matter and, within that, the difference between mass and weight. He posed questions, much as in other examples, to find out what they did and did not understand.
At the same time, he noticed and considered how they framed what was taking place. Terry chose to set the question of mass versus weight aside, rather than give an explanation on the distinction (which they had “covered”), and he allowed them to continue to use the (formally incorrect) term weight rather than mass, because he wanted to encourage their genuine engagement with the ideas. He remarked on Barb's uncertainty, when she asked “matter?” And he focused on eliciting their reasoning. He confirmed that solid objects are matter, and that water is matter, but not until the students seemed to reach their own consensus on these points.
With the class divided on whether air is matter, he made them responsible for the “issues we need to resolve,” taking his lead from them, with one exception: When he asked “can we weigh it?,” the students said no, but he proceeded anyway, asking “how can we weigh it?” If that was a lapse, possibly he was trying to press for a resolution, to allow them to move on with the plan for the day. Still, his question did not convey his view on whether air is matter.
Terry had the class continue this new inquiry on a new, unplanned topic. Like Ball hearing Shea's idea that six is an odd number, Terry changed his goals for the lesson in response to what he heard in their thinking. We present one more snippet, to highlight more of the formative assessment inherent in Terry's close attention to the students' reasoning.
As Barb inflated the balloon, Mikela commented “that one's stretching because she's blowing air into it,” and Terry asked if that meant there was a different “amount of stuff in it.” Students spoke over each other, some saying “yes” and some “no”; among the remarks was Lauren's: “Air and matter is closed up.” Barb said that air was “occupying space in there,” referring to the balloon, and Terry asked if air is “occupying space in the room.” Again, students spoke over each other, giving a mix of answers.
Terry: Are you saying? And I'm asking, I'm not telling. That it takes up space when its' in HERE [in the balloon], but it doesn't take up space when it's in the room.
Terry: Is that the general consensus?
Barb: No! Actually that's right cause you can't put something inside that balloon with air it it.
[Several students speaking at once]
Terry: OK. What would happen to the air in the balloon, if I put water in it too?
Barb: There wouldn't be as much air.
India: The water's taking up space.
Terry: Okay. What would happen to the air in the balloon, if I put water in it too?
Barb: There wouldn't be as MUCH air.
India: The water's taking up space.
Terry: OK, so…
Laura: The air is the space.
Terry: Say it again.
Laura: The air IS the space.
Terry: So air IS the space. Are you saying it takes up space? Is that the idea?
Ari: The air is the space that gets taken up.
Terry: So it's an empty space until I put water in it? I'm trying, I'm trying to work your way… I'm not trying to say you're right or wrong, I'm asking. This is not a graded assignment or anything.
Terry: Yes? How many people agree with that? Air is empty space that the water is going to take up when I pour water in. If I were more daring I would've brought a couple of water balloons too. I'm afraid they'll blow up in here… So think about this, some of you have this look on your face like “I don't know for sure,” is this just empty space which we filled up with water, or is there something in there?
Brianna and Laura (simultaneously): There's something in there.
Terry: Okay, what's the something?
Students: Air! Air!
Terry: So, does it take up space?
Laura: I'm confused!
India: Oh my god!
Terry: You don't sound convinced, you're giving me “ummmm.” Yea, go ahead.
India: But when, when something else goes in there, doesn't some of the air leave?
Later in the period the class tried to compare the weights of an inflated and empty balloon, found no difference, and discussed what that meant, including the possibility that the scale was not sufficiently sensitive. The next day, they tried again, with a more sensitive scale, which showed the inflated balloon a bit heavier. When a student raised the concern that blowing up a balloon would contaminate the air with saliva, another student suggested using an air pump from the gym, which Terry let her fetch. In the end, the class concluded that air is matter.
Of course there are a number of issues we could consider about this class: Is it practical to take so much time on a “review” topic? Was the conversation too dominated by a few individuals? What might Terry have done differently to address the students' needs? These are important issues, but our focus here is on Terry's attention to the disciplinary substance of his students' thinking as an example of formative assessment.
Again, we begin by considering the students' thinking. For many, seeing the balloon inflate showed air taking up space. Barb and India argued that, if there were water in the balloon, then there would have to be less air, because the water would be taking up some of the space. For others, though, the idea that air takes up space seemed to be in tension with their intuition that air does not take up space in the room. Lauren and Ari gave a clear articulation to a very different way of thinking, that air and space are the same thing.
Along with the evidence of their conceptual understanding, there was evidence of how they were approaching the topic. Rather than looking to Terry or the textbook, in this snippet, and rather than quoting terminology, the students were arguing on their own terms based on their own reasoning and observations. The conversation was lively and robust, with more students entering the fray.
Terry was attending—and responding—at both levels, trying both to understand their reasoning and to assure them that was what he wanted to be doing. He emphasized that he was “asking… not telling,” in an energetic interrogation of their thinking. When Lauren spoke up with her very different view, it was not clear to Terry what she meant, and he worked to understand, telling her “I'm trying to work your way” and, “I'm not trying to say you're right or wrong.” Ari's clarification seemed to help, and Terry threw the idea back to the class, to find out how many others thought similarly, remarking on students' uncertain faces. Laura had changed her mind, apparently, but still seemed torn, and despite the overwhelming “yes” from students that air is “something” and that it takes up space, he tried to keep the question open, making room for students to continue to question.
From Strategies to Attention
As an example of formative assessment, this account of Terry's class is similar in several respects to the examples we quoted from the literature: He posed questions to students, listened to their answers, and what they had to say informed how he moved forward. That is, if we use the literature for guidance, we could support the claim that this is formative assessment simply by considering what Terry was doing.
However, we argue, it is not sufficient to consider only the teacher's actions. The core of formative assessment lies not in what teachers do but in what they see. The point is teachers' awareness and understanding of the students' understandings and progress; that's what the strategies are for. To appreciate the quality of a teacher's awareness, it is essential to consider disciplinary substance: What is happening in the class, and of that, what does the teacher notice and consider?
In our critique of the examples from the literature, we argued that they (1) neglected the disciplinary substance of student thinking, (2) presumed traditional targets of science as a body of information, selected in advance, (3) treated assessment as strategies and techniques for teachers. In our presentation and analysis of Terry's class, we worked to do something different in each of these respects.
First, as we have discussed, we began with student thinking in our analysis. Second, like Terry, we considered student thinking not only with respect to its alignment with the canonical ideas but also with respect to the nature of the students' participation. Students' acceptance that air is matter (or that plants feed by photosynthesis or that sunspots are magnetic phenomena, etc.) could be seen as alignment with the canonical ideas. However, if students accept those ideas on the teacher's authority, rather than because they see them supported over other ideas by evidence and reasoning, then they are at odds with the practices of science. For this reason, it is essential that formative assessment—and accounts of it in the literature—consider more in student thinking than the “gap” (Black & Wiliam, 1998a; Sadler, 1989) between student thinking and the correct concepts.
Moreover, it was Terry's attention to the disciplinary substance of student thinking that led him to abandon his original plan for the lesson. Formative assessment created objectives for him that he did not have at the outset, and again at two levels. One objective was conceptual, that students understand the concept of matter. Another was at the level of how students approach the topic, and there we could see Terry working to move students into engaging the material as nascent scientists, and away from the “classroom game” (Lemke, 1990) of telling him what they think he wants to hear.
To the third point, it is not possible to distinguish any particular strategies from the activity as a whole. Conceptualizing assessment as attention, Terry was formatively assessing student thinking by closely attending to it. He wanted to understand what they were thinking and why, as would any participant in any meaningful discussion. Formative assessment should be understood and presented as nothing other than genuine engagement with ideas, which includes being responsive to them and using them to inform next moves.
Terry's formative assessment was continuous with what he hoped students to learn: practices of assessing the quality of ideas for their fit with experience and reasoning. Effective assessment is part of the substance students should learn. It is important that they understand whether air is matter; it is also important that they understand what goes into deciding whether air is matter, and that, fundamentally, is the assessment of an idea. Thus the students were learning to assess ideas as nascent scientists, rather than as compliant students. Understanding these discipline-based assessment criteria is part of what educators should help students learn. As students learn to engage in disciplinary assessment, they are learning a fundamental aspect of science (Coffey, 2003a; Coffey & Hammer, in preparation; Duschl, 2008).
We have argued that, in focusing attention to strategies and techniques, the literature on formative assessment has generally presumed traditional notions of disciplinary content as a body of information. These notions have shaped the filters and criteria for what counts as disciplinary substance in student thinking, and so we see teachers and researchers attending to how students' thinking aligns with the target information, with an emphasis on terminology, more than to the meanings students are trying to convey or to the rationality of their reasoning. In this, we have argued, the literature is at odds with research on learning, and it is at odds with disciplinary practices. If assessment criteria are incongruous with what happens in the discipline, educators can misconstrue what counts within disciplinary activities and distort for students what engaging in science activities entails.
There are, clearly, a variety of reasons for this state of affairs, including the ways summative assessments are constructed and valued in the much larger educational system. It is only natural that practices of formative assessment will tune to support desired outcomes. Much, certainly, has been discussed in the literature over the influence of high-stakes standardized testing (e.g., Valli, Croninger, Chambliss, Graeber, & Buese, 2008).
In closing this article, we consider another possible reason for the literature's emphasis: Researchers and teacher educators seem to believe that strategies are what teachers need first or most, to help them engage in formative assessment. At least two of the research projects cited above (Black et al., 2003; Shavelson, 2008) explicitly organized their professional development efforts around the premise that enacting well-defined assessment strategies will elicit and facilitate teachers' awareness of student understanding, awareness that is difficult to achieve.
Discussing on-the-fly assessment that “occurs when ‘teachable moments’ unexpectedly arise in the classroom” (p. 4), Shavelson (2006) wrote:
Such formative assessment and pedagogical action (“feedback”) is difficult to teach. Identification of these moments is initially intuitive and then later based on cumulative wisdom of practice. In addition, even if a teacher is able to identify the moment, she may not have the necessary pedagogical techniques or content knowledge to sufficiently challenge and respond to the students. (p. 4)
There is support for this position in research on teacher “noticing” (Sherin & Han, 2004), in findings that most pre-service and inservice teachers have difficulties noticing student thinking (Jacobs, Franke, Carpenter, Levi, & Battey, 2007; Sherin & Han, 2004). Other research on formative assessment has argued that, although teachers can often make reasonable inferences about student understanding, they face difficulties in making “appropriate” instructional moves (Heritage, Kim, Vendlinski, & Herman, 2007).
From these perspectives, the examples we gave earlier of responsive teaching reflect the work of experienced, accomplished practitioners. There is little hope of recruiting or training a million teachers like Deborah Ball, the reasoning goes, but it is not difficult to imagine large-scale implementation of well-defined strategies, such as “traffic-lighting” (Black et al., 2003) “two stars and a wish” (Keeley, 2008), reflective toss (van Zee & Minstrell, 1997), and wait time (Rowe, 1974). These represent clear, tangible steps teachers can take in class, and so research on formative assessment has produced professional development to leverage such strategies (Black, Harrison, Lee, Marshall, & Wiliam, 2004a; Furtak et al., 2008).
The work we have done in the first half of this paper belies that premise. Examining particular instances in four articles from prominent work, we showed that, while the teachers were using the strategies they had been taught, they were not engaging with student ideas. The example we provided in the second half of this paper, in contrast, shows a teacher's early success in becoming more aware of and responsive to student thinking, without the benefit of any particular strategies. Of course, Terry was an experienced teacher, but his entrance into these practices did not begin with strategies. It began, rather, with a shift of attention, with a shift in how he framed, and asked his students to frame, what was taking place in class.
An orientation towards responsiveness to students' ideas and practices resonates with work in teacher education, particularly in mathematics, that has pushed for more practice-based accounts of effective preparation (Ball, Thames, & Phelps, 2008; Kazemi, Franke, & Lampert, 2009), and that comes with calls for learning to teach “in response to what students do” (Kazemi, Franke, & Lampert, 2009; p. 1) and more attention to “demands of opening up to learners ideas and practices connected to specific subject matter” (Ball & Forzani, 2011; p. 46).
We challenge the view that it is difficult for teachers to learn to attend to the substance of student thinking. Recent work in science and math teacher education (Coffey, Edwards & Finkelstein, 2010; Kazemi et al., 2009; Levin, Hammer, & Coffey, 2009; Levin & Richards, 2010; Singer-Gabella et al., 2009; Windschitl, Thompson, & Braaten, 2011), has presented evidence of novice teachers' attention to student thinking, novices whose preparation emphasized awareness and interpretation of student thinking as evident in video records and written work. By this reasoning, much depends on how teachers frame what they are doing, and a primary emphasis on strategies may be part of the problem. Assignments that direct teachers and teachers-in training to what they are doing may inhibit their attending to what students are thinking. Our analyses above show that the same applies to researchers.
Thus, in closing, we argue for a shift of researchers' attention to attention, that is from the strategies teachers use to the focus of their attention in class, and with that a re-framing of what assessment activities entails. First and foremost, we propose that it is essential for teachers to frame what is taking place in class as about students' ideas and reasoning, nascent in the discipline. Formative assessment, then, becomes about engaging with and responding to the substance of those ideas and reasoning, assessing with discipline-relevant criteria, and, from ideas, recognizing possibilities along the disciplinary horizon. Framed as such, assessment demands attending to substance, in research, professional development, as well as in classrooms. With this reframing, many teachers will be able to do something much more akin to what Ball was doing, provide systemic support for that focus of their attention to substance. As the example from Terry's classroom illustrates, with attention on substance and a framing of making sense of and responding to ideas, formative assessment moves out of strategies and into classroom interaction, with roots in disciplinary activity and goals.