SEARCH

SEARCH BY CITATION

Keywords:

  • Cognitive science;
  • History;
  • Artificially intelligent tutoring;
  • Reading comprehension;
  • Mathematics education;
  • Physics education

Abstract

  1. Top of page
  2. Abstract
  3. 1. Meanwhile at the Office of Naval Research
  4. 2. A political crisis in educational research
  5. 3. The intervening years: 1984–2008
  6. 4. In summary: Accomplishments of cognitive science research in education
  7. 5. Future directions
  8. References

This paper reviews 30 years of progress in U.S. cognitive science research related to education and training, as seen from the perspective of a research manager who was personally involved in many of these developments.

When the Cognitive Science Society was founded in 1978, I had already been working for the U.S. National Institute of Education (NIE) for 2 years, attempting to attract cognitive researchers to work on significant educational problems. Just as I joined the agency, proposals for a Center for the Study of Reading were being reviewed, and I sat in on the review. The RFP (request for proposals) already had a cognitive science character, calling for the participation of researchers from multiple disciplines: psychologists, linguists, educators, and perhaps more. The emphasis of the RFP was on reading comprehension. Both a small workshop (Miller, 1973) and a massive conference in 1974 that produced a 10-volume report concluded that reading comprehension should be the priority. It was felt that “we” knew how to teach early reading, and the gap in reading achievement between minority group and majority group students was closing for the early years of schooling. The Center for the Study of Reading went to Richard Anderson at the University of Illinois and his many collaborators, including Ann Brown, Nancy Stein, and Andrew Ortony, all recognized members of the cognitive science community.

Shortly thereafter, I unexpectedly took over the management of a large grants competition in Teaching and Learning. This competition had several subtopics, including reading comprehension, mathematics learning, testing, and research on teaching—where the goal was movement from behaviorist research to discourse analysis. The reading comprehension competition revealed that there were few researchers working on reading comprehension: Proposals on letter and word recognition were about as close as one could get. To borrow behaviorist terminology, shaping of researchers’ efforts went on over several years of NIE’s investments so that, ultimately, researchers were beginning to address questions of reading comprehension. The reading comprehension research program of Just and Carpenter began under the NIE grants program, and Walter Kintsch also received an NIE grant early in the development of his work on comprehension. The emphasis that NIE placed on reading comprehension made it a major topic in cognitive psychology and cognitive science. For a few years, the NIE investment in the single topic of reading comprehension was about the same as the entire budget of the Memory and Cognition program at NSF. It was enough to shape the direction of the field, an effect parallel to the effect of the recent large investments in cognitive aging research.

NIE’s basic skills agenda included mathematics learning. At that time, research on mathematics learning was even less well developed than research on reading. Very few people had been able to make a career out of doing research on mathematics learning and teaching. If one were to review a topic or question, most of the research that would be found would be unpublished doctoral dissertations in education. The career researchers who did exist functioned as unquestioned gurus of the topic areas they addressed because there were so few researchers: the field was below critical mass. The contrast with the situation in cognitive developmental research, with lively arguments among researchers exploring Piagetian tasks, as one example, was striking. Some research on topics relevant to mathematics education was, of course, being done by psychologists. However, the two groups of researchers, mathematics educators and cognitive psychologists, rarely read, cited, or were in any way influenced by each other’s research, exacerbating the critical mass problem. A few research efforts from that era stand out in my mind: Sandra Marshall did research that combined the analysis of mass test data with computational cognitive modeling of problem-solving processes, eventually leading to her schema theory of math word problem solving (Marshall, 1995). Ginsburg and Russell (1981) did research demonstrating that disadvantaged children had the same basic number concepts as more advantaged children, if only one asked the questions in just the right way. Lave (1988) received a grant that was supposed to illuminate the way people use mathematics in their lives. In addition, research by Greeno, Resnick, and others at the Learning Research and Development Center was supported via institutional grant support.

In 1978, just as the Cognitive Science Society was founded, I was asked to be the NIE program officer on a joint NIE-NSF program that was called “Cognitive Processes and the Structure of Knowledge in Science and Mathematics.” The intent was to get excellent cognitive researchers to work on problems in mathematics and science education, and to support researchers originating in traditional scientific fields who had begun doing research of a cognitive character on the teaching and learning of science. The creation of this grants program was probably the long-delayed effect of the first conference bringing psychologists together with people concerned with mathematics and science education, reported in Bruner’s (1960) book The Process of Education. Although only two competitions were held, in 1978 and 1979, some very well known and influential research was supported by it, such as McCloskey, Caramazza, and Green’s (1980) research on physics misconceptions. Larkin and Simon’s (1987) well-known work on the role of diagrams in scientific thinking was also supported in this program. A side effect of the program was the recruitment of many young cognitive psychologists to this kind of research, largely through their service as reviewers. For example, Susan Carey, winner of the 2008 Rumelhart Prize, served as a panel reviewer in the first year and then later submitted a proposal that initiated her well-known program of research on children’s understanding of heat, temperature, and related concepts.

In addition to the grants, at NIE we issued some RFPs for specific research, some of it quite basic in character. These were for larger projects that could not reasonably be awarded out of the regular grants competition budget, projects that we had to design and sell to upper management in order to get funding allocated. Interesting examples included “Research on the Perception and Comprehension of Graphs and Charts” for which the key performers were Kosslyn and Pinker (Kosslyn, 1989). Another was “The Cognitive Demands and Consequences of Computer Learning” that pursued the claims then being made by Papert. Another led to a survey of programs that claim to teach general thinking, learning, or problem-solving skills (Nickerson, Perkins, & Smith, 1985).

1. Meanwhile at the Office of Naval Research

  1. Top of page
  2. Abstract
  3. 1. Meanwhile at the Office of Naval Research
  4. 2. A political crisis in educational research
  5. 3. The intervening years: 1984–2008
  6. 4. In summary: Accomplishments of cognitive science research in education
  7. 5. Future directions
  8. References

At the same time as cognitive educational research was getting underway at NIE, the Office of Naval Research was becoming a significant factor in supporting the emerging field of cognitive science. At the level of relatively basic research on learning and teaching methods, the distinction between education and training is not very important. In what was then called the Personnel and Training research program, ONR was supporting rather diverse cognitive research in some way relevant to training and testing applications. There was a special interest in investigating the cognitive processes involved in doing test items and also in ending the disconnect between psychometric theory and modern cognitive understanding of knowledge and skill (Frederiksen, Glaser, Lesgold, & Shafto, 1990; Nichols, Chipman, & Brennan, 1995). Both Robert Sternberg and John Anderson became ONR grantees as soon as they finished graduate school. There was something of an emphasis on spatial and imagery abilities (Carpenter and Just, Kosslyn, Hunt, and Pellegrino) because the Armed Services Vocational Aptitude Battery did not include any tests of spatial ability, despite its evident importance to many military tasks. During this period, ONR also became known for its support of research on expertise that could be seen as defining cognitive objectives for instruction (Chi, Glaser, & Farr, 1988). In the broader research community, many viewed cognitive science research approaches as too risky and preferred the analytic approach characterizing experimental psychology—studying very simple, elementary tasks in the laboratory that might, hopefully, one day cumulate to something that could be applied. For ONR, I think the prospect of application to complex, realistic training within a reasonable amount of time (perhaps 30 years) compensated for increased scientific risk. This is one major reason why ONR became so important to the development of cognitive science.

Like the other DoD agencies, ONR had played an important role in the development of education and training technology, but ONR became particularly important to the development of artificially intelligent tutoring technology, probably because ONR had the strongest basic research orientation and therefore the longest term outlook on research investments. The first ONR award in artificially intelligent tutoring was made to the late Jaime Carbonnell at BBN in 1969. The computer used in the research cost $2 million. Alan Collins, John Seeley Brown, and Kurt VanLehn (then a graduate student at MIT) were also involved in ONR-supported tutoring research at BBN.

2. A political crisis in educational research

  1. Top of page
  2. Abstract
  3. 1. Meanwhile at the Office of Naval Research
  4. 2. A political crisis in educational research
  5. 3. The intervening years: 1984–2008
  6. 4. In summary: Accomplishments of cognitive science research in education
  7. 5. Future directions
  8. References

When I went to work at NIE in 1976, I was surprised to discover that there was a right-wing way of teaching reading (phonics) and a left-wing way of teaching reading (an emphasis on comprehension and interpretation), a right-wing way of teaching math (memorization), and a left-wing way of teaching math (again an emphasis on comprehension). It seemed to me that these should be empirical questions, and that the answers were likely to be more complex than envisioned by the participants in the reading wars and the math wars. Most of the curriculum development projects at both NSF and NIE had already been terminated because of pressures from the political right. Ironically, cognitive science played an important role in right-wing opposition to the curriculum development projects, although traditional issues of local control and states’ rights were also involved. The bête noire of the right wing was a curriculum called MACOS (Man: A Course of Study). The outline of this curriculum was virtually identical to a very early (1960–1961) cognitive science course that Jerome Bruner and George Miller taught together at Harvard: Psychological Conceptions of Man. What most provoked the right wing were films by anthropologist Irven DeVore that expressed an evolutionary viewpoint and cultural relativism.

When Reagan was elected President, his administration moved rapidly to implement the right-wing educational agenda. The budget of the Science Education Directorate at NSF promptly went to zero and almost the entire staff was dismissed. The demise of NIE came more slowly. Many of the grants which had been selected in the last grant competition fell victim to a budget recision taking back money that had previously been allocated. The internal environment became extremely and unpleasantly politicized.

During this catastrophic period for educational research funding, the James S. McDonnell Foundation under the leadership of John Bruer stepped into the breach, creating their program Cognitive Studies for Educational Practice. A very select group of grantees was able to continue their research with support under that program.

3. The intervening years: 1984–2008

  1. Top of page
  2. Abstract
  3. 1. Meanwhile at the Office of Naval Research
  4. 2. A political crisis in educational research
  5. 3. The intervening years: 1984–2008
  6. 4. In summary: Accomplishments of cognitive science research in education
  7. 5. Future directions
  8. References

3.1. ONR and artificially intelligent tutoring

Early in 1984, I moved from NIE to ONR, a relatively apolitical environment where an ONR program officer position happened to be open in the Personnel and Training Research area. To my surprise, by December 1985, I found myself primarily responsible for managing, reviewing, and defending the ONR program. At that time, the program addressed a very broad cognitive research agenda, with spotty coverage. Over the years I refined the focus of the program to emphasize what seemed to make ONR unique in the total federal funding scene: computational theories of human cognitive architecture and artificially intelligent tutoring systems (ITS).

As of 1984, Henry Halff had just obtained special funding for the first practical application of ITS to a Navy maintenance training system (Towne, 2007), as well as several years of substantial funding for a combination of intelligent tutoring research and research related to basic skills training (i.e., remedial reading and math). The basic skills emphasis may seem surprising, but the military inherits the failures of the schools. Some of the most significant publications produced under that Navy Training research budget were: Anderson, Boyle, Corbett, & Lewis (1990), Clancey (1986), Wenger (1987), Schofield (1995), and Marshall (1995).

When this Navy Training program began, building an intelligent tutor was legitimately a basic research project, although the constructive nature of the work made it quite different from typical scientific investigations of natural phenomena. By the time it ended, a limited community knew how to build tutors that could be expected to be effective. Tutors were becoming an applied enterprise. At ONR, basic research investments in intelligent tutoring turned to the unsolved problem of true natural language interaction. Computational linguists played a major role in this instructional research, and they began to discover and define instructional strategies at a very fine grain, as they appear in the interactions of human tutors and students. This was very exciting multidisciplinary cognitive science research because phenomena that had never been studied before were now subject to very careful study by multidisciplinary teams, in order to support high-quality artificial imitation of the human tutorial behavior.

Eventually, Evens was able to produce the first tutor with true natural language interaction capability that was good enough to be used with actual students for real instruction—in this case medical students of cardiac physiology. The story of this tutor and much related work can be read in Evens and Michael (2005), Graesser, VanLehn, Rosé, Jordan, and Harter (2001) and Chipman (2004). Another landmark event in this research came when Stanley Peters added natural language interaction—including high-quality speech recognition and generation—to a tutor of shipboard damage control (fire-fighting) that David Wilkins had been working on.

3.2. Reading comprehension

Under the Reagan administration, priorities for reading research abruptly switched to phonetic approaches to reading instruction. The RFP for the renewal of the Center for the Study of Reading called for a review of research on the value of phonics instruction, even though the major publication of the center, Becoming a Nation of Readers (Anderson, Hiebert, Scott, & Wilkinson, 1985), had already endorsed the importance of early phonics instruction. Under the leadership of Reid Lyon, NICHD invested heavily in phonetic approaches to reading instruction. This culminated in the recently completed Reading First Impact Study (Gramse, Jacob, Horst, Boulay, & Unlu, 2008), a large-scale implementation study emphasizing phonetic instruction that showed no positive impact on student reading comprehension and only a minor improvement in decoding skills. This entire politically motivated episode made me feel that we had cycled back to my entrance on the scene 30 years ago.

However, research on reading comprehension did continue during these years, primarily under the support of NIMH; it had become a major topic for research in cognitive psychology. The McDonnell Foundation’s CSEP program also supported a number of projects that addressed reading and writing for learning: Ann Brown, Scardemalia and Bereiter, Kathy Spoehr, and Kintsch. The lack of attention to reading comprehension instruction during the Reagan and Bush administrations did mean that no one was paying attention when useful research results were emerging, results that might have application in instruction. No one was actively promoting that application to new curricula for reading comprehension instruction.

Despite this setback, today several promising approaches to reading comprehension instruction have emerged. Under the Center for the Study of Reading, Palincsar and Brown (1986) developed the reciprocal teaching approach, which has the major virtue of being easy for teachers to understand and implement. I myself revisited the problem of reading comprehension instruction when the Navy training establishment asked for reading comprehension courseware suitable for use with Navy recruits reading at the grade 5–8 level. I published a Small Business Innovation Research (SBIR) topic that called for the development of reading comprehension courseware aimed at adult readers and cited key publications about reading comprehension. One award emphasized the teaching of inference making. A second emphasized the use of carefully graded reading passages and automatically tutored summary writing, and a third attempted to approximate reciprocal teaching (without, however, natural language capability). Tom Landauer’s company, Knowledge Acquisition Technologies, also developed a program called Summary Street that provides interactive individual tutoring in shaping up a quality summary. Evaluations have shown that its use can make significant improvements in reading comprehension as well as in the quality of summaries that students write (Caccamise, Franzka, Eckhoff, Kintsch, & Kintsch, 2007). Another major effort has been the research program of McNamara, O’Reilly, Rowe, Boonthum, and Levenstein (2007) that produced both a human implemented training approach called Self Explanation Reading Training (SERT) and iSTART, a computerized version of SERT. SERT and iSTART attempt to teach several different reading comprehension strategies. Evaluation results are promising.

After 30 years of research on reading comprehension, there are promising new approaches to comprehension instruction that school reformers could find and implement. A significant investment in developing computerized instruction for reading comprehension could combine all of these promising approaches into a program that would probably be highly effective. The time may be ripe for the Department of Education to fund another large experiment that actually focuses on teaching reading comprehension.

3.3. Mathematics learning

A project supported by the McDonnell Foundation CSEP program, to which I was an advisor, realized one of the goals that I had hoped to achieve in the NIE program: effectively teaching early number understanding at the time of school entry in a way that closed the gap between advantaged and disadvantaged children (Case, Griffin, & Kelly, 1999). Case had defined what he considered to be the cognitive structure of early number understanding, and a preschool curriculum of game-like activities was shown to result in much improved number understanding, closing the socioeconomic gap in readiness for the normal elementary school math curriculum. Further, it was shown that teachers could easily and effectively implement the curriculum. In my personal view, there is sufficient research evidence to support the policy that every school serving disadvantaged populations should be using it. Case and his associates did continue to pursue this approach into the teaching of more advanced mathematical content, including rational numbers and functions (Kalchman, Moss, & Case, 2001), with impressive success for rational numbers, obtaining effect sizes of 2.5–3.5 standard deviations (Moss & Case, 1999). Unfortunately, this line of research seems to have ended as a result of Case’s untimely death.

During these years, research on mathematics learning shared the funding problems of all educational research, but it was also hindered by the rising situated cognition movement. Both the teaching of pure mathematics and the standard cognitive approach of analyzing and researching the underlying cognitive structures in order to teach them relatively directly came under attack. Although I believe that the situated cognition movement highlighted some research issues worthy of more attention—issues of motivation and cultural influences—it also trashed the accomplishments of the standard cognitive approach just as they were becoming ready for practical application.

Despite skirmishes on the right and the left (Lave, 1988), the standard cognitive approach was well represented in these years by the artificially intelligent tutors of programming and high school mathematics built by John Anderson and his associates. Anderson began his work with intelligent tutors primarily because he wanted to study human learning on a realistic scale (numerous personal communications), but he was also interested in improving mathematics education. The LISP Tutor, an ONR project, was in fact thoroughly exploited as a laboratory in which to study processes of human learning and instructional strategies (Anderson, Conrad, & Corbett, 1989). Anderson’s first Geometry Tutor, which taught proof-making, was developed at about the same time under NSF support and was the first tutor to be tested in an almost normal school environment. Schofield’s observational ethnographic study of the introduction of the Geometry Tutor, an ONR project (Schofield, 1995), revealed that students were very much engaged with the tutor, being reluctant to leave it, despite the very plain, unadorned interface. Apparently there is considerable motivational value in a tutor that meets the student exactly where he or she is, enabling mastery of a subject with a reputation for difficulty.

Eventually, tutors were developed for first-year algebra, for the proof-less version of geometry then being taught in the Pittsburgh high schools, for second-year algebra, and for Pascal programming in AP computer science. A general computer programming tutor, which could tutor the same programming problem in multiple computer languages, was supported by the Army Research Institute and used by Anderson in teaching an introductory programming course at Carnegie-Mellon University.

Conventional publishers were not interested in publishing and selling Anderson’s tutors; thus, Carnegie Mellon University assisted the creation of a company for that purpose. To date, the first-year algebra tutor has emerged as the one tutor with large-scale use in the schools. There is considerable evidence that it substantially improves achievement in first-year algebra (Anderson, Corbett, Koedinger, & Pelletier, 1995). It has undergone considerable refinement over the years. According to CarnegieLearning’s Web site, more than 500,000 students in 2,600 schools have used the various tutors, and they are also available and supported for home schooling. Positive evaluations are typical (effect sizes of .3–1.3), the more so when a 3-year sequence of Cognitive Tutor instruction (algebra, geometry, and algebra II) is completed.

3.4. Science learning

The pioneering NIE-NSF study of physics misconceptions (McCloskey et al., 1980) has been cited thousands of times, particularly in the science education literature. Others in the science education community were also investigating misconceptions around that time. In the intervening years there have been at least four significant efforts that show promise of improving physics education, one at the high school level and several at the college level. Jim Minstrell, an outstanding high school physics teacher who became a researcher while still teaching, developed a classroom method for overcoming physics misconceptions. Initially funded by NIE, Minstrell later partnered with Earl Hunt and received grants from the McDonnell Foundation CSEP program (Hunt & Minstrell, 1994). These showed, first, that Minstrell’s teaching was far more effective than average, and secondly that other teachers could be taught to implement the method and be equally effective. Unfortunately work under a more recent NSF grant (Minstrell & Kraus, 2007) has shown that spreading this teaching approach more widely is not going to be easy.

At the college level, a significant community of researchers—from a physics background—has developed, researchers who are doing cognitive research on the teaching and learning of physics (Redish, 2003). Redish, a physicist, points out that cognitive science is the foundational field for this kind of research. Reif (2008), a pioneer of such research, continued to work throughout this period. A great many experimental investigations, often with many iterations, have been done to develop more effective ways of teaching particular topics in physics. A leader of this community is Lillian McDermott of the University of Washington, who has received very substantial support from NSF over many years and has produced two textbooks embodying these new teaching approaches. This work in physics is the first substantial example of the formalization of what Shulman (1986) has called pedagogical content knowledge.

There is also a well-known line of research on the learning of physics that is in the mainstream of the cognitive science community—the work of Chi, VanLehn, and Forbus, among others. Chi’s work on expert and novice classification of physics problems and her painstaking think-aloud study of physics learning are well known and heavily cited (e.g., Chi, Bassok, Lewis, Reimann, & Glaser, 1989). VanLehn simulated the learning of Chi’s students and also produced programs that assessed student performance against a computational model of problem-solving performance. All of this work was primarily supported by the basic research program at ONR. Having already modeled physics problem solving, VanLehn was in a good position to develop an artificially intelligent tutor for physics when some internal Navy politics led to the idea of developing intelligent tutors for the Naval Academy as applied research projects. Kurt VanLehn and several academy professors collaborated in producing and testing the Andes physics tutor, a coach aiding the solution of physics homework problems (VanLehn et al., 2005). Although Andes did not implement mastery learning because of severe constraints on Academy students’ time, it was shown to be somewhat effective in improving student achievement, and in the final years the content was extended from heavily researched mechanics learning to electricity and magnetism. The Andes Tutor is available to the wider community through the Open Learning Initiative.

A second project at the Academy involved the most thoroughgoing implementation of Ken Forbus’ CyclePad (Forbus et al., 1999). Chi Wu, a thermodynamics professor at the Academy totally restructured the thermodynamics curriculum at the Academy and wrote several textbooks for use in teaching thermodynamics with CyclePad. CyclePad also became the platform for Carolyn Rosé’s research on natural language for tutoring, also involving collaboration with an Academy professor.

4. In summary: Accomplishments of cognitive science research in education

  1. Top of page
  2. Abstract
  3. 1. Meanwhile at the Office of Naval Research
  4. 2. A political crisis in educational research
  5. 3. The intervening years: 1984–2008
  6. 4. In summary: Accomplishments of cognitive science research in education
  7. 5. Future directions
  8. References

In all of the areas reviewed, reading comprehension, mathematics learning, and physics education, significant progress has been made. Defining and pursuing cognitive objectives for instruction (Greeno, 1976) has proved fruitful. The cognitive science research approach has yielded instructional methods, sometimes embodied in artificially intelligent tutoring systems, sometimes in more traditional human-implemented teaching, that have been shown to produce significant improvements in student achievement.

Education was not among the disciplines, or even the hyphenated disciplines, of the original 1978 Sloan Foundation report on the state of the art in Cognitive Science (Keyser, Miller, & Walker, 1978). Now it appears in the logo on the cover of the journal Cognitive Science. The birth date of cognitive science itself has been variously identified as September 11, 1956 at a Symposium on Information Theory at MIT or in 1960 when Bruner and Miller founded the Center for Cognitive Studies at Harvard. The involvement with education and educational research came very quickly (Bruner, 1960; Miller, 1973). Education proved to be an important focus of and motivator for research in cognitive science. Much has been accomplished, but much is yet to be done.

5. Future directions

  1. Top of page
  2. Abstract
  3. 1. Meanwhile at the Office of Naval Research
  4. 2. A political crisis in educational research
  5. 3. The intervening years: 1984–2008
  6. 4. In summary: Accomplishments of cognitive science research in education
  7. 5. Future directions
  8. References

Given current enthusiasms, it is certain that for the next few years, much of the investment in research intended to inform education will involve neuroimaging of cognitive processes. The demands of neuroimaging result in an emphasis on very simple experimental tasks, contrary to cognitive science’s characteristic emphasis on dealing with complexity in human learning and performance. Realistically, improved instruction requires a behavioral research approach, and computational modeling with AI techniques aids in dealing with the complexity of both school learning and military training. Fortunately, the more traditional cognitive science approach to research on learning and instruction continues in the NSF-supported research center at Pittsburgh and elsewhere. One can hope that discipline-based communities of cognitive science research, such as that now flourishing in physics, will emerge with foci on additional subject matter: biology, chemistry, and medicine. In mathematics education, there is still a serious need to integrate the efforts of psychologists, cognitive scientists, and more traditional mathematics educators in order to speed up progress. Dissolving the barriers between disciplines is a challenge for research managers. Emerging cognitive science departments may benefit from providing a welcoming academic home for such researchers, who always occupy uncertain positions in the subject matter departments (physics, mathematics, chemistry, and biology).

References

  1. Top of page
  2. Abstract
  3. 1. Meanwhile at the Office of Naval Research
  4. 2. A political crisis in educational research
  5. 3. The intervening years: 1984–2008
  6. 4. In summary: Accomplishments of cognitive science research in education
  7. 5. Future directions
  8. References
  • Anderson, J. R., Boyle, D. F., Corbett, A. T., & Lewis, M. T. (1990). Cognitive modeling and intelligent tutoring. Artificial Intelligence, 42, 749.
  • Anderson, J. R., Conrad, F. G., & Corbett, A. T. (1989). Skill acquisition and the LISP tutor. Cognitive Science, 13, 467505.
  • Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4, 167207.
  • Anderson, R. C., Hiebert, E. H., Scott, J. A., & Wilkinson, I. A. G. (1985). Becoming a nation of readers: The report of the Commission on Reading. Champaign, IL: Center for the Study of Reading.
  • Bruner, J. S. (1960). The process of education. Cambridge, MA: Harvard University Press.
  • Caccamise, D., Franzka, M., Eckhoff, A., Kintsch, E., & Kintsch, W. (2007). Guided practice in technology based summary writing. In D.McNamara (Ed.), Reading comprehension strategies: Theories, interventions and technologies (pp. 375396). New York: Psychology Press.
  • Case, R., Griffin, S., & Kelly, W. M. (1999). Socioeconomic gradients in mathematical ability and their responsiveness to intervention during early childhood. In D. P.Keating & C.Hertzman (Eds.), Developmental health and the wealth of nations: Social, biological, and educational dynamics (pp. 125149). New York: Guilford Press.
  • Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: how students study and use examples in learning to solve problems. Cognitive Science, 13, 145182.
  • Chi, M., Glaser, R., & Farr, M. (1988). The nature of expertise. Hillsdale, NJ: Lawrence Erlbaum Associates.
  • Chipman, S. F. (2004). Overview: The U.S. Office of Naval Research training technology R&D. In Advanced technology for military training. Report number RTO-MP-HFM-101, paper 25. Neuilly-sur-Seine, France: NATO Research and Technology Organisation. Available at http://www.rto.nato.int
  • Clancey, W. (1986). From Guidon to Neomycin and Heracles in twenty short lessons: ONR final report, 1979–1985. AI Magazine, 7(3), 4060.
  • Evens, M., & Michael, J. (2005). One-on-one tutoring by humans and computers. Mahwah, NJ: Lawrence Erlbaum Associates.
  • Forbus, K. D., Whalley, P. B., Evrett, J. O., Ureel, L., Brokowski, M., Baher, J., & Kuehne, S. E. (1999). CyclePad: An articulate virtual laboratory for engineering thermodynamics. Artificial Intelligence, 114(1/2), 297347.
  • Frederiksen, N., Glaser, R., Lesgold, A., & Shafto, M. G. (1990). Diagnostic monitoring of skill and knowledge acquisition. Hillsdale, NJ: Lawrence Erlbaum Associates.
  • Ginsburg, H. P., & Russell, R. L. (1981). Social class and racial influences on early mathematical thinking. Monographs of the Society for Research in Child Development, 46(6), 69.
  • Graesser, A. C., VanLehn, K., Rosé, C., Jordan, P., & Harter, D. (2001). Intelligent tutoring systems with conversational dialogue. AI Magazine, 22(4), 3951.
  • Gramse, B. C., Jacob, R. T., Horst, M., Boulay, B., & Unlu, F. (2008). Reading first impact study. Final Report (NCEE 2009-4038). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
  • Greeno, J. G. (1976). Cognitive objectives for instruction: Theory of knowledge for solving problems and answering questions. In D.Klahr (Ed.), Cognition and instruction (pp. 123159). Hillsdale, NJ: Lawrence Erlbaum Associates.
  • Hunt, E., & Minstrell, J. (1994). A cognitive approach to the teaching of physics. In: K.McGilly (Ed.), Classroom lessons: Integrating cognitive theory and classroom practice (pp. 5174). Cambridge, MA: MIT Press.
  • Kalchman, M., Moss, J., & Case, R. (2001). Psychological models for the development of mathematical understanding: Rational numbers and functions. In S. M.Carver & D.Klahr (Eds.), Cognition and instruction: Twenty-five years of progress (pp. 138). Mahwah, NJ: Lawrence Erlbaum Associates.
  • Keyser, S. J., Miller, G. A., & Walker, E. (1978). Cognitive Science, 1978: Report of the state of the art committee to the advisors of the Alfred P. Sloan Foundation.
  • Kosslyn, S. M. (1989). Understanding charts and graphs. Applied Cognitive Psychology, 3, 185225.
  • Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65100.
  • Lave, J. (1988). Cognition in practice: Mind, mathematics and culture in everyday life. Cambridge, England: Cambridge University Press.
  • Marshall, S. P. (1995). Schemas in problem solving. Cambridge, England: Cambridge University Press.
  • McCloskey, M., Caramazza, A., & Green, B. (1980). Curvilinear motion in the absence of external forces: Naïve beliefs about the motion of objects. Science, 210, 11391141.
  • McNamara, D. S., O’Reilly, T., Rowe, M., Boonthum, C., & Levenstein, I. (2007). iSTART: A web-based tutor that teaches self-explanation and metacognitive reading strategies. In D. S.McNamara (Ed.), Reading comprehension strategies: Theory, interventions, and technologies (pp. 397420). New York: Psychology Press.
  • Miller, G. A. (Ed..), 1973). Linguistic communication: Perspectives for research. Newark, DE: International Reading Association.
  • Minstrell, J., & Kraus, P. A. (2007). Applied research on implementing diagnostic instructional tools (Final Report for ESI 0435727): National Science Foundation.
  • Moss, J., & Case, R. (1999). Developing children’s understanding of rational numbers: A new model and instructional curriculum. Journal for Research in Mathematics Education, 30(2), 122147.
  • Nichols, P., Chipman, S., & Brennan, R. (Eds.) (1995). Cognitively diagnostic assessment. Hillsdale, NJ: Erlbaum Associates.
  • Nickerson, R. S., Perkins, D. N., & Smith, E. E. (1985). The teaching of thinking. Hillsdale, NJ: Lawrence Erlbaum Associates.
  • Palincsar, A. S., & Brown, A. L. (1986). Interactive teaching to promote independent learning from text. Reading Teacher, 39, 771777.
  • Redish, E. L. (2003). Teaching physics with the physics suite. Hoboken, NJ: John Wiley & Sons.
  • Reif, F. (2008). Applying cognitive science to education: Thinking and learning in scientific and other complex domains. Cambridge, MA: The MIT Press.
  • Schofield, J. W. (1995). Computers and classroom culture. Cambridge, England: Cambridge University Press.
  • Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(4), 414.
  • Towne, D. M. (2007). Enhancing human performance via simulation-based training and aiding. Rotterdam, England: Sense Publishers.
  • VanLehn, K., Lynch, C., Schulze, K., Shapiro, J. A., Shelby, R., Taylor, L., Treacy, D., Weinstein, A., & Wintersgill, M. (2005). The Andes physics tutoring system: Lessons learned. International Journal of Artificial Intelligence in Education, 15, 147204.
  • Wenger, E. (1987). Artificial intelligence and tutoring systems: Computational and cognitive approaches to the communication of knowledge. San Francisco, CA: Morgan Kaufmann.