Navigated learning: An approach for differentiated classroom instruction built on learning science and data science foundations

Classroom teachers are often provided with instructional resources and assessment systems that dictate one pathway for every student's learning and evaluation. These practices remain common despite new affordances available through data-rich, emerging digital technologies that draw on data science and learning science foundations to complement and enhance traditional instruction. This paper presents a conceptual framework for Navi-gated Learning, a pedagogical approach that operationalizes learning principles using emerging ideas in artificial intelligence and data science, resulting in the continuous, real-time generation of students' cognitive and noncognitive data to support a teacher's ability to utilize the system to customize instruction. The paper articulates the learning principles underlying the pedagogical approach and the features afforded by the Learning Navigator system. The paper concludes with two cases of very different implementation of Navi-gated Learning focused on fifth grade and ninth grade students' learning of mathematics

(Organization for Economic Co-operation and Development [OECD], 2015). On PISA mathematics, American students ranked below the OECD average at 35/44. Economic analyses inform us that a growing proportion of jobs are concentrated in STEM fields and that economic growth and financial stability are connected to STEM education systems (OECD, 2019). Policy documents also reveal that even as many young adults are unemployed or underemployed, there is a shortage of high-skilled workers with the training needed for jobs of the future.
Employers state that future workers need to be: (a) digitally skilled with the ability to continuously refresh their knowledge and skills, (b) fluent in critical thinking and collaboration, and (c) able to take ownership of their own learning (Pompa, 2015). Globalization has resulted in multilingual and multicultural learners with diverse knowledge and interests, many of whom have largely not been allowed to experience learning opportunities to meet these new demands. In sum, research and policy analyses suggest that to be globally competitive, we need strong K-12 STEM education programs that support problem-based, active learning-focused, and differentiated instruction.
Far too often, classroom teachers are provided with instructional resources and assessment systems that dictate one pathway for student learning and evaluation. In other words, a great deal of classroom instruction is focused on getting all students in the same class to the same "destination"-that is, the standards, curricular, or assessment metrics that we expect students to demonstrate proficiency on by the end of the course. Differentiated instruction is the presentation of customized and adaptive curricula optimized for each learner's knowledge, skills, abilities, and interests (Tomlinson et al., 2003). With the high degree of academic, cultural, and linguistically diverse populations in American classrooms, teachers face enormous challenges to provide meaningful, differentiated instruction to all students.
As stated by Tomlinson et al., "While heterogeneous instruction is attractive because it addresses equity of opportunity for a broad range of learners, mixed-ability classrooms are likely to fall short of their promise unless teachers address the learner variance such contexts imply. In such settings, equality of opportunity becomes a reality only when students receive instruction suited to their various readiness levels, interests, and learning preferences, thus enabling them to maximize the opportunity for growth (Tomlinson et al., 2003, p. 120).
Emerging digital technologies present both an incredible opportunity and new challenges in many contexts (Yan, Gaspar, & Zhu, 2019), including classroom learning. Advances in artificial intelligence and data science afford new possibilities for teacher customization and differentiation of learning. As stated by Collins and Halverson (2010), "While the industrial revolution gave rise to a universal schooling system where none had previously existed, the information technology revolution presses a very real, active system to reconsider its fundamental practices" (Collins & Halverson, 2010, p. 18).
It is not surprising that realizing the opportunities afforded by emerging digital technologies is not straightforward. While emerging technologies have been available for several years, few learning systems exist that provide such affordances to address the challenges of differentiated instruction in today's classrooms. One system, Carnegie Learning's Cognitive Tutor, uses frequent assessments to offer individualized learning similar to a tutoring experience. The Cognitive Tutor implements cognitive science principles into the system to promote efficient student learning and engagement. Research results demonstrate that learners are a grade level ahead of students who engage in more traditional learning approaches (Ritter, Anderson, Koedinger, & Corbett, 2007). i-Ready empowers teachers and students with data to help take ownership of learning and to differentiate instruction. The diagnostic and assessment reports support teacher decisions to help students gain proficiency. A case study of an elementary school in northwest Florida implementing all aspects of the i-Ready suite saw student proficiency increase from 52 to 59% as an overall school score, which is just shy of earning a top-rated school credential (i-Ready, 2018 presents a conceptual framework for Navigated Learning, a pedagogical approach that operationalizes learning principles using emerging ideas in artificial intelligence and data science resulting in the continuous, real-time generation of students cognitive and noncognitive data.
The system utilizes a learning data backbone, that is, data and information from online and offline learning, to support customized and informed decision-making by teachers or greater ownership and monitoring of one's own learning whether it is occurring in online or offline settings. The paper also presents empirical results to address the question, How did different teachers integrate Navigate Math into their classroom, and what evidence of learning was demonstrated?
The paper concludes with data and examples from the implementation of in the Learning Navigator in two-fifth grade and two-ninth grade classes of students learning mathematics in diverse schools in Northern California, USA.

| WHAT IS NAVIGATED LEARNING?
The focus of our work is interdisciplinary research that goes beyond merely the design of a data-rich backbone (e.g., the technical side) to include the design and examination of an educational approach (e.g., the human side of emerging digital technologies). Navigated Learning is a pedagogical approach with foundations in the learning sciences that are made operational through data science. In other words, Navigated Learning is realized when fundamental principles of learning are identified and operationalized within a data-rich technology system called the Learning Navigator. The system realizes continuously updated information on each student's customized learning pathway.

| The learning navigator
The data-rich technology system, the Learning Navigator, is comprised of three technology enablers (see Figure 1). These are: (a) the Navigator Competency Model (NCM), which organizes the learning space, (b) the Learning Data Backbone, which captures continuous data in real time and informs the suggestions and re-routes, and (c) the Learning Apps, which include all of the resources and intelligent components that present a differential interface for each stakeholder (e.g., students, teachers, and administrators).
NCM is the conceptual model and framework for the Navigator ties associated with it and information regarding proficiency (e.g., student struggles or depth of knowledge of the competency).
The collective set of resources and tags organized by the framework become central components of the (Learning) Navigator system.
The Learning Data Backbone is the rich and complex set of data collected when students and teachers interact with the system resulting in continuous updates on the student, teacher, and catalog information. As mentioned, all learning resources and assessments in the Navigator are aligned to competencies with many metadata tags.
Each interaction within the system results in newly created links between user information and resources. Metadata are computed from the activity stream data and the efficacy of the learning activities are measured against the competencies mastered using information from the catalog. Once the system has complete activity stream data from a set of learner's interactions, the system computes learner vectors to continuously update a more precise location of the learner. A similar process exists to operationalize the learning principles. The interaction of the learner with the system creates a data activity stream. This then informs the action suggested to the learner based on the learning principles.
The learning data backbone underlies the Learning Apps for students, teachers and administrators (collectively referred to as Users).
The Apps focus on providing the stakeholders data, analysis, and suggestions using open educational resources. Users can then make decisions toward achieving learning outcome gains on the competencies that the student needs to learn. Navigated Learning enables a variety of providers the ability to bring content, tools and implementation services to benefit a number of users including their own prioritized students, teachers, parents, and/or administrators.

| Four elements of navigated learning
As mentioned, Navigated Learning is a pedagogical approach with foundations in the learning sciences. Navigated Learning is premised on the principle that effective guiding and supporting of learners begins with the gathering and organizing of the data and information made available by the Technology Enablers. Data organization and application towards student learning is accomplished through the four elements of Navigated Learning (see Figure 2). The first element, Locate, refers to the set of competencies, metatags, rules, and F I G U R E 1 The three technology enablers of the Learning Navigator interactions that result in the designation of a real-time, successive approximation of a "location" of the learner's current knowledge, skills, and noncognitive attributes at any learning time and place. In other words, in order for a teacher, a learning system or a student to guide learning, the Navigator system and the Users must gather and organize successive approximations of what knowledge and preferences each learner has and has not demonstrated. In Locate, the system uses machine learning techniques to precisely locate the learner based on data about the learner's knowledge, skills, and noncognitive data, such as the student's interests. Recognizing that any characterization of a learner is always an imperfect approximation of what an individual actually knows and likes, the system filters and organizes continuously generated and updated data for increasingly improved successive approximations of location (Diwan, Srinivasa, & Ram, 2018).
In Locate, proficiency information is gathered at the competency level. Student struggles and depth of knowledge are identified at each competency. Struggles refer to areas where student errors or alternative ideas are common. When a student demonstrates a common struggle in response to a particular assessment item, the system will generate a Hint and Solved Examples from open education resources (OER) to guide the learner in next steps to continually update the learner's location. Table 1 provides an example of a cognitive competency from Common Core Math Standards and resources associated with each competency. For each competency, the design team creates a Student-friendly Display Name.
Competencies can be organized into an instructional sequence (i.e., Learning Pathway) by either the metadata tags themselves (e.g., Common Core Standards), or through customization by the teacher or student. A competency will generally also be identified with a prerequisite list of competencies that need to be acquired in order to be able to engage successfully with the focal competency.
Once the collective set of metadata and tags are assigned for each competency, the Locate element of Navigated Learning uses the learner's interactions with these metadata and tags to continuously characterize and represent a learner's highest level of competency have demonstrated within the system (see Figure 3, green line). Each Skyline represents a student's competencies in the three-dimensional metric space. The green line represents the grade line, for example, the end destination of that learning pathway for that course. Subjects can be a core curriculum, noncognitive skill or vocational skill. At any given moment, the teacher or student can identify a student's aggregate competency level, for example, the students' Skyline, by looking at the line connecting the competency levels of a collection of competencies in a given Learning Pathway.
The second element of Navigated Learning is Curate. Curate Additionally, Signature Assessments are offered to students as a suggestion and ask students to think more critically and apply their knowledge demonstrating a greater complexity of understanding.
Assessment data can be entered into the system in a variety of ways, including through direct entry or through photo-entry of off- After learning principles were identified, the design team established a set of models for the operationalization of the learning principles called Event Condition learning Principles and Action (ECPAs). These models "listen" to events and based on the condition about a learner, the models trigger actions which are suggestions based on the learning principles associated with the model. Table 3 illustrates four ECPAs associated with decision-making in the Naviga- F I G U R E 3 Three-dimensional Skyline science. Figure 5 shows the flow chart of ECPA ending in the suggestion informed by the learning principles.
To train the algorithm to operationalize the learning principles, F I G U R E 4 Dashboards from Live Assessments show the answers that students got correct and incorrect in real-time. This image shows data by student (green if correct and red if incorrect). The teacher can click any question number to see a specific student answer. The high level overview allows the teacher to review a question that posed problems or for other reasons. Teachers can also quickly get a big-picture view for individuals or groups by color. Other views can sort by ascending and descending scores, time spent on each task, and student reactions as an emoji for each item T A B L E 2 Five learning principles selected for operationalization in the Navigator system Learning principles Principle 1: Students learn best when they are actively engaged in constructing new learning on a foundation of prior knowledge and experience (e.g., NRC, 2000;NRC, 2018) Principle 2: Students learn best when their learning opportunity is a stretch learning experience; e.g., it builds on what they know and provides guidance but also extends or applies what they know in a new way, whether that extension is to a new context, a chance for them to make an inference, an analogy, or to a surprising next step (e.g., Vygotsky, 1978;NRC, 2019).
Principle 3: Students learn best when they have the opportunity to revisit an idea or concept multiple times, including revisiting in response to feedback and when revisiting is new flavors or variations on the original (e.g., Bruner, 2009;Reiser, 2004).
Principle 4: Assessment is always an imperfect measure of what someone knows. Therefore, frequent embedded assessment, multiple levels of challenge and multiple kinds of evidence are the best means to generate a solid estimate of progress of critical thinking, knowledge and skills (e.g., Bransford et al., 2005).
Principle 5: Choice, within reasonable limits and with supports, fosters engagement, confidence-building, and perseverance. Learning environments that foster trust and risk-taking with guidance foster deeper engagement, confidence-building, and perseverance (e.g., National Research Council, 1987).
Independently a list of possible events that can occur were listed.
The events included interactions with resources, assessments, and practices, and activities that teachers realize in association with the data and resources within the Navigator system in order to monitor progress and personalize suggestions. In Facilitate, the teacher plays a critical role in differentiating instruction using data offered by the system. For example, Suggest in Facilitate provides real-time data to the teacher who then uses these data to offer suggestions to the student.
The teacher's dashboard has high-level views that illustrate proficiency and progress as well as an ability to do a deep dive by student and domain and standard. Three views of data and information from the Teacher Dashboard are illustrated in Figure 6. As seen in these views, teachers are able to analyze data in the moment and obtain information to guide individual work with students, organize pairs or small groups, or otherwise tailor instruction to student needs.
Teachers are also able to monitor and adjust instruction for the whole class based on data covering student engagement, performance, and proficiency. In the dashboard showing Course Activities, teachers see results from the Live Assessment, which provide real-time data regarding student answers, time on each problem, and student reactions to questions. This dashboard allows a quick review of a challenging problem and/or confirmation if a concept and skills was well understood and by which students. From the data, teachers can make suggestions to individual students, such as offering extra resources to see content in a new context or more challenging practice problems to aid in extending knowledge development.
For students, the Navigator also provides valuable, real-time feedback through data reports and a competency-based proficiency dashboard. Students can view reports at any time to see their engagement, performance, or reaction to any content they have studied. Students are also able to view their personalized proficiency dashboard to analyze their competency progress and see the scores on assessments, time spent on collections and assessments, and proficiency by competency.
In summary, the four elements of Navigated Learning articulate the dimensions of our interdisciplinary system that take into account both the technical and the human side of emerging digital technologies in real settings. The four elements work in concert to support the possibility of greater differentiated learning. The following two cases F I G U R E 5 A flow chart for event, condition, learning principle, and actions. Each event, condition and overlying learning principles go into the suggestion system to help determine the action suggested to the learner. The process starts with an event triggered by a student interaction with the system. Each event has a condition associated with it. There can be one to several learning principles that are triggered from the event and condition to then determine the action suggested to the student present illustrations of how Navigated Learning was used with two different populations of students and teachers.

| HOW ARE STUDENTS AND TEACHERS USING NAVIGATED MATH?
To evaluate the Navigator system and the Navigated Learning pedagogical model, we implemented one Navigator course, Navigate Math, within two different classroom settings in diverse Northern California classrooms.  Table 4). While the backbone content within the system was drawn from

| Navigated math instructional materials
OERs, teachers and students were encouraged to bring in other content, projects, and practices to support learning from their existing district approved curriculum and then take assessments in the system to track learning and proficiency. The metadata tags, including tags on standards and depth of knowledge, continuously updated information available to both the learner and teacher so that each could make informed and focused decisions to promote learning. The instructional materials were organized to create a flexibility to be variously adapted to a range of F I G U R E 6 Teacher dashboards. On the left, teachers see a high level overview of students' proficiency on course material. In the middle, the class roster view includes suggestions to give the students based on a student's performance on an assessment. The teacher can visualize the data over different periods of time. On the right, the teacher visualizes a student's skyline and can track competencies gained and in progress over time. The teacher can make suggestions in areas of demonstrated struggle or gaps implementation models as desired by teachers in different classrooms and with different audiences and goals. When learners were working on activities tied to Common Core Standards, teachers and students could observe real-time data to make suggestions or follow suggestions to review or extend the learning, such as to try more challenging content.

| Navigate math assessment materials
The Navigate Math assessment system provided a series of organized, embedded assessments that provided data to the Navigator system to continually update a learner's Skyline. To track student progress and proficiency, learners took Course Assessments within the course map along each learner's personalized Learning Pathway. All assessments in the course were mapped to competencies. The assessments within the Navigate Math course were also designed to assess conceptual understandings. Course Assessments were typically five to seven questions that cover depths of knowledge one through three (Webb, 2002). Webb's depth of knowledge is on a scale of 1-4 where level 1 is recall, level 2 is skills and concepts, level 3 is short-term strategic thinking, and level 4 is extended thinking. In addition, students interacted with embedded Course Assessment items which were used to identify gaps in student proficiency, and to confirm an accurate diagnosis of student performance. The Navigate Math proficiency was defined as follows: If a learner earned >80% on a Course Assessment, the student's Skyline indicated proficiency for that competency and the student was offered an additional in depth Signature Assessments to demonstrate a greater depth of knowledge on the competency. If a student earned less <80% on the Embedded Assessment, an additional highly curated collection was suggested that was targeted to the gaps identified from the assessment.

| What did professional development look like?
Teacher training was essential to ensure teachers were comfortable in using the Navigator to foster Navigated Learning. All Navigate Math  p ≤ .0001) shows a direct positive correlation to competencies mastered in Navigator for Math to overall Winter MAP scores. On average, students who gained more competencies in Navigator for Math had higher overall MAP scores. In addition, after 3 months of interaction with Navigate Math but a relatively small dose (e.g., average of 7.8 hr per student), 70% of students in one class and 50% of students in the other class met the MAP goals that were set by their teachers in the fall MAP testing period (see Table 5).

| Case 2: Remedial ninth grade math
In Case 2, the teacher used the Learning Navigator with two full periods of remedial math where students were 3-5 grade levels behind in math.  Table 6) and the covariate corresponding to competencies gained on Navigate Math was jointly significant across dependent variables (F = 4.09, p = .027). As mentioned earlier, there was no correlation between competencies in Navigator Math and Winter MAP scores on the other topics that were not studied using Navigator Math, such as Geometry. These ninth grade student results were particularly encouraging to the classroom teacher and school administrators as they provided promising early results for students who had not had a great deal of prior success in these mathematics topics.
Despite the technology-rich system and data-rich backbone, a cen-