Edinburgh Research Explorer Dissecting learning tactics in MOOC using ordered network analysis

Background: Select and enact appropriate learning tactics that advance learning has been considered a critical set of skills to successfully complete highly flexible online courses, such as Massive open online courses (MOOCs). However, limited by analytic methods that have been used in the past, such as frequency distribution, sequence mining and process mining, we lack a deep, complete and detailed understanding of the learning tactics used by MOOC learners. Objectives: In the present study, we proposed four major dimensions to better interpret and understand learning tactics, which are frequency, continuity, sequentiality and role of learning actions within tactics. The aim of this study was to examine to what extent can a new analytic technique, the ordered network analysis (ONA), deepen the understanding of MOOC learning tactics compared to using other methods. Methods: In particular, we performed a fine-grained analysis of learning tactics detected from more than 4 million learning events in the behavioural trace data of 8788 learners who participated in a large-scale MOOC ‘ Flipped Classroom ’ . Results and Conclusions: We detected eight learning tactics, and then chose one typical tactic as an example to demonstrate how the ONA technique revealed all four dimensions and provided deeper insights into this MOOC learning tactic. Most importantly, based on the comparison with different methods such as process mining, we found that the ONA method provided a unique opportunity and novel insight into the roles of different learning actions in tactics which was neglected in the past. Takeaway: In summary, we conclude that ONA is a promising technique that can benefit the research on learning tactics, and ultimately benefit MOOC learners by strengthening the strategic support.


| INTRODUCTION
Massive open online courses (MOOCs) have made education more accessible to many learners around the world (Hew & Cheung, 2014).
Learners can sign up for a variety of MOOCs offered at no or a reasonably low cost, study at their own pace, and earn educational credentials from globally recognized universities.Since external support from instructors or peers are usually limited in MOOCs, the ability to self-regulate learning (SRL), that is, select and enact goal-oriented learning tactics that advance learning (Winne & Hadwin, 1998;Winne & Marzouk, 2019), comes to be a critical set of skills to successfully complete coursework (Kizilcec et al., 2017;Virtanen et al., 2017).Productive engagement in self-regulated learning, however, presents a challenge, in part because many learners struggle to enact effective learning tactics, that is, sequences of learning processes that a learner performs to master learning content and meet instructional expectations in a course (Azevedo et al., 2005;Bjork et al., 2013).This further hinders the already high attrition rate among MOOC learners (75%-90%) (Li & Baker, 2018;Reparaz et al., 2020).
Skilful self-regulated learners thus know which learning actions to invoke, how to compose a learning tactic from those actions and when to enact the tactic to advance their learning (Winne & Hadwin, 1998;Winne & Marzouk, 2019).Learning actions are specific learning events recorded in raw trace data (Fan et al., 2022), for example, a learner's click to play a video in MOOC is indicative of a 'Content_Access' action.Learning tactics are 'considered as learners' cognitive routines used for performing specific learning tasks' (Fan, Saint, et al., 2021, p. 1), that is, how learners engage with different learning actions.For example, a learner may engage in the following learning actions to construct knowledge from course content: navigate a MOOC environment to find a page of interest, revisit a video lecture they watched earlier on that same page, and after that, read a relevant book chapter.It can be articulated from previous research (Azevedo et al., 2010;Bannert et al., 2014;Molenaar & Järvelä, 2014;Winne, 2010;Winne & Marzouk, 2019;Winne & Perry, 2000) that each learning action measured from trace data within different tactics can be described across four major dimensions (i) frequency, that is, how many times an action has been observed within a tactic in a given learning session; (ii) continuity, that is, learners' continually uninterrupted engagement with one specific action within a tactic; (iii) sequentiality, that is, the probability of an action preceding/succeeding another action, and (iv) role, that is, the significance an action contributes to a tactic, for example, is it a primary or supportive action.These four methodological dimensions related to how to describe and understand different learning actions and what kinds of operations are validly applied to reveal different learning tactics.
Researchers have reported positive relationships between use of learning tactics and academic success (cf., Broadbent & Poon, 2015;Dent & Koenka, 2016).To promote the use of learning tactics in a MOOC environment and boost learning performance, it is hence very important to understand: (i) which learning tactics learners typically enact when learning online, and (ii) whether learners can use these tactics efficiently and appropriately.Many researchers to date have examined learning tactics in online learning environments using different methods (e.g., Broadbent & Poon, 2015;Dignath & Büttner, 2008;Fan, Matcha, et al., 2021;Fincham et al., 2019;Jansen et al., 2019;Matcha, Gaševi c, Ahmad Uzir, et al., 2019;Matcha, Gaševi c, Uzir, et al., 2019) and most often relying upon learner trace data (Bernacki, 2018).However, most studies only interpreted and explained learning tactics based on the frequency and temporal distributions of learning action.A more comprehensive insight into learning actions and learning tactics regarding above all four dimensions has yet to be gained.
Traditionally, researchers have recorded learning actions as frequency counts, which ignores the information about context, for example, learning actions that precede or follow (Aleven et al., 2010;Azevedo et al., 2010;Saint et al., 2021;Winne, 2010).This approach, therefore, can provide only a partial understanding of the learning tactics studied.Recently, researchers have begun increasingly utilizing analytic techniques that go beyond frequency-based approaches, for example, process/sequence mining, cluster and network analysis, to study learning actions and tactics (Fan, Matcha, et al., 2021;Jovanovi c et al., 2017;Matcha, Gaševi c, Uzir, et al., 2019;Saint et al., 2021;Siadaty et al., 2016).These analytical techniques, for example, process mining in Fan, Saint, et al. (2021); Saint et al. (2021) (Winne & Hadwin, 1998), and it refers to the function or functions one action plays in a specific learning tactic, that is, how the learner's engagement in such action will support his or her own learning process and serve his or her own usage of a specific tactic.For example, the 'Search' learning action can be considered a primary action in the 'Content Revisiting' tactic and a supportive action in the 'Self-Assessment' tactic, because learners may use this action for different purposes, that is, finding key terms vs finding answers in course materials.The role of a learning action can hence determine whether and to what degree educators should promote learner engagement with that action to maximize the benefits of a corresponding tactic.In other words, understanding a role of a learning action within a tactic is a critical step towards tailoring appropriate support to learners who struggle to compose and effectively use learning tactics.However, most methods (e.g., process mining) used in previous studies failed to reveal all the dimensions, including the role of a learning action, when studying learning tactics.
To this end, we explored the viability of using ordered network analysis (ONA) (Tan et al., 2022) to further our understanding of learning tactics from the aforementioned four dimensions.ONA is a technique for identifying and quantifying directed connections among elements in data by accounting for the order of events, and visualizing these connections in network models.Such models not only measure the strength of connections and illustrate the direction of connections, they also create a meaningful metric space for interpretation.In the present study, we applied ONA in investigating the directed connections among learning actions in a learning tactic.In particular, we performed a fine-grained analysis of learning tactics detected from behavioural trace data 1 of learners who participated in a large-scale MOOC 'Flipped Classroom'.The goal of this paper is to examine and demonstrate methodological advantages and affordances of the ONA analytic technique.In order to achieve this, we compared the results generated using ONA to the results generated using process mining (van der Aalst, 2016), another advanced analytic technique that researchers have previously used to study learning tactics (Saint et al., 2022).We detected MOOC tactics used by learners and choose one tactic as an example to analyse the above-mentioned four dimensions using both ONA and process mining techniques.Based on the comparison between different methods, our results confirmed that both approaches can reveal the continuity and sequentiality of learning actions as they dynamically unfold throughout learning sessions we observed.The ONA technique provided additional information on frequency, and high-level insight into the roles of learning actions, depicted as interpretable positions of nodes in the generated network graphs.The contribution of this paper is mainly in terms of methodology, and the tactic examples given in the paper are used to compare a process mining method with the ONA method.Due to its length and scope, the current paper offers relatively limited interpretations of other learning tactics and specific self-regulation processes of MOOC learners.

| RELATED WORK
2.1 | The process-based approach: process mining As an advanced alternative to conventional statistical methods (Reimann et al., 2014), process mining techniques have been applied to investigate sequential and temporal characteristics of processes captured by trace data (Saint et al., 2021;van der Aalst, 2016).For this reason, process mining has sparked the interest of SRL researchers, and they have been increasingly using the process mining techniques, for example, Heuristics Miner, Inductive Miner, Fuzzy Miner, and pMineR (Saint et al., 2021), to study learning as a temporal and sequential process.Researchers who used process mining techniques have typically created process maps separately for each learning tactic, where the process maps show interconnected learning actions, usually in a form of graph with nodes that represent learning actions and edges that represent the probability of transition between any pair of the actions observed.
The examples include studies conducted by Matcha, Gaševi c, Uzir, et al. (2019) and Fan, Saint, et al. (2021).For instance, Matcha, Gaševi c, Uzir, et al. (2019) applied a process mining technique to detect learning tactics based on the trace data generated by learners in a flipped classroom course.In particular, Matcha et al. identified tactics as sequences of learning actions, including Assessment-oriented Tactic and Diverse Assessment-oriented Tactic (Matcha, Gaševi c, Uzir, et al., 2019).The findings indicated that learners who enacted more diverse learning tactics throughout the semester outperformed their colleagues who enacted a single tactic.Researching another MOOC, Fan, Matcha, Uzir, et al. (2021) applied the process mining technique to identify four major learning tactics that learners enacted and demonstrated that learners' selection of learning tactics was related to learning opportunities imposed by the instructional design.
Not only were the process mining techniques successful in detecting diversity and temporality of learning tactics throughout semester, but also, more recently, in advancing the understanding of learning tactics.For example, Fan, Saint, et al. (2021) analysed temporal sequences of different learning actions within learning tactics and found that learners who engaged in the assessment-related tactics enacted more metacognitive evaluations throughout learning sessions than the learners who primarily utilized reading-oriented tactics.Even though the evaluation actions were detected in both tactics, process mining could hardly reveal the role of evaluation in each of these tactics, for example, the researchers may expect that learners engage in evaluation for different purposes such as evaluation of prior knowledge in a reading-oriented tactic or evaluation of learning performance in the assessment-oriented tactic.Moreover, this same action may even play a dual role in another tactic, for example, in a monitoring tactic where a learner simultaneously evaluates both prior knowledge and immediate learning performance.Additional research is needed to further analytic means that can provide a more comprehensive picture of learning tactics, including roles of learning actions that compose a tactic.We thus explored network analytics as a promising venue to this end, as network analytic techniques can allow for meaningful interpretation of the observed learning actions relative to their position in the network space, which, in turn, can cast light on the roles the action plays in a learning tactic.

|
The network-based approach: ordered network analysis (ONA) Researchers have recently begun introducing network analytic approaches to the study of SRL.For example, Shea et al. (2013) analysed online learner self-regulation using social network analysis (SNA) and quantitative content analysis.Li et al. (2020) examined the temporal dynamics of SRL behaviours in STEM learning by conceptualizing learner interactions in network models.There are also some studies that combined process mining and epistemic network analysis (ENA), that is, a fundamental method that ONA is built upon, as a complementary method to analyse SRL processes.For example, Saint et al. (2020) combined ENA and process mining to examine the sequential and temporal nature of SRL behaviours and identified behaviours that differentiate between learners across performance levels.Melzner et al. (2019) combined ENA and process mining to analyse how learners regulate collaborative learning activities when faced with motivational or comprehension related problems and found that ENA and process mining, when applied jointly, can provide a richer description of collaborative learning activities than using a single method only.
However, even though the combination of process-and networkbased approaches advanced the understanding of SRL, the lack of directional information between learning actions in ENA models prevented researchers from gaining deeper insights into the associations between actions, which is, in turn, critical information for understanding learning actions observed in the learner data and further for tailoring appropriate support to MOOC learners.Moreover, Saint et al. (2020) and Melzner et al. (2019) noted that more comprehensive information about SRL would be revealed in their studies if the directions between pair-wise actions were identified in ENA.
As an extension of ENA, researchers have introduced the ONA technique to account for the direction of associations between components of a studied phenomenon (Tan et al., 2022).The ONA technique thus captures connections between elements and represents both the strength and direction of those connections statistically and visually (Tan et al., 2022).As the network space formed in this way can meaningfully represent the directed connections between learning actions, in the present study, we explored the viability of using ONA to deepen understanding of learning tactics.To this end, we separately performed the ONA and process mining analyses on the same learning tactic that encompasses nine learning actions and compared the results generated by these two methods.One central research question guided our study: To what extent can the use of ONA technique deepen understanding of learning tactics and actions enacted by learners in a MOOC, compared to process mining?

| Study context
The study was conducted in the context of a MOOC named 'Flipped Classroom'.Most of the participants were teachers, 80% (in-service) and 8% (pre-service).Of the participants, around 60% of them were female learners.The average age of the participants was 36 years (SD = 9 years).In this course, the teachers learned about the flipped classroom pedagogy, including the application of this pedagogical concept in their teaching practice.Each MOOC offering was 7 weeks long, covering one unit per week.At the beginning of each week, the teaching team released the unit learning resources and the learners were required to spend 3-5 h watching videos, participate in discussion, browse reading materials, finish quizzes, and conduct peer reviews throughout the week.The scores learners earned on all quiz tasks were accounted for 25%, the peer review scores accounted for 35%, the forum participation accounted for 20%, and the final exam (administered in week 7) accounted for 20% of the final grade in the course.Overall, the completion rate of this course was 6.48%.
From 2016 to 2018, 97,475 learners were enrolled across 12 offerings of this MOOC.Only about half of these learners logged in to the course and participated in different learning activities.Many learners dropped out of this MOOC only after few logins or a very short stay.We included in our analysis only those learners who were active in the course for more than 3 weeks.As a result, we obtained a sample of 8,788 learners who produced more than 4 million data points from traced interactions recorded in the MOOC platform.

| Analytic approach to detect learning tactics
To detect learning tactics from this big dataset, we followed the analytic approach shown in Figure 1, which was also used in (Fan, Matcha, et al., 2021;Fan, Saint, et al., 2021;Matcha, Gaševi c, Ahmad Uzir, et al., 2019;Matcha, Gaševi c, Uzir, et al., 2019).We first defined learning sessions during which a learning tactic would be observed, and then developed an action library to translate raw trace data into nine meaningful learning actions.Based on the sequences of actions identified in each learning session, we used a clustering method to group those sessions and map them to corresponding learning tactics.

| Learning sessions
In previous studies, the 'unreasonably long dwell times between two events' were commonly used as indicators or markers to segmenting learning events in trace data into 'learning sessions' (Gasevic et al., 2017;Kovanovic et al., 2015).In this study, we used 45 minutes as the 'unreasonably long dwell times' to segment learning sessions, which means any action with a duration equal to or longer than 45 min marked the end of a learning session.A more detailed rationale of this threshold to segment sessions is provided by Fan, Matcha, Uzir, et al. (2021).Based on this segmenting approach, we divided 4,664,214 unique learning events from the log data into 201,038 learning sessions.

| Learning actions
To model different learning processes, researchers have utilized raw trace data that learners generate as they interact with digital course resources, for example (Davis et al., 2016;Kizilcec et al., 2017;Maldonado-Mahauad et al., 2018;Sinha et al., 2014).We harnessed trace data collected in this study to model/detect learning actions performed in the MOOC learning environment.To this end, we first defined and labelled nine learning actions, as shown in Table 1, which is similar as we did in Fan, Matcha, Uzir, et al. (2021).It is worth noting that each approach to operationally defining data (such as our action library) has its own constraints and affordances, which can influence what the analytic method can and cannot reveal.We also discuss this issue in Section 5.

| Learning tactics
In order to detect learning tactics, we first generated first-order Markov model (FOMM) of actions for all learning sessions using the pMi-neR R package (Gatta et al., 2017).In this way, we obtained a transition matrix for every session with transition probabilities between any pair of learning actions (Figure 1).Then, we used the expectation-maximization (EM) algorithm to cluster all 201,038 sessions based on the transition matrix generated by FOMM.Here, we used the gap statistic method (Tibshirani et al., 2001) to estimate the optimal number of clusters.This clustering approach has been found useful for detecting learning tactics in several previous studies (Ahmad Uzir et al., 2019;Fan, Matcha, et al., 2021;Fan, Saint, et al., 2021;Matcha, Gaševi c, Ahmad Uzir, et al., 2019;Matcha, Gaševi c, Uzir, et al., 2019).Last, we used the exploratory sequence analysis implemented in the TraMineR R package to examine the distribution of learning actions in detected tactics, that is, frequency and temporal distribution of actions within learning sessions (Gabadinho et al., 2011)

| ONA technique
We conducted the ONA analysis using the ONA R package.To model the directed connections among learning actions, we used the nine types of learning actions (Table 1) as codes and binary-coded their The size of each node is proportional to the frequency of that learning action that occurs in a learning tactic, specifically when its occurrence is subsequent to other actions (i.e., representing the frequency dimension), with larger nodes indicate higher frequency.The coloured circle within each node is proportional to self-transition.
That is, in addition to transitioning to other learning actions, this learning action makes transitions to itself (i.e., representing the continuity dimension).In other words, this learning action's preceding action is itself.The larger the coloured circle is, the more selftransited that learning action is.For example, in Figure 2 When the connection strengths are identical between two codes, the chevron will appear on both edges.Lastly, in a ONA network, node positioning is a distinguishing feature that explains the characteristics of this network relative to other networks.For example, in

| RESULTS
In this section, we show the results of the process mining and ONA analyses aimed at deepening understanding of learning actions and tactics enacted by learners in a MOOC.

| Learning tactics identified
We identified eight optimal clusters in our data set, as per the gap statistic analysis (Tibshirani et al., 2001).Each cluster encompasses similar patterns of learning actions which are considered as a learning tactic in our study.In Table 2, we provide the brief descriptions and tactic names of the eight clusters.In the present study, we chose the Assessment Content Forum and Search Tactic (hereinafter abbreviated to ACFS-Tactic) as an example to explain this tactic in more detail based on the frequency, continuity, sequentiality and role of different learning actions encompassed within this tactic.The reason why we choose this tactic is that it is more complicated than many of other tactics (such as Focus On Content Tactic and Focus On Assessment Tactic), and it occupies a larger proportion (7.37%) of all learning sessions than several other tactics (such as Assessment Content Forum Tactic which is 3.22%).Here, we first used the two distributions which were also used in most previous studies (Fan, Matcha, et al., 2021;Jovanovi c et al., 2017;Matcha, Gaševi c, Uzir, et al., 2019;Saint et al., 2021) to understand and interpret tactics: the frequency distribution and temporal distribution plots.
As the frequency distribution of this tactic shows (see Figure 3left), Assessment, content related actions (Content_Access and Conten-t_Revision), Overview, Forum and Search actions were all prominent actions in the ACFS-Tactic.On the temporal distribution plots of ACFS-Tactic (see Figure 3-right), the i-th bar represents the probability of each learning action learners engaged with as their i-th action of sessions when using the ACFS-Tactic.For example, the first bar shows for each action type the probability that it would be the type of the first action in the sessions of a particular tactic 3 .Upon looking at temporal distribution of these actions, we noted the overall pattern that learners started their learning sessions from Overview and Assessment and went to Content, Search and Forum in this tactic (see the temporal distribution in Figure 3).Therefore, we named this tactic as Assessment Content Forum and Search Tactic due to its prominent actions and overall temporal distribution.The more detailed descriptions for all eight clusters can be found in the supplemental document.
The above two distribution plots, which were mainly used in previous studies (Fan, Matcha, et al., 2021;Jovanovi c et al., 2017;Matcha, Gaševi c, Uzir, et al., 2019;Saint et al., 2021), can only reveal two dimensions of learning tactics: the frequency and sequentiality of learning actions.They failed to further explain and understand learning tactics from the dimensions of the continuity and the role of learning actions.Therefore, in the following subsections, we continue to use ACFS-Tactic as an example to demonstrate and compare the process mining and ONA analytic approaches towards understanding and explaining learning tactics.

5.65%
Learning with Search were central to this learning tactic.
3 It is worth noting that the x-axis of the temporal distribution figure represents the length of action sequences, and the i-th bar on x-axis was generated based on the distribution of the ith actions from learning sessions that length equal to or longer than i.The number of sessions available for analysing actions decreases proportional to the session length.For example, the 5th bar was generated based on the distribution of the 5th action from all sessions that contained at least 5 learning actions; and the 15th bar was generated based on the distribution of the 15th action from all sessions that contained at least 15 learning actions; and the sample size of longer sessions is much smaller than the shorter sessions.
Therefore, when interpreting Figure 3, it should be noted that the sample size of each bar is different.

| Analysing learning tactics with process mining technique
Firstly, the FOMM of the ACFS-Tactic highlighted the continuity of actions via self-looping transition probabilities in process mining (Figure 4).For example, Figure 4 revealed that learners who used the ACFS-Tactic tended to continuously engage in Forum and Search actions.
Secondly, the process mining technique successfully revealed the sequentiality characteristic of actions.The process mining technique represents the sequences as the actions (nodes) interconnected with the edges.In this way, the sequential information provides additional insight into the action.For example, the analysis so far demonstrated that Forum was not only among the most frequently (accounted 17.24% in the frequency distribution) and most continuously (with a 79% self-looping transition probability) enacted learning actions, but it was also one of the most prominent actions overall in the ACFS-Tactic, because it was connected to many other nodes (see edges in

| Analysing learning tactics with ONA technique
To triangulate with above findings, and more importantly, to gain a deeper insight into the learning tactic, we created and analysed the ONA graph (Figure 5) of learning actions that comprise the ACFS-Tactic.
In Figure 5, node size depicts the frequency of a learning action act as a response to other actions in the ACFS-Tactic.The more frequent that action was subsequent to other actions, the larger the corresponding node is.For example, Assessment, Content_Revisit, Overview, Forum and Search actions occurred as a response to other actions relatively more frequently, as we also showed in Figure 3.This means that learners were relatively more engaged with these learning actions than others.Further, the ONA technique highlighted the continuity of actions via the coloured circles within each node in ONA (Figure 5).For example, the coloured circles in Forum and Search actions were proportionally larger than other actions, meaning that learners who used this tactic tended to continuously engage in Forum and Search actions, which is consistent with the findings generated by the process mining technique.Another key dimension towards a deeper understanding of a learning tactic is to examine the sequentiality of learning actions that comprise the tactic and, again, the ONA technique successfully revealed the sequential characteristics of learning actions.For instance, as shown in Figure 5, the chevron pointing from Overview and towards Assessment is placed on an edge with the darkest saturation, representing the directed connections with the greatest strength from Overview to Assessment.This means that relative to other learning actions, the order of browsing general course information (Overview) before working on quizzes and homework (Assessment) was the most frequently used order when learners used ACFS-Tactic.
Importantly, the network space with meaningful node positions created using the ONA technique provides researchers with an additional layer of information compared to the results generated by the process mining technique.In ONA, instead of placing nodes by prioritizing aesthetics criteria such as avoiding edge-crossing as process mining technique usually does, ONA determines its nodes placement by accounting for connection weights across the network.Therefore, the position of nodes can be used to interpret the dimensions of the

| Summary of findings
Researchers studying SRL have been increasingly interested in applying analytic methods that can capture sequential and temporal characteristics of learning actions and tactics (Fan, Matcha, et al., 2021;Fan, Saint, et al., 2021;Jovanovi c et al., 2017;Matcha, Gaševi c, Uzir, et al., 2019;Saint et al., 2021;Siadaty et al., 2016).This line of F I G U R E 5 Using the ONA technique to understand and interpret the ACFS-Tactic research has resulted in improved understanding of learning actions, not only in terms of their frequency, but also in terms of their continuity and sequentiality, another two dimensions that characterize SRL processes (Winne & Hadwin, 1998).However, learning tactics are typically complex structures, containing multiple interconnected learning actions that may play different roles relative to instructional context.
The information about frequency, continual and sequential trends of actions is often not enough to provide researchers with insights into the role of an action within a tactic.As the understanding of the role is important to support tactics use (e.g., by promoting primary actions over the less prominent ones) and thus help MOOC learners engage in productive SRL, we investigated whether the network-based analytic technique ONA can reveal the role of a learning action, motivated by the capability of this approach to create a network of actions that can be meaningfully interpreted relative to each other.We examined the Assessment Content Forum and Search (ACFS) tactic using ONA and process mining techniques.Following, we summarize and discuss our findings (see Table 3 for an overview).
The frequency of learning actions can easily be revealed through the frequency distribution method (proportions) and the ONA method (the meaningful node size), but not revealed in the process mining results as the node size in FOMMs is meaningless.We demonstrated that, using either process mining or ONA, researchers can obtain information about continuity and sequentiality of learning tactics.
Continuity is represented via self-loops in process mining and coloured circles within nodes in ONA.Sequentiality is depicted using the edges with single directions in process mining and the edges with bilateral directions in ONA.
By meaningfully positioning learning actions in a network space (Figure 5), for example, grouping studying related actions on the left and assessment related actions on the right side of the network space, the ONA technique also revealed information about the role of an action.Specifically, we found that Content_Access and Assessment actions played primary roles in the ACFS tactic, an example learning tactic we opted to investigate in this study.

| Research and instructional implications
Following, we discuss research and instructional implications of our findings.
Researchers who study learning tactics unpacked using the ONA technique can gain a deeper insight into a theorized learning processes that interplay within and across learning tactics and, in that way, improve their understanding of how self-regulated learners enact and monitor learning tactics.For instance, being able to observe a fine-grained structure of learning tactics and processes as they change over a semester in the context of evolving task requirements, researchers may identify SRL behaviours that distinguish between more and less productive self-regulated learners at multiple points in a semester, for example, between those who participate in a MOOC forum mainly to understand the assessment requirements and those who strategically utilize forums for multiple purposes following the course dynamics.These analyses may further allow for confirming or challenging different theoretical propositions about SRL, for example, use of learning tactics is determined by task requirements and instructional goals (Fan, Saint, et al., 2021); prior metacognitive and domain knowledge affect how a tactic has been composed and engaged (Taub et al., 2014;Trevors et al., 2014); external feedback a learner receives (often in a form of a score or grade) will determine whether a learner will continue to enact the same learning tactics or they will adjust learning tactics accordingly in the remainder of the course or in subsequent, similar courses (Binbasaran Tuysuzoglu & Greene, 2015; Butler & Winne, 1995).
From a practical perspective, the analytics on learning tactics use in a form of ONA graphs has the potential to provide learners with a detailed overview of their learning engagement over the selected learning period and, combined with the information about learning performance and course requirements, may prompt learners to evaluate whether the way they have studied in a MOOC was beneficial to their learning success.For instance, upon looking at the analytics about their engagement in the Assessment Content

| Limitations and future works
Following, we note the potential limitations of our study and recommend steps for future research.
In this study, ONA modelled the directed connections among learning actions by accounting for the order and interdependence of events.We used a stanza window size of two to accumulate the connections between each pair of adjacent learning actions, which treated learning tactics as a step-by-step process.Such approach allowed us to investigate the close relationship between learning actions that tend to happen right before or after each other.Given the flexibility of stanza window size in ONA to 'capturing recent temporal context' (Siebert-Evenstone et al., 2017, p.126), future work could explore using a different stanza window size to model connections between learning actions in a broader temporal proximity.
We also note that even though the ONA technique generated more comprehensive information about learning actions observed in this study compared to process mining, this should not necessarily mean that process mining techniques cannot contribute to better understanding of learning tactics.For instance, the process mining technique based upon the transition probability that we applied in this study is only one among many process mining techniques.The pMi-neR method has its own limitations, such as (i) focuses only on the probability matrix and ignores the frequency of actions; (ii) assumes that the current action is only affected by the previous action; and (iii) unable to calculate and reveal the time intervals of transitions of actions.There are other process mining algorithms (e.g., Fuzzy Miner) (Saint et al., 2021) that researchers can apply and explore their benefits in studying learning tactics in the future.As well, studying SRL processes by combining different techniques (Ahmad Uzir et al., 2020;Saint et al., 2020), instead of applying methods individually, may be another way of improving the validity of SRL measurements.For instance, combining techniques can be used to enable much more fine-grained analysis to discover low-frequency transitions between actions that are still important and theoretically meaningful.
From a data processing point of view, our action library only operationally defined learning actions at a relatively coarse-grained level and lacks theoretical explanatory power, which limited what the ONA method can reveal.For example, the Help_Seeking action can be further unpacked into (i) learners seeking information about how to use the MOOC platform, or (ii) learners seeking information to understand concepts in the course.These more fine-grained learning actions will enable methods such as ONA to further reveal the nature of learning tactics.Another level of data interpretation, such as a pattern library or process library will overcome the limitation in terms of limited theoretical explainability (Fan, Saint, et al., 2021;Saint et al., 2021;Siadaty et al., 2016).For example, extracting SRL processes based on action patterns can enable methods such as ONA to model and visualize learners' regulation processes when using different learning tactics.
Last, we acknowledge that, although we analysed a quite extensive sample of student data collected over multiple offerings of the 'Flipped Classroom' MOOC, validating and generalizing our results across different MOOCs remains an important step for future research.

| CONCLUSION
In general, our results show that both basic method and process mining technique failed to address all the dimensions we defined, but the ONA technique successfully revealed all four dimensions (frequency, continuity, sequentiality, and role) which provided deeper insights into the learning actions of learning tactics.The ONA technique provided a unique opportunity and novel insight into the roles of different learning actions in tactics, which also corresponds to different MOOC course modules or resources.Our findings related to MOOC learning tactics also provide practical implications for instructors and designers to better support learners' productive engagement in self-regulated learning.
or process mining and network analysis in Ahmad Uzir et al. (2020); Matcha, Gaševi c, Uzir, et al., 2019; Saint et al. (2020), have advanced research on learning tactics use, in particular deepening the understanding of temporal and sequential relations among learning actions that compose a tactic.It, however, remains less clear whether and to what degree the role of a learning action differs across learning tactics, as the same learning action may serve different purposes across learning tactics.The role of action relates to learners' operation phases of self-regulated learning based on the COPES model

3. 3 |
Understand learning tactics from trace data3.3.1 | Process mining techniqueAfter obtaining the learning tactics, we applied the same pMineR R package to build process maps of action for different tactics.It is worth noting that, there are many different algorithms and visualization methods for process mining(Saint et al., 2021).Here, we decided to use first order Markov models (FOMMs) and the pMineR package because it provides better insights into learning tactics than others such as Inductive Miner and Heuristics Miner(Saint et al., 2021) and it was also the most frequently used algorithm in previous studies in learning tactics(Fan, Matcha, et al., 2021;Matcha et al., 2020;Matcha, Gaševi c, Ahmad Uzir, et al., 2019; Matcha, Gaševi c, Uzir,   et al., 2019;Saint et al., 2020;Saint et al., 2021).The Inductive Miner and Heuristics Miner are more suitable to be used to seek process model soundness in a more structured set of learning paths, and the process models they generated for exploratory SRL models proved to be difficult to interpret(Saint et al., 2021).The pMineR could model and visualize the temporally ordered sequences of learning actions in learning sessions, which is proven to be suitable in similar research contexts and the results are easy to interpret(Fan, Matcha, et al., 2021;Matcha et al., 2020;Matcha, Gaševi c, Ahmad Uzir, et al., 2019; Matcha, Gaševi c, Uzir, et al., 2019;Saint et al., 2020;Saint et al., 2021).Like other methods, pMineR has its own limitations, which are also discussed in section 5.This process mining method allowed us to reveal the relationship between nine actions in the form of transition probabilities.As shown in Figure2-left, which is a process map example, the nodes represent the actions and the edges indicate the transitions between the nodes with different transition probabilities (numbers on edges).We used 5% as a threshold which means edges with transition probabilities below 5% were not shown in the process maps for tactics 2 .For example, in Figure2-left, the nodes A and B represent learning actions in one tactic and the transition probability from A to B was 22%.However, the edge from B to A was not shown in Figure2-left because the corresponding transition probability was below 5%.
presence and absence in the identified learning tactics.The nine types of learning actions were represented as nodes in the resulted ONA network graphs.With the binary-coded learning actions, we set a moving stanza window of two for the ONA algorithm to identify and accumulate the directed connections formed between each pair of current action and its preceding action.Such directed connections were represented as edges in ONA graphs.Because ONA uses an optimisation routine to determine node position(Tan et al., 2022), the resulted ONA metric space can be interpreted based on the location of nodes.To demonstrate how to interpret the nodes, edges, and the metric space in ONA network graphs within the aforementioned fourdimension framework (i.e., frequency, continuity, sequentiality, and role), we use Figure 2-right as an example.
Figure2-right, the triangle with a chevron pointing from A towards B represents the frequency of B as A's subsequent action.In other words, the frequency that learning action B act as a response to A. Given that the triangle pointing from A to B is thicker and more saturated than the other way around, it is more frequent that B follows A (i.e., representing sequentiality dimension).Between any pair of nodes, if there is a bidirectional connection, the chevron only appears on the side with stronger connections.This helps viewers differentiate heavier edges in cases such as between node B and C, where the connection strengths from both directions are similar.

F
Figure 2-right, node B and C are located closer in the network space compared to their distance with node A, meaning that the qualitative meaning that learning action B and C carry similarly explains the feature of this network.Therefore, by comparing the node position of learning actions within a ONA network besides the connections between them, we can make sense of the role of each learning action in defining various learning tactics.

Figure 4 )
Figure4).For example, this process map of learning actions revealed that learners who used this tactic may browse the Forum before and after engagement with the Assessment and Overview, and Forum action are sequentially related to Help_Seeking action.

F
I G U R E 3 Frequency (left) and temporal (right) distribution of actions in ACFS-Tactic F I G U R E 4 Using the process mining technique to understand and interpret the ACFS-Tactic More importantly, the process mining technique could hardly reveal the role of different learning actions in this tactic.
Figure5, relatively strong connections were primarily made by nodes that are located in the middle and the right side of the network.In contrast, nodes that are located on the left side of the space about studying new content made relatively weak connections overall.This indicates that the characteristics of ACFS-Tactic are less about studying new content but more about engaging in activities and assessments.Referring back to Forum as an example, the corresponding information from the ONA graph indicates that this learning action was frequent (node size), used over prolonged time intervals (coloured circle), often co-occurred with other learning actions (directed connections with other nodes), and used for different purposes, that is, roles, (node position).Learners, therefore, mainly went to discussion forums to support their studying (e.g., to find answers about course content) and assessment activities (e.g., to get clarification about course and exam requirements).
Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12735 by Edinburgh University, Wiley Online Library on [14/02/2023].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License (Table 1).F I G U R E 1 The analytical approach to detect learning tactics T A B L E 1 Learning actions and definitionsLabelAction definition 1-Content_Access A learner for the first time interacts with learning materials that include videos, documents, pdf, and non-score quizzes T A B L E 3 Understanding learning tactics using basic, process mining and ONA techniques Forum and Search Tactic during semester, a lowachieving learner may notice that this engagement was insufficient in the weeks before course exams.As well, this learner may also notice that they underutilized the 'Help_Seeking' processes to support Content Revist, that is, the role of the 'Help_Seeking' action in this tactic was primarily to support assessment, reflected by the position of the 'Help_Seeking' node which was located close to the 'Assessment', but far from the 'Content Revisit' node.The effects of SRL interventions based on ONA on learners' engagement in metacognitive monitoring and control of tactics use remains an important topic for future research.