The full text of this article hosted at iucr.org is unavailable due to technical difficulties.

Original Article
Free Access

Exploring Flipped Classroom Effects on Second Language Learners’ Cognitive Processing

First published: 08 May 2017
Cited by: 1

The first three authors contributed equally to this work. The first and fourth authors are the corresponding authors.

Jeong‐eun Kim (PhD, Georgetown University) is Assistant Professor, Department of English Education, Hongik University, Seoul, Korea.

Hyunjin Park (PhD, Korea University) is Invited Professor, Institute for General Education, Korea University, Seoul, Korea.

Mijung Jang (PhD, Korea University) is Invited Professor, Institute for General Education, Korea University, Seoul, Korea.

Hosung Nam (PhD, Yale University) is Associate Professor, Department of English Language and Literature, Korea University, Seoul, Korea, and Senior Scientist, Haskins Laboratories, New Haven, CT.

Abstract

This study investigated the cognitive effects of the flipped classroom approach in a content‐based instructional context by comparing second language learners’ discourse in flipped vs. traditional classrooms in terms of (1) participation rate, (2) content of comments, (3) reasoning skills, and (4) interactional patterns. Learners in two intact classes participated and were taught in either a flipped classroom (n= 26) or a traditional classroom (n = 25). In the flipped class, the learners listened to an online lecture before class and participated in a small‐group discussion in class. In contrast, the learners in the traditional class listened to a teacher‐led lecture in class and then immediately participated in a small‐group discussion in class. The learners’ discussions were audiorecorded. Quantitative and qualitative analyses indicated no difference in participation rates; however, the students in the flipped classroom produced more cognitive comments involving deeper information processing and higher‐order reasoning skills and showed more cohesive interactional patterns than did the students in the traditional classroom. These results indicate that flipped classrooms can effectively promote higher‐order thinking processes and in‐depth, cohesive discussion in the content‐based second language.

Introduction

Besides using a second language (L2) proficiently, international students who enroll in postsecondary programs of study are expected to skillfully process information, analyze and evaluate it, judge its validity and value, and use it to solve problems. However, it has been reported that L2 speakers often find this to be challenging (Jones, 2005) because processing L2 content knowledge in the L2 consumes more cognitive capacity than processing content in a first language (L1), leaving less room for skillful, higher‐order thinking. In contrast to traditional classrooms in which new concepts are learned in class and practiced as homework, flipped classrooms utilize a form of blended learning that offers learners private time and space to learn content and organize their thoughts prior to class. For L2 learners, having time to acquire content knowledge at their own pace outside of class may free up cognitive capacity during class, enabling them to engage in more skillful, concise, and cohesive thinking and may thus facilitate participation in complex in‐class activities. The cognitive benefits, then, of a flipped instructional model could be expected to affect how well L2 learners process information, understand content, and organize their thoughts in their L2.

The purpose of this study was to examine the pedagogical effectiveness of flipped classrooms for L2 learners, specifically in a content‐based classroom where content knowledge was solely taught in the L2 (Korean). In order to investigate the benefits of flipped classrooms over traditional classrooms for Korean L2 learners’ cognition and thinking processes, the study compared participation rates, the content of learners’ comments, the types of reasoning skills they displayed, and interactional patterns across flipped and traditional class formats.

Literature Review

The Flipped Classroom: Definitions, Theoretical Relevance, and Previous Findings

The flipped classroom is a pedagogical strategy that reverses the traditional classroom process by delivering the instructional content usually, but not always, online before class and then engaging learners in interactive group learning and/or critical problem‐solving activities that are carried out under the teacher's guidance during class (Bergmann & Sams, 2012; Bishop & Verleger, 2013; Herreid & Schiller, 2013). Recently, the development of educational technology has allowed flipped classrooms to be easily adopted in K–12 and higher education contexts (Hamdan, McKnight, McKnight, & Arfstrom, 2013).

This learning environment can be characterized as student‐centered—students are expected to come to class having already gained the knowledge necessary to actively engage in problem‐solving activities with their peers. Throughout the cycle of instruction, they maintain an active role at the center of learning. The practice is based on the assumptions that meaningful interaction among peers encourages knowledge building and that teachers can provide more timely and personalized guidance and feedback during in‐class activities. The pedagogical relevance of the flipped classroom is supported by a range of student‐centered learning theories in the field of educational psychology (Bishop & Verleger, 2013), including cooperative learning (Johnson, 1984; Ormrod, 1995; Rottier & Ogan, 1991; Sharan, 1990; Slavin, 1991), collaborative learning (Goodsell, Maher, Tinto, Smith, & MacGregor, 1992), peer tutoring (Gartner, Kohler, & Reissman, 1971; Tabacek, McLaughlin, & Howard, 1994), peer‐assisted learning (Topping & Ehly, 1998), problem‐based learning (Barrows, 1996), and active learning (Michael, 2006; Prince, 2004). Similar to Vygotsky's (1978) sociocultural theory, which describes humans as active participants in their own learning and social interaction as crucial in the process of learning, these theories commonly view learning as the outcome of joint cognitive activity in which students take an active part (Foot & Howe, 1998). Previous meta‐analytic studies (e.g., Johnson, Johnson, & Smith, 1998; Springer, Stanne, & Donovan, 1999) have reported that the collaborative, cooperative nature of student‐centered learning makes it more conducive to academic achievement, self‐esteem, retention in academic programs, and the perception of greater social support than traditional, teacher‐centered learning with its individualistic and competitive nature.

Specific claims for the pedagogical benefits of the flipped classroom include: (1) students and instructors’ positive perceptions of the active learning environment (e.g., Butt, 2014; Gilboy, Heinerichs, & Pazzaglia, 2015; Herreid & Schiller, 2013; Lage, Platt, & Tregalia, 2000; Love, Hodge, Grandgenett, & Swift, 2014), (2) more active engagement during class (e.g., Deslauriers, Schelew, & Wieman, 2011; McLaughlin et al., 2014), and (3) superior achievement on formative/summative assessments (e.g., Amresh, Carberry, & Femiani, 2013; Day & Foley, 2006; Pierce & Fox, 2012). For example, when Love et al. (2014) implemented a flipped linear algebra course, the majority of students reported that the approach helped them develop a deeper understanding of the content. Amresh et al. (2013) reported that students in a flipped introductory computer programming course performed significantly better than did students in a comparable traditional course design on assignments and exams. McLaughlin et al. (2014) observed a significant increase in students’ perception of their own active participation and engagement in a flipped basic pharmaceuticals class. In Herreid and Schiller's (2013) study, science/technology/engineering/mathematics instructors reported perceiving that flipped classrooms allowed students to spend more time on authentic research and work with scientific equipment, and it led to more active student involvement and enjoyment of the learning process.

In a recent study that explored students’ perceptions of a flipped classroom in a language acquisition context, Sung (2015) offered a college‐level content‐based course in an English as a foreign language program in Korea. The students completed formal and informal course evaluations and reflective learning logs. Initially, the students reported that they found it hard to adjust to the format of the class but generally came to appreciate it for three main reasons: (1) immediate and sufficient feedback, (2) ample opportunities to discuss topics in depth, and (3) time to develop their critical thinking about topics before class. These findings suggest that flipped classrooms may also be effective in content‐based L2 classrooms.

As this brief review suggests, research has found evidence of the positive effects of flipped classrooms. Yet very few studies have investigated either quantitative or qualitative differences in students’ engagement and learning processes during flipped vs. traditional class time. Moreover, the existing studies, in both L1 and L2 contexts, have been mostly based on survey questionnaires, self‐reports, focus group interviews, and students’ test performance after class. The data have usually been nonconcurrent, being collected after rather than during class time, and thus represent the outcomes of learning processes. One criticism of the use of nonconcurrent data is that they do not capture individual students’ cognitive effort, depth of information processing, or type of cognitive processing (Leow, Grey, Marijuan, & Moorman, 2014). In other words, such research has not observed the processes of learning or knowledge construction. Thus, the previous research has not provided process‐based evidence for how and why flipped classrooms may lead to better outcomes than traditional classrooms. In contrast, this study employed a concurrent data collection method in order to directly examine whether flipped classrooms and traditional classrooms support different learning processes that may in turn result in differential construction of knowledge.

Evaluating Learning Processes and Knowledge Construction

In the field of education, content analysis of students’ output, recorded during class, is employed as a method to gain insight into learners’ cognitive processes and knowledge construction (Ozcinar & Ozturk, 2013, p. 231). It is particularly common in research on computer‐supported collaborative learning. Previous research in a number of theoretical domains has employed content analysis, including cognitive learning (Henri, 1992), cognitive and constructive learning (Newman, Webb, & Cochrane, 1995), social constructivism (Järvelä & Häkkinen, 2002; Lockhorst, Admiraal, Pilot, & Veen, 2003; Pena‐Shaff & Nicholls, 2004; Weinberger & Fischer, 2005), community of inquiry (Anderson, Rourke, Garrison, & Archer, 2001; Garrison, Anderson, & Archer, 2001; Rourke, Anderson, Garrison, & Archer, 1999), and social network theory (Fahy, Crawford, & Ally, 2001). These studies examined the quality of learners’ interaction with quantitative measures, identifying information that is not captured on the level of transcripts and endeavoring to gain an in‐depth understanding of cooperative learning and collective learning processes.

Henri's (1992) seminal work using content analysis viewed interactivity as a source of learning and claimed that in‐depth analysis of message content is crucial to understanding learning processes. As outlined in Table 1, her analytical framework for content analysis consisted of five key dimensions: (1) participative (e.g., number of messages), (2) social (e.g., statements unrelated to formal content), (3) interactive (e.g., chains of connected messages), (4) cognitive (e.g., statements exhibiting knowledge and reasoning skills), and (5) metacognitive (e.g., statements showing awareness). Unlike a simple quantification of the number of messages produced by participants, this framework allowed for the multidimensional observation of messages and an investigation of the dynamics of the collective learning processes.

Table 1. Analytical Framework for Content Analysis (adapted from Henri, 1992, p. 125)
Dimensions Definitions Indicators
Participative Compilation of the number of messages or statements transmitted by one person or group Number of messages; Number of statements
Social Statement or part of statement not related to formal content of subject matter Self‐introduction; Verbal support “I'm feeling great…”
Interactive Chain of connected messages “In response to Celine…”; “As we said earlier…”
Cognitive Statement exhibiting knowledge and skills related to the learning process Asking questions; Making inferences; Formulating; hypotheses
Metacognitive Statement related to general knowledge and skills and showing awareness, self‐control, and self‐regulation of learning “I understand…”; “I wonder…”

Henri (1992) further divided the cognitive dimension into five categories according to reasoning skill: (1) elementary clarification, (2) in‐depth clarification, (3) inference, (4) judgment, and (5) strategies (Table 2). These five categories were based on Ennis's (1986) 14 aptitudes and 12 cognitive skills related to critical reasoning activities. This multilevel analysis captured the cognitive skills that are needed to acquire knowledge and the cognitive potential of participants. Henri also proposed that cognitive messages can be reclassified based on the depth of information processing: Surface‐level processing refers to “repeating the information contained in the statement of the problem without making inferences or offering an interpretation,” whereas in‐depth processing refers to “linking facts, ideas, and notions in order to interpret, infer, and judge” (p. 130). According to Ng and Murphy (2005), elementary clarification is related to surface‐level processing, and the other skills (i.e., in‐depth clarification, judgment, inference, strategies) have to do with in‐depth processing.

Table 2. Analytical Model for the Cognitive Dimension (adapted from Henri, 1992, p. 129)
Reasoning Skills Definitions Indicators
Elementary clarification Observing or studying a problem, identifying its elements, and observing their linkages in order to come to a basic understanding •Identifying relevant elements •Reformulating the problem•Asking a relevant question •Identifying previously stated hypotheses •Simply describing the subject matter
In‐depth clarification Analyzing and understanding a problem to come to an understanding that sheds light on the values, beliefs, and assumptions that underlie the statement of the problem •Defining the terms •Identifying assumptions •Establishing referential criteria •Seeking out specialized information
Inference Induction and deduction, admitting or proposing an idea on the basis of its link with propositions already admitted as true •Drawing conclusions •Making generalizations •Formulating a proposition that proceeds from previous statements
Judgment Making decisions, statements, appreciations, evaluations, and criticisms Sizing up •Judging the relevance of solutions •Making value judgments •Judging inferences
Strategies Proposing coordinated actions for the application of a solution, or for following through on a choice or a decision •Deciding on the action to be taken •Proposing one or more solutions •Interacting with those concerned

Henri's (1992) model has been utilized in many education studies examining the construction of interactivity in online discussion (e.g., Hara, Bonk, & Angeli, 2000; McKenzie & Murphy, 2000; Ng & Murphy, 2005). For example, Hara et al. (2000) analyzed electronic conferencing discourse from four classes over 10 weeks. They found that students’ social messages decreased as they became less formal with each other toward the end of the semester. The researchers also observed that the types of cognitive comments varied depending on the number and the type of discussion questions. When the online conference was highly interactive, the participants showed reasoning skills related to judgment more frequently than the other skills. Analyzing the depth of information processing, the researchers found that more student comments involved in‐depth than surface processing. The researchers concluded that the online conference format gave students “time to reflect on course content and make in‐depth cognitive and social contributions” (Hara et al., 2000, p. 140).

McKenzie and Murphy (2000) also used Henri's (1992) framework in their study evaluating the effectiveness of online discussion as part of a learning environment. Over 11 weeks, they observed higher participation rates by the students than by the staff, a higher frequency of interactive than independent statements, and a higher occurrence of in‐depth information processing (i.e., in‐depth clarification, inference, judgment, strategies) than surface processing (i.e., elementary clarification). The researchers concluded that online discussion groups can be an integral part of a learning environment by providing a medium for lively interaction and promoting associated learning processes.

Because Henri's (1992) multidimensional analytical framework is based on cognitivist approaches, it analyzes individual messages in isolation. It therefore “gives no impression of the social co‐construction of knowledge by the group of individuals as a group, in a discussion or a seminar” (Lally, 2001, p. 401). Its use for understanding interaction and the process of knowledge construction is thus limited. The current study therefore provides a qualitative analysis of interactional networks and interactional patterns among discussion participants as well as a quantitative analysis based on Henri's framework.

The Current Study

This study examined the impact of a flipped classroom format on students’ learning processes in content‐based instruction, which has recently gained support in foreign language education as a means of fostering academic growth and developing language proficiency (Pessoa, Hendry, Donato, Tucker, & Lee, 2007). Following Henri's (1992) analytical framework, the study investigated learning processes in terms of (1) participation rate, (2) content of L2 comments, and (3) types of reasoning skills shown during class activities. The study also investigated knowledge construction in the flipped classroom by examining (4) interactional patterns during class activities. Using a quasi‐experimental design to compare experimental (i.e., flipped classroom) and control (i.e., traditional classroom) groups, the study addressed four research questions:

  1. Participation rate: Do learners in flipped vs. traditional classrooms exhibit different rates of participation in L2 discussion?
  2. Content of comments: Do learners in flipped vs. traditional classrooms exhibit differences in the content of their utterances—social, interactive, cognitive, and metacognitive—during L2 discussion?
  3. Types of reasoning skills: Do learners in flipped vs. traditional classrooms exhibit different reasoning skills—elementary clarification, in‐depth clarification, inference, judgment, and application of strategies—in their cognitive processing of information?
  4. Interactional patterns: Do learners in flipped vs. traditional classrooms show different patterns of interaction when exchanging information during L2 discussion?

Method

Participants

The study was conducted at a Korean university with Korean as a foreign language learners as the participants. Data were collected during one activity (a small‐group discussion) in a single class (Week 12) of two sections of a semester‐long general requirement course (GETE 006 Thinking and Writing). The course is designed to develop international undergraduate students’ abilities to use Korean for academic purposes. The two sections were randomly chosen from among the four intermediate‐level classes that were offered during the semester: One section was conducted as a flipped classroom (n = 26), and the other section was conducted as a traditional classroom (n = 25). The students in the two sections (males = 28; females = 23) were from various countries (e.g., China, Indonesia, Malaysia, Saudi Arabia, Turkey, Thailand, the United States) and had different L1 backgrounds. Most of them had studied Korean for at least several months (M = 23.61, SD = 11.12) after puberty in a formal context (e.g., private/public language schools) in Korea or in their home countries. Prior to their participation in the study, all students completed informed consent forms. The students in each section were given the opportunity to transfer into other sections, although none opted to do so. Table 3 summarizes the two groups’ demographic data.

Table 3. Demographic Data by Group
Flipped Group (n = 26) Traditional Group (n = 25)
Age M = 20.58, SD = 1.31 M = 21.75, SD = 1.72
Gender Males = 12, Females = 14 Males = 16, Females = 9
Year
Freshman 22 15
Sophomore 0 3
Junior 3 7
Senior 1 0
Korean proficiency levels
Intermediate 15 14
High‐intermediate 11 11
Length of Korean study (months) M = 24.13, SD = 9.15 M = 26.96, SD = 11.98

An independent‐samples t test and a Pearson chi‐square revealed no significant difference for age (t(49) = −1.601, p > 0.05), study length (t(49) = −0.189, p > 0.05), gender (χ2(1) = 1.639, p > 0.05), Korean proficiency level (χ2(1) = 0.015, p > 0.05), or school year (χ2(3) = 6.907, p > 0.05) between the two groups, indicating that the participants in the two sections were fairly homogeneous.

Materials and Activities

PowerPoint‐Based Lecture

In both the traditional and flipped course designs, the topic of Week 12 was Product Placement: Light and Shadows. Product placement (PPL) is an advertising technique used by companies to subtly promote their products through a nontraditional advertising technique, usually through appearances in film, television, or other media (e.g., Chevrolet in the American movie Transformers). The use of PPL in Korean media has increased with the rise of production costs; it is pervasive in contemporary Korean television and movies. The PowerPoint‐based (PPT) lecture conveyed critical, core information about PPL, including definitions and characteristics, reasons for use, legitimacy, and marketing effectiveness. Effective and ineffective examples from current television programs were provided in accompanying video clips.

Quiz for Comprehension Check

Students in each section were also given a 10‐question quiz, consisting of true‐false, multiple‐choice, and short‐answer items. These quiz items were drawn from the PPT lecture and the video examples. The questions were developed following Bloom's revised taxonomy of the cognitive process, with its six levels: remember, understand, apply, analyze, evaluate, and create (Anderson & Krathwohl, 2001). The quiz was designed to support students’ learning of the topic with questions of increasing cognitive complexity. The first five questions were at the first two levels (i.e., remember and understand), the next two questions were at the third level (i.e., apply), and the final three questions were at the fourth level (i.e., analyze). The fifth and sixth levels (i.e., evaluate and create) were not covered in the quiz but were addressed by the small‐group and whole‐class discussions in class.

Small‐Group Discussion

In both sections, the small‐group discussion was designed to lead students to evaluate PPL‐related issues based on criteria and standards (checking and critiquing) by discussing and debating the following opinion‐based question: “Many people claim that excessive use of PPL decreases the quality of television programs. Do you think that PPL should be prohibited by the government? Why or why not?” This question was chosen to enable the students to review previously learned materials, to evaluate current practices of PPL, and to provide further suggestions based on their claims.

Questionnaire

All students also completed a questionnaire on their biographical information, including age, gender, nationality, first language(s), major, school year, L2 learning experiences (e.g., length of time studying Korean), and Korean proficiency level.

Procedures

The students in the flipped classroom and the traditional classroom both encountered the same instructional materials but at different times and in different manners (Figure 1). The students in the flipped classroom watched the online PPT lecture (40 minutes) and completed the comprehension quiz (20 minutes) before class. The lecture and quiz were posted on Blackboard Learn, which provides an interactive online interface for learning. On the day of the class, the students broke into six groups for small‐group discussion (15 minutes) on the opinion‐based question. One student in each group audio‐recorded the discussion using a smartphone, and sent the audio file to the instructor after class. A teacher‐led, whole‐class discussion (45 minutes) followed the small‐group discussion. This whole‐class discussion was intended to stimulate learners’ critical thinking on the topic by giving them an opportunity to share their opinions, reflect on their experiences, come to final conclusions, and provide further suggestions.

image
Experimental Procedure

For the students in the traditional classroom, the same PPT lecture was delivered by the instructor in class. Instead of the quiz, the instructor asked comprehension questions from time to time during the lecture to ensure the students’ understanding. The integrated lecture and comprehension check (60 minutes) was followed by a small‐group discussion (15 minutes). There were six groups. As in the flipped classroom, one student audiorecorded their discussion with a smartphone and sent the audio file to the instructor. The traditional classroom did not allow time for a whole‐class discussion; instead, the students were given a homework assignment, which instructed them to summarize their opinions and provide suggestions for the better use of PPL. All students completed the biographical questionnaire at the end of the class (five minutes).

Coding and Analysis

Participation Rate

The students’ output recorded during the small‐group discussion was coded for further analysis. Participation rates during the small‐group discussion were measured in terms of (1) communication unit (C‐unit) per minute (Chaudron, 1988), (2) word phrases per minute, and (3) sentences per minute, where a higher rate of C‐units, word phrases, and sentences per minute represented more active participation. The number each of C‐units, word phrases, and sentences in each student's utterances was counted, divided by the number of seconds each student spent producing the utterances, and multiplied by 60. The mean values for the two classrooms were calculated by averaging each student's C‐units, word phrases, and sentences per minute.

Content of Learner Comments

Following Henri's (1992) analytic framework, the C‐units from the small‐group discussion were coded into four categories based on content: (1) social, (2) interactive, (3) cognitive, and (4) metacognitive (Table 4). Three researchers coded the entire dataset separately, showing 94.27% consistency. Inconsistent coding (5.73%) was negotiated until consensus was reached. No data were excluded from the analysis for inconsistency. The ratio of C‐units in each of the four categories was first computed as percentages for each student and was then averaged across students.

Table 4. An Example of Data Coding
Cognitive
Ss’ # Utterances C‐unit # Social Interactive EC DC IF JG ST Metacognitive
1 Let's start. 1 1
1 Riken, do you think PPL should be banned? 2 1
2 I disagree with banning PPL. 3 1
1 Why? 4 1
2 One of the reasons is that PPL helps people learn about new products. 5 1
3 Do you think so? 6 1
3 I also disagree that PPL should be banned. 7 1
3 Sometimes PPL can help directors make a film that otherwise would be impossible due to a limited budget. 8 1
1 That's actually true. 9 1
1 Income from PPL allows them to make very high‐quality TV shows using tools such as computer graphics. 10 1
4 You guys are talking about the producers’ position. 11 1
4 For viewers, sometimes PPL seems totally unrelated to the story, randomly appearing in the show. 12 1
4 So PPL is not always good. 13 1
3 Well, instead of simply banning PPL, one solution could be that producers think more how they could use it well, respecting the flow of the program. 14 1
1 But I think some weird PPL is fun. 15 1
1 Don't you think so? 16 1
Total 16 1 5 9 1
1 4 1 2 1
  • Notes: EC = elementary clarification; DC = in‐depth clarification; IF = inference; JG = judgment; ST = strategies; S = student

Reasoning Skills

The C‐units coded as cognitive were recoded in terms of Henri's (1992) five reasoning skills: (1) elementary clarification, (2) in‐depth clarification, (3) inference, (4) judgment, and (5) strategies. An example is provided in Table 4. The same methods were used for the data coding and analysis, with two raters. Their initial consistency was 96.31%, and they reached consensus on the remaining 3.69% of the data through negotiation.

Interactional Patterns

Conversation threading was employed to reconstruct the hierarchical structure of the utterances. This method has been widely used to visually group messages with their replies in online discussion. The discussion in the current study could be broken down into one or more messages and the following replies, so that a message followed by related replies constituted a thread. In a thread, a message contains a main idea as a head, and the following replies support the idea. Note that both a message and a reply can have subordinate replies. A long thread with replies at deeper levels may indicate coherent discussion, whereas a short thread with replies at shallower levels may involve distracted discussion. Three researchers working separately grouped the discussion data into conversation threads with 96.31% consistency; inconsistencies (3.69%) were negotiated until consensus was reached.

Results

For the first research question, the students’ participation rates were quantified by three types of measure (i.e., C‐units per minute, word phrases per minute, and sentences per minute) and were compared. The descriptive statistics (Table 5) indicated that the flipped group and the traditional group produced similar amounts of C‐units (flipped, M = 2.23, SD = 1.56; traditional, M = 1.94, SD = 1.18), word phrases (flipped, M = 15.74, SD = 10.30; traditional, M = 12.22, SD = 6.83), and sentences per minute (flipped, M = 1.68, SD = 1.14; traditional, M = 1.70, SD = 1.03).

Table 5. Descriptive Statistics for the Measures of Participation Rate Across Groups (%)
Flipped (n = 26) Traditional (n = 25)
Measures M Mdn SD SE M Mdn SD SE
C‐units/min 2.23 1.79 1.56 0.31 1.94 1.64 1.18 0.24
WP/min 15.74 13.81 10.30 2.02 12.22 11.29 6.83 1.37
S/min 1.68 1.51 1.14 0.22 1.70 1.42 1.03 0.21
  • Notes: M = mean; Mdn = median; SD = standard deviation; SE = standard error; WP = word phrases; S = sentences

Shapiro‐Wilk tests indicated the absence of a normal distribution for the C‐units per minute (W = 0.922, df = 51, p = 0.003), word phrases per minute (W = 0.916, df = 51, p = 0.002), and sentences per minute (W = 0.940, df = 51, p = 0.012), so the medians of each were submitted to a nonparametric t test (Mann‐Whitney U). The Mann‐Whitney U found no significant difference between the two groups for the C‐units per minute (U = 311, p = 0.792), word phrases per minute (U = 270, p = 0.300), or sentences per minute (U = 313.5, p = 0.828) (Table 5). These results indicate that the students in the flipped classroom and the traditional classroom participated in the small‐group discussions at similar rates.

As for the second research question, the occurrence ratio of the four types of content was compared across groups. Table 6 shows that both groups’ most frequent utterances were cognitive comments (flipped = 439, traditional = 456), followed in order of frequency by interactive comments (flipped = 84, traditional = 174), social comments (flipped = 25, traditional = 24), and metacognitive comments (flipped = 22, traditional = 38).

Table 6. Number of Occurrences of Content Types Across Learning Conditions
Social Interactive Cognitive Metacognitive Total
Flipped (n = 26) 25 84 439 22 570
Traditional (n = 25) 24 174 456 38 692
Total 49 258 895 60 1262

A chi‐square analysis was conducted to observe any relationship between the learning conditions and content. For this analysis, the learning condition variable (two levels: flipped vs. traditional) was entered into the left‐hand column, and the frequency variable of content type (four levels: social, interactive, cognitive, and metacognitive) was entered into the top row of the table. The 2 × 4 chi‐square analysis revealed a significant association between the learning conditions and content, χ2(3, N = 1262) = 24.440, p < 0.001, Cramér's V = 0.139. The sizes of the standardized residuals were then observed to track the cells that made a significant contribution to the overall significant association. Basically, if the absolute value of standardized residuals is equal to or greater than 1.96, then the difference between the observed frequency and the expected frequency is statistically significant at the 0.05 level. If any standardized residuals have an absolute value that is equal to or greater than 2.58, then the difference is statistically significant at the 0.01 level. The sign (±) of the standardized residual generally indicates whether the observed frequency is above (+) or below (−) the expected frequency. The standardized residuals indicated that the significant relationship was particularly due to the frequency of interactive comments and cognitive comments. As for the flipped classroom learners, they produced significantly more cognitive comments (observed frequency (O) = 439, expected frequency (E) = 404.2, standardized residual (R) = 2.0) and significantly fewer interactive comments (O = 84, E = 116.5, R = −3.0) than expected. On the other hand, the traditional classroom learners produced a significantly higher number of interactive comments (O =174, E = 141, R = 2.7) and a significantly lower number of cognitive comments (O = 456, E = 490, R = −2.1) than expected. No statistical differences were found for social (flipped: O = 25, E = 22.1, R = 0.6; traditional: O = 24, E = 26.9. R = −0.6) or metacognitive comments (flipped: O = 22, E = 27, R = −1.0; traditional: O = 38, E = 32.9, R = 0.9).

As for the third research question, the occurrence ratios of the five types of reasoning skills were analyzed across groups. Table 7 shows that the flipped group most frequently used in‐depth clarification (304), followed in order of frequency by inference (56), judgment (33), elementary clarification (29), and strategies (17). The traditional group also most frequently used in‐depth clarification (319), followed in order of frequency by judgment (42), inference (39), elementary clarification (37), and strategies (19).

Table 7. Number of Occurrences of Reasoning Skill Types Across Learning Conditions
Elementary Clarification In‐depth Clarification Inference Judgment Strategies Total
Flipped (n = 26) 29 304 56 33 17 456
Traditional (n = 25) 37 319 39 42 19 439
Total 56 623 105 75 36 895

A chi‐square analysis was conducted to test the association between the learning conditions and students’ reasoning skills. For this analysis, the learning condition variable (two levels: flipped vs. traditional) was entered into the left‐hand column, and the frequency variable of reasoning skill type (five levels: elementary clarification, in‐depth clarification, inference, judgment, and strategies) was entered into the top row of the table. The 2 × 5 chi‐square analysis revealed a significant relationship between learning condition and content, χ2(4, N = 895) = 17.890, p < 0.001, Cramér's V = 0.146. The standardized residuals indicated that the significant relationship was due mainly to the frequency of elementary clarification and inference. As for the flipped classroom learners, they exhibited significantly more inference (O = 69, E = 53, R = 2.2) and significantly less elementary clarification (O = 16, E = 26, R = −2.2) than expected. On the other hand, the traditional classroom learners showed a significantly higher amount of elementary clarification (O = 37, E = 27, R = 1.9) and a significantly lower amount of inference (O = 39, E = 55, R = −2.2) than expected. No statistical differences were found for in‐depth clarification (flipped: O = 304, E = 305.6, R = −0.1; traditional: O = 319, E = 317.4, R = 0.1), judgment (flipped: O = 33, E = 36.8, R = −0.6; traditional: O = 42, E = 38.2. R = 0.6), or metacognitive skills (flipped: O = 17, E = 17.7, R = −0.2; traditional: O = 19, E = 18.3, R = 0.2).

For the fourth research question, the interactional patterns were visualized and analyzed in R, a programming language and software for statistical computing and graphics (R Core Team, 2015) using the ingraph (Csardi & Nepusz, 2006) and network (Butts, 2015) packages. For this analysis, two groups that produced comparable numbers of utterances during the 15‐minute discussion were chosen from each classroom condition: Group 5 (127 utterances) from the flipped classroom and Group 4 (124 utterances) from the traditional classroom. Recall that the class discussions were threaded to observe their hierarchical structures. To visualize the hierarchical structures of the discussions, node and edge information were encoded in the ingraph and network packages in R. Node information consists of utterance IDs, comment types, and speaker IDs. Edge information denotes the flow of utterances—that is, pairs of utterances in order. All of the utterances were coded as one of four comment types: social, interactive, cognitive, or metacognitive. The cognitive utterances were recoded by the five reasoning skills: elementary clarification (CogEC), in‐depth clarification (CogDC), inference (CogIF), judgment (CogJG), and strategies (CogST). For example, “utterance 5, cognitive, speaker 3” is node information, whereas “utterance 5 à utterance 12” is edge information.

The results shown in the visual representations of the four groups’ interactional patterns (Figures 2 and 3) were generally in line with the findings from the previous content analyses. One of the major differences found between the two groups’ interactional patterns is that the flipped group created fewer conversation threads (Figure 2, four threads) than did the traditional group (Figure 3, five threads). This difference indicates that the traditional group exchanged a smaller amount of information in each conversation thread while the flipped group exchanged a larger amount of information per conversation thread. As for the relatively longer conversation threads in the traditional group (Figure 3, thread #2 with 54 edges; thread #4 with 82 edges), these threads’ interactional patterns primarily consisted of interactive comments and cognitive comments involving shallow information processing, such as elementary clarification. In contrast, the conversation threads of the flipped group (Figure 2) appeared to consist of cognitive comments involving deep information processing, such as in‐depth clarification, inference, and application of strategies, as well as interactive comments. This network analysis of interactional patterns using R suggests that the participants in the flipped group created more coherent discussions than did the participants in the traditional group.

image
Interactional Pattern of Flipped Classroom (Group 5: 127 Utterances; Four Major Threads)
image
Interactional Pattern of Traditional Classroom (Group 4: 124 Utterances; Five Major Threads)

In order to examine whether knowledge was differentially constructed in the flipped vs. traditional groups, the subsequent qualitative analysis identified the core ideas discussed within each conversation thread. The results of the qualitative analysis are summarized in Tables 8 and 9, which list core ideas, or constructed knowledge, from the various information exchanges during the discussions. Recall that the discussion question was whether they agreed or disagreed with PPL. Hence, the conversation threads in both classroom conditions tended to be initiated by judgment comments (e.g., “I agree/disagree with the current use of PPL”) followed by supporting ideas with other cognitive comments using a variety of reasoning skills (e.g., elementary/in‐depth clarification, inference, and strategies). In most cases, the core ideas, or knowledge, were constructed as a result of multiple numbers of cognitive comments processed at a deeper level (e.g., in‐depth clarification, inference, and strategies). In contrast, when multiple numbers of comments were exchanged at a shallow processing level (e.g., elementary clarification), no core ideas were formed and the discussion tended to end without reaching a conclusion. In line with the previous content analyses, in which the flipped students used inference significantly more and elementary clarification significantly less than did the traditional students, the qualitative analysis also found that greater numbers of core ideas were discussed per conversation thread in the flipped group.

Table 8. Thread Summary of Flipped Classroom (Group 5: 127 Utterances; Four Major Threads)
Thread Starting Edge # # of Edges Ideas discussed with deep information processing
1 2 12
  • Frequent use of PPL reduces viewers’ interest in the show.
  • Using PPL violates viewers’ right to enjoy the show.
2 15 19
  • Producers should use reasonable amounts of PPL so that it does not distract viewers from watching the show.
3 32 26
  • Sometimes producers use PPL without thinking much about whether the products can be naturally associated with the show.
  • Producers should be responsible for how PPL is used.
4 39 65
  • PPL is a win‐win strategy for the media production companies and the companies whose products are advertised.
  • Separate commercials inserted in between parts of a TV show are very distracting.
  • Effective use of PPL can increase a company's profits.
  • PPL is less distracting than separate commercials.
Table 9. Thread Summary of Traditional Classroom (Group 4: 124 Utterances; Five Major Threads)
Thread # Starting Edge # # of Edges Ideas discussed with deep information processing
1 3 18 None
2 10 54
  • Korean movies and shows are now seen by people in other countries, so PPL is good for international marketing.
3 14 0 None
4 15 82
  • PPL is effective for advertising products because the items, such as how to use them, can be naturally introduced in the show.
5 19 5 None

Discourse analysis of the two groups’ utterances also shows the flipped group's deeper information processing, more cohesive discussion, and effective construction of knowledge. For instance, Example 1, taken from one of the discussion threads of the flipped group (see Figure 2), illustrates how the students asserted and justified their thoughts with relevant supporting ideas. In Example 1, Hou states his opinion that PPL should be banned (Edge 2) and then elaborates by explaining the reasons why PPL should be banned: e.g., that excessive use of PPL interrupts the storyline and distracts viewers (Edges 2–6). Hou then infers that PPL has negative effects (Edge 7). Based on Hou's claim, Jin further infers that viewers will have a negative attitude toward PPL (Edge 8) and producers’ irresponsibility (Edge 9). From Edge 10 to Edge 11, Addie partially addresses the positive role of PPL but nevertheless infers that its excessive use can violate viewers’ rights. Thus, Example 1 illustrates that the learners supported the main idea of the thread—that PPL should be banned—by expressing a range of relevant ideas and revealing deep information processing, and that they consistently maintained the coherence of the argument in the thread. All names in the examples are pseudonyms.

Example 1

Name Edge Utterance Content
Hou 2 I agree with the idea that PPL should be banned. JG
3 There are so many PPLs inserted in the TV shows. CogDC
4 I see that some PPLs do not perfectly fit in the story of the shows. CogDC
5 These PPLs interrupt the storyline of the shows, CogDC
6 and people get distracted. CogDC
7 So PPL is not always good. CogIF
Jin 8 Yes, so it can make viewing very unpleasant. CogIF
9 I would say the producers are very irresponsible. CogIF
Addie 10 PPL is a way to sponsor the producers CogDC
11 so we can watch TV shows of better quality. CogIF
12 But I agree too many PPLs in the show distract viewers’ attention. CogDC
13 Then the viewers’ rights are violated. CogIF
Lynn 14 OK, any different opinions? Social

Besides being more coherent, discussions in the flipped classroom appeared to involve more creative and higher‐order arguments in which speakers offered solutions or further actions to be taken to address the problems under discussion. For example, in Example 2, after Lynn and Hou agree that PPL still has some benefits (Edges 55–58), Jin proposes that government regulations could control the excessive use of PPL while retaining its benefits (Edges 59–62). These utterances indicate that Jin is interacting with the concerns about PPL to suggest ways to overcome the problem. A similar cognitive activity is shown by Addie (Edges 65–67), who suggests that PPL should be monitored to protect viewers’ rights. Thus, the example illustrates that the flipped classroom students’ discussion was not limited to the exchange of content knowledge (e.g., the advantages and disadvantages of PPL) but reached the level of conceptual application and evaluation of content knowledge.

Example 2

Name Edge Utterance Content
Lynn 55 There are good things and bad things about PPL, CogIF
56 so PPL is just like many other things. CogIF
Hou 57 That's true. Interactive
58 I agree with you. Interactive
Jin 59 I think PPL should be strictly reviewed by the government, CogST
60 so both the quality issue and budget issue can be resolved. CogST
61 There are too many advantages of PPL to completely ban it, CogDC
62 and there are too many disadvantages of PPL to completely not regulate it. CogDC
Lynn 63 Right, right… Interactive
Hou 64 That's a good point. Interactive
Addie 65 So, yes, the use of PPL should be strictly monitored, CogST
66 so the viewers’ rights are respected. CogST
67 Viewers’ rights are important. CogIF

On the other hand, the traditional group typically demonstrated a lack of coherent discussion and deeper processing of information. Example 3 is representative of the discussions in the traditional class. Phil asserts his opinion that PPL should be banned (Edge 14) and elaborates on this opinion with supporting ideas that PPL is not more useful than commercials (Edges 15–17). Then Mia states that she agrees with the prohibition of PPL (Edges 18 and 24), but her supporting ideas, such as that PPLs allow companies to make more profit, which in turn leads to higher levels of employment (Edges 20–22), support the opposite position. Although it can be assumed that she meant to argue that PPLs should not be banned, her comment in Edge 23 that more people will receive higher education does not show a strong logical connection with her previous comment that the country will be richer. Phil's additional comments (Edges 36–37) support his negative position toward PPL but do not show any cognitive elaboration, as they are merely anecdotal. Moreover, after Choo argues that PPL is a good marketing strategy (Edges 38–41), the speakers begin to digress. Instead of arguing for the benefits or disadvantages of PPL, Xiu (Edges 42–43) and Choo (Edges 44–46) start to focus on the quality of Chinese products and their popularity among Americans. Thus, Example 3 illustrates how, in contrast to the flipped group (e.g., Examples 1–2), the traditional group failed to achieve cohesive discussion and demonstrated only shallow information processing.

Example 3

Name Edge Utterance Content
Phil 14 I think PPL should be banned. JG
15 PPLs don't provide accurate information about the products. CogDC
16 Commercials are better at giving accurate information, CogDC
17 and people understand about the products more easily. CogDC
Mia 18 Hello. Social
19 I agree that PPL should be banned. JG
20 That is, because PPL helps the companies make more profit, CogDC
21 the companies will hire more people. CogDC
22 It will make the country richer, CogDC
23 and more people can receive college education, CogEC
24 so I agree that PPL should be banned. JG
John 25 I disagree that PPL should be banned, JG
26 because customers can get information from them while watching CogDC
27 the shows. JG
For this reason, I agree with the use of PPL.
Mia 28 That's what I mean. Interactive
John 29 What? Interactive
30 Let me clarify. Interactive
31 I agree with the use of PPL, JG
32 because the companies can profit. CogDC
33 PPL is different from advertisements, CogDC
34 because it is beneficial for customers as well as companies. CogDC
35 Customers can see and learn about the new functions of a product in the show. CogDC
Phil 36 Many Korean TV shows are aired in China. CogEC
37 There are so many PPLs in the shows. CogEC
Choo 38 Can I add on to my thoughts? Social
39 So I mean there are many Chinese PPLs in American movies. CogDC
40 PPL is a great way to advertise things to other countries. CogDC
41 Chinese brands are getting popular among Americans these days. CogDC
Xiu 42 Is that true? Interactive
43 Americans don't buy Chinese products because of poor quality. CogDC
Choo 44 Well, I don't think so. Interactive
45 If we use good marketing strategies, Americans will buy them. CogDC
46 And I think PPL is one of the ways. CogDC

Discussion

This study compared L2 discussion in flipped classrooms and traditional classrooms in terms of (1) participation rate, (2) content of comments, (3) types of reasoning skills, and (4) interactional patterns. The learners in the two classroom settings participated equivalently according to the quantitative measures of numbers of C‐units, word phrases, and sentences produced per minute. This result differs from previous claims that the flipped classroom approach encourages students’ active participation in tasks (Hannafin, Hill, & Land, 1997; James, Chin, & Williams, 2014). This inconsistency may have been caused by other learner‐internal and/or external factors. Recall that the class topic dealt with Korean media. The popularity of the Korean cultural wave, or Hallyu, among international students residing in Korea may mean that all the students had extensive prior knowledge of the topic, which would have positively affected their participation rate. Indeed, the transcripts of the students’ oral output showed that the students in both classroom conditions frequently mentioned particular scenes, television shows, and actors to exemplify their opinions and that these comments were expanded and elaborated on by other students who clearly had the same prior knowledge. In addition, the students knew they were being recorded. They may therefore have felt some pressure or felt that their participation would be noted by the instructor later on, which could have led them to produce more than they may have otherwise.

As for the content of comments and reasoning skills, the students in the flipped classroom produced significantly more cognitive comments and showed significantly more use of reasoning skills involving deeper information processing (e.g., inference) than did the traditional classroom students, who were more likely to use interactive comments and reasoning skills involving shallow information processing (e.g., elementary clarification). Thus, the traditional classroom and the flipped classroom differentially affected the learners’ cognitive processes. That is, the flipped classroom more effectively supported the skills required for critical thinking and problem solving because the flipped classroom students had a substantial amount of content knowledge in advance and worked on new knowledge acquisition and problem resolution.

The major difference between flipped and traditional classrooms is the timing of content learning: Students in a flipped classroom learn the content before class in their own private time and space and at their own pace, while traditional classroom students learn it in class in a way and at a pace that are determined by the instructor. Therefore, the rich cognitive activity (e.g., in‐depth information processing and higher‐order thinking processes) observed in this study's flipped classroom may have been due to the students having more time to think about the content and activate relevant prior knowledge related to the content. The cognitive load theory (Sweller, 1988, 2004, 2007) developed in the field of educational psychology supports this scenario. According to this theory, learners’ “cognitive capacity in working memory is limited, so that if a learning task requires too much capacity, learning will be hampered” (de Jong, 2010, p. 105). Hence, the available knowledge structures in long‐term memory, or a large, permanent store of organized information, are essential for preventing working memory overload and for guiding cognitive processes. Drawing on Sweller's cognitive load theory to design instructional systems could give students more opportunities to expand their knowledge and develop the relevant schemas in their long‐term memory, which would later guide their information processing and strategies during L2 discussion. Indeed, previous research (e.g., Gall & Hannafin, 1994; Lawless & Kulikowich, 1996; Scheiter, Gergets, Vollmann, & Catrambone, 2009) has consistently reported that students with higher levels of prior knowledge utilize deeper processing strategies because the prior knowledge guides their information selection and thus reduces their cognitive load during learning. In the current study, the traditional students who learned the content on the same day may not have had enough time to expand their knowledge and establish the schemas before the discussion. Hence, their discussion would not have been supported by executive guidance, which would explain why the students remained at a superficial level of information processing and strategies.

The study's fourth focus of interest was the students’ interactional patterns. The results indicate that the students in the flipped classroom interacted more deeply and cohesively than the students in the traditional classroom, as shown by the flipped groups’ longer conversation threads with a higher number of edges. Furthermore, the subsequent analysis of the content illustrated that the longer conversation threads of the flipped classroom led to more core ideas, or constructed knowledge, within each thread. These results illustrate that students in the flipped classrooms were more likely to elaborate on their thoughts while maintaining the flow of discussion more cohesively and coherently; such interaction patterns, in turn, lead to successful knowledge construction.

Knowledge elaboration refers to “using prior knowledge to continuously expand and refine new material based on such processes as organizing, restructuring, interconnecting, integrating new elements of information, identifying relations between them, and relating the new material to the learners’ prior knowledge” (Kalyuga, 2009, p. 402). According to Kalyuga, the knowledge elaboration process takes place efficiently when students continuously balance executive guidance based on the current state of their knowledge in a domain. This study's results suggest that successful knowledge elaboration is more likely in the flipped classroom than in the traditional classroom, which supports the notion that the prior knowledge that flipped students acquire before class helps them activate schema‐based information processing, reducing their cognitive load during learning. The students in this flipped classroom may have been able to use their cognitive resources more efficiently (e.g., by focusing on critical ideas and ignoring redundant statements), allowing higher‐level knowledge elaboration. According to Kalyuga, the process of knowledge elaboration is essential for knowledge construction because it allows “the learners to organize knowledge into a coherent structure and integrate new information with existing knowledge structures” (p. 402). No evidence for knowledge elaboration was found in the discussion data of the traditional students, who did not acquire a relevant knowledge base before class and therefore may have had fewer cognitive resources available to them during the discussion.

Despite interesting findings, the present study has some limitations. The number of participants was small (n = 51, five groups in each condition), so the findings are exploratory rather than widely generalizable. While the participants were homogeneous in age and language proficiency, their participation and interaction may still have been affected by other individual factors, such as prior knowledge, cultural background, motivation for learning, cognitive abilities in the L1, and self‐directed learning abilities. As one of the anonymous reviewers suggested, this limitation could be partially resolved by employing a crossover design, in which two groups receive both instructional interventions but at different times. In addition, it would be interesting to consider the possible effects of individual differences in flipped vs. traditional classrooms.

Another limitation of the study is derived from the limited amount of data collected from the one‐time 15‐minute group discussion. The limitations of the small data set (and some of the limitations of the small number of participants) could be overcome by extending the instructional interventions over a month or semester and by collecting the data from several 15‐minute discussions from each group. Indeed, Love et al. (2014), who compared the effectiveness of a flipped model and traditional lecture in a college linear algebra course, conducted a semester‐long experiment with two randomly chosen sections of the course (flipped = 27 students; traditional = 28 students). While Love et al.'s study and this study had similar numbers of participants, the former offered more generalizable results because it had a larger data pool obtained throughout the semester. Further research that replicated the present study's method but gathered more data would validate and increase the generalizability of the results.

In addition, an anonymous reviewer pointed out that the higher quality of L2 discussion in the flipped classroom may have been due to the additional time allowed for discussion compared to the time allowed in the traditional classroom. However, having time between content learning (online) and discussion is one of the benefits of flipped classrooms. It is worth noting that the difference between flipped and traditional classrooms is not the presence or absence of additional time but how and where the time is used. In other words, because students in traditional classrooms are encouraged to preview chapters at home before class, it is not quite accurate to say that the flipped classroom had additional time. Rather, both classes had the same amount of time, but different timing—which was the focus of the study. Therefore, the different qualities of the two groups’ discussions suggest that the timing of activities in the flipped classroom was more effective because it gave the learners additional time to internalize knowledge. In contrast, the timing of activities in the traditional classroom was less effective because it gave the learners additional time prior to the content learning and discussion. Put differently, the flipped classroom may be more effective for learning because it improves students’ learning behavior by creating a voluntary learning environment.

Conclusions and Pedagogical Implications

Notwithstanding the identified limitations, the quantitative and qualitative findings of the present study demonstrated that the flipped classroom did appear to promote L2 learners’ deeper cognitive processing, higher‐order thinking skills, cohesive discussion, and higher‐level knowledge elaboration, all of which effectively supported their knowledge construction during discussion in a content‐based instructional context. These benefits may be attributable to the executive guidance based on knowledge schema that were established before class through prelearning. No significant difference was found between the students’ participation rates in the traditional and the flipped classrooms, but the quality of content and knowledge discussed in the two types of classroom was fundamentally different, suggesting that the flipped classroom provided more substantial and meaningful learning experiences for the students.

From a pedagogical perspective, this study sheds light on the importance of a learning environment that encourages active and cohesive interaction in classrooms. It also contributes to considering how the environment for active learning can be better achieved by incorporating technology in pedagogical practices. The study emphasizes the fact that L2 students in content‐based instruction can benefit from accessing lecture materials prior to class meetings because having prior knowledge enhances their ability to engage in cohesive discussion, in‐depth thinking, and construction of knowledge during class. Developing critical thinking ability has been a longstanding goal in content‐based L2 classrooms; there is general agreement that this skill does not naturally grow along with L2 development but requires intensive training. All in all, the present study has provided a glimpse into how flipped classrooms can facilitate critical thinking in an L2. More research needs to be conducted in diverse classrooms to gain a greater understanding of the effectiveness of flipped classrooms for learners with different social/cultural/linguistic backgrounds and learner‐internal traits and of possible ways to overcome pedagogical limitations experienced by particular groups of learners.

Acknowledgments

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF‐2016S1A5A8019396).

      Number of times cited: 1

      • , Investigating the factors that influence higher education instructors' decisions to adopt a flipped classroom instructional model, British Journal of Educational Technology, , (2018).