Journal of Computer Assisted Learning

2.1 Argumentation in science classroom

The teaching of argumentation, in which students need to build arguments by considering evidence and counter evidence, using appropriate reasoning, and evaluating alternative standpoints, has become increasingly prevalent and also an essential goal for science education (Duschl, Schweingruber, & Shouse, 2007; Moon, Stanford, Cole, & Towns, 2017; Osborne et al., 2016). It has been reported that argumentation is rarely found in the science discourse of most students, because regular classroom discourse typically follows a pattern in which the teacher initiates discussion by using a question with a known answer, the students respond to the question, and then the teacher evaluates the students' responses (Driver, Newton, & Osborne, 2000; Kilinc, Demiral, & Kartal, 2017; Lemke, 1990).

A number of online learning programs were developed to support students' use of scientific languages in argumentation (Yang et al., 2015; Golanics & Nussbaum, 2008; Hasancebi & Gunel, 2013). The programs situate learners in collaborative, virtual, and explorative environment, enable them to explore relevant information from multiple resource and support them to make statements and reflections based on multiple views (Meluso et al., 2012). In our previous work, an online argumentation learning environment was developed to facilitate students' use of scientific language and application of scientific evidence in argumentation (Lin, Hung, & Hung, 2017). An argumentation feature was noticed during the analysis of the arguments generated by the students in the low prior knowledge group. They tended to base their arguments on personal experiences, such as what they had done, found, and been told, rather than on scientific knowledge. This finding implied that it is necessary to activate students' prior knowledge and experience in order to help them acquire meaningful learning. However, there have still only been a limited number of empirical studies that have actually investigated game‐based learning (GBL) and the relationship between contextualized factors and science outcomes (Meluso et al., 2012).

In order to provide appropriate scaffoldings, research studies have applied Toulmin's argumentation pattern (Toulmin, 1958) as a key theoretical basis (Lin et al., 2017; Erduran, Simon, & Osborne, 2004; Weinberger, Stegmann, & Fischer, 2010). Theoretically, there are six components in Toulmin's model: data, claims, warrants, backings, qualifiers, and rebuttals. Previous studies have indicated, however, that the application of the model can be problematic when clarifying exactly what counts as data, warrants, and backings (Kelly, Druker, & Chen, 1998; Oliveira, Akerson, & Oldfield, 2012). So, like a number of previous studies, we reduced Toulmin's concepts of “warrants,” “data,” and “backings” into a single category called “warrants” (Erduran et al., 2004; Zohar & Nemet, 2002). Furthermore, it should be noted that argumentation is a kind of social and communicative activity and, relatedly, that the quality of students' argumentation is related to the questioning strategies and questions provided by the teacher and their peers (Chin, 2007; González‐Howard & McNeill, 2016; Martin & Hand, 2009). This implied us that it was important to take the utterance of questions into account in our data analyses.

2.2 The effects of GBL on science learning

GBL is thought to be an effective tool that can promote learning experiences (Connolly, Boyle, MacArthur, Hainey, & Boyle, 2012) and motivation (Hung, Sun, & Liu, 2018; Papastergiou, 2009; Perini, Maria, Fradinho, & Manuel, 2017). Kebritchi and Hirumi (2008) identified the following five reasons for applying GBL to instruction: (a) GBL emphasizes peers' interactions, (b) GBL creates personal motivation and satisfaction, (c) GBL accommodates multiple learning styles, (d) GBL reinforces mastery of skills, and (e) GBL provides a collaborative and decision‐making context. Furió, González‐Gancedo, Juan, Seguí, and Rando (2013) indicated that GBL could provide multiple benefits to teachers in terms of helping them to develop appropriate strategies for achieving a number of instructional purposes. When learning through games, students typically take greater interest in and have more opportunities for discovering, talking, exploring, and questioning (Perini et al., 2017; Yang, 2012). However, the use of GBL does not guarantee that learners will consistently achieve better learning results because it is not possible to identify a clear causal relationship between students' gameplay experiences and academic performance (Byun & Joung, 2018; Meluso et al., 2012; Wrzesien & Raya, 2010). As O'Neil, Wainess, and Baker (2005) articulated, educational GBL should take the prior knowledge and life experiences of individuals into account in order to enhance learning outcomes. In one study, Wrzesien and Raya (2010) found that sixth‐grade students who received instruction via a science GBL environment had higher engagement levels than students in a control group; however, there was no direct evidence to prove that the GBL led to significantly better learning achievement among the students who took part in it as compared with the students taught in a traditional class.

In science education, students' science abilities, such as their problem‐solving, questioning, and argumentation abilities, as well as their ability to engage in scientific inquiry, are emphasized in contemporary educational reform, and GBL has been advocated as a promising approach by which to make changes from traditional learning to a more innovative form of learning in which learners' interests and willingness to take part are emphasized (Barab & Dede, 2007; Hung et al., 2018; Maxmen, 2010; Perini, Luglietti, Margoudi, Oliveira, & Taisch, 2017). In fact, even though previous research has yielded both positive and negative evidence regarding the effectiveness of using GBL to enhance learning, researchers still regard GBL as being potentially beneficial to educational reform efforts because it provides a new perspective from which researchers can potentially better understand and explain cognitive as well as affective learning issues.

2.3 Comprehension and contextualized environments

Comprehension theories basically agree on the notion that reading is a constructive process in which readers activate their prior knowledge and integrate it with the new information they are receiving from the text to achieve a coherent understanding (Mason, Tornatora, & Pluchino, 2013; Peterson, 2016). For example, Kintsch and Van Dijk (1978) constructed the construction–integration model based on the assumption that reading comprehension is a bottom‐up process. A reader may build three different type of mental representations of a text: verbatim, semantic, and situational representations (Kintsch, 1998). The initial phase of reading involves meaning construction in which a reader constructs a network of propositions both from the text and from prior knowledge. This initial phase can be chaotic, regulated by weak production rules, and even result in contradictory outputs. However, the second phase, which is called integration, consists of a refinement process in which “the network of propositions spread their activation until it becomes stable” (Sanjose, Vidal‐Abarca, & Padilla, 2006, p. 7).

Based on Kintsch's theory, Marzano and Kendall (2007) explained that a comprehension process involves integrating and symbolizing phases. To integrate, a learner must recognize the basic structures of any new concept encountered and be able to identify it (i.e., the new concept) as noncritical or critical. To symbolize, a learner must activate their prior knowledge and then create a useful symbolic representation for the new concept under consideration. Previous studies on visual representations have focused on how to use multimedia to enhance contextualized learning and reading comprehension (Fillipatau & Pumfrey, 1996; Freitas & Castanheira, 2007). It was reported that a reader's interpretations of graphics, texts, and images involve inferential processes (Kearsey & Turner, 1999; Kress & van Leeuwen, 1996). That is, individuals usually rely on contextualization cues to obtain meaning when reading (Gumperz, 1986). For example, Stylianidou, Ormerod, and Ogborn (2002) analyzed the images in science textbooks and students' associated reading comprehension of those books. They suggested that teachers should spend sufficient time discussing textbook images with their students. In other words, constructing a more contextualized environment for students as they engage in reading supports meaningful learning. At the same time, language education researchers have reported that decontextualized environments provide another means by which to learn, with various pieces of evidence showing that such decontextualized environments do improve learners' knowledge acquisition (Hunt & Beglar, 2005; Schmitt, 2008). In this regard, there are at least two different approaches for learning and reading: the decontextualized approach and the contextualized approach. In a study investigating how best to learn a foreign language, Nation (2011) indicated that decontextualized environments constitute an important approach. This approach is derived from an ancient Greek word and mostly used while memorizing. It is a common sense that a learner should expend effort in order to make sense of a new word in multiple contexts; however, the merits of spending too much time to learn just one word have been questioned (Ünaldi, Bardakci, Akpinar, & Dolas, 2013). As such, whether language‐related instruction should be carried out in context or out of context remains a matter of some debate among professionals in language education (Oxford & Scarcella, 1994; Ünaldi et al., 2013).

In science education, we believe that a similar situation exists. That is, the question of how to arrange and present multimedia and concepts and the question of the level to which contextualized environments should be utilized in order to improve students' learning comprehension were important research questions for the present study. An online learning context is usually constructed by applying virtual media, which are created by computer programs, and real‐world media, which are taken from reality (Ge & Er, 2005). One advantage of using virtual media, such as computer simulations, is that doing so eliminates complex factors and avoids dangerous processes that may exist in real‐world situations (Chen et al., 2013; Chen & Hwang, 2017). Importantly, such forms of online learning and simulations have great potential to help learners acquire deeper understandings of science (Kim & Hannafin, 2011; Lai, Hwang, & Tu, 2018; Lindgren, Tscholl, Wang, & Johnson, 2016; Park et al., 2009). In contrast with simulations, digital materials from the real world usually involve details. These details can also be seen as contextualization cues that can assist in activating learners' prior knowledge. The media used for constructing online learning contexts should be based on the learners' prior knowledge in order to support their meaningful learning (Hwang et al., 2015; Taguchi, Li, & Tang, 2017). This principle enabled us to define three levels of contextualized media.

3.1 Participant

Four 10th‐grade classes totaling 148 students participated in the study and were randomly assigned into four groups: the mildly contextualized group (N = 37), the moderately contextualized group (N = 36), the highly contextualized group (N = 37), and the control group (N = 38). There were five learning topics that were presented to all the participants: (a) balance levers, (b) evolution, (c) refraction and dispersion, (d) plants and seasons, and (e) fermentation.

3.2 Procedure

An online game‐based science argumentation (OGSA) program was created for our quasi‐experimental design. Before the experiment, the students in the three experimental groups attended two 25‐min introductory lessons. The first one introduced them to basic knowledge regarding science argumentation, including the four categories of utterances in argumentation: claims, warrants, rebuttals, and questions, while the second introduced them to the basic operations of the OGSA using the biology exercise topic “how to identify male and female butterflies.” In order to facilitate the students' coconstructions and coevaluations of each other's arguments in the OGSA, a heterogeneous grouping strategy was used in which each larger experimental group was further divided into smaller groups consisting of just four to five members.

After the introductory lessons, the students in the three experimental groups received similar inquiry‐based instructions and then participated in the OGSA in order to complete one topic of learning. Specifically, they were encouraged to investigate the learning topic by conducting experiments in a laboratory, exploring the relevant literature online and in the school library, making observations, and visiting a science museum near the school. Then, they spent two 40‐min classes to complete one topic of the OGSA learning in a school computer classroom, with a total of 16 study hours being completed over 3 months. The topic of fermentation is one example of the learning topics covered. For that topic, the students in the three experimental groups were invited to make homemade wine at the beginning of the research semester. In order to acquire a deep understanding of the scientific concept of fermentation, they were encouraged to explore the relevant literature online and interview their friends and family. Moreover, they were asked to record their observations of their homemade wine, including observations of its colour, smell, and other characteristics, by using their cell phone cameras and notebooks every 2 or 3 days throughout the 3 months. A respective cloud database was created for each of the three experimental groups. The experimental group students were asked to import the collected data (literatures, reports, notes, photos, and videos) into the respective cloud database. Moreover, they were encouraged to review and discuss the data collected by themselves and their peers in the same experimental group. Only the data collected by the students in the highly contextualized group were reviewed and then selected by our research team as materials for constructing a highly contextualized version of the OGSA. The research team included the teacher, two researchers, and one professor of science education. Among the objects that were selected were the media (including two photos and one 1‐ to 4‐min video) created by the students that could best represent their science inquiry experiences. Before the experiment, we briefly explained to the students in the three experimental groups that the photos and videos they created might be used as materials for constructing the OGSA. We had their permission to use the media. Basically, they viewed it as an honour if their media were selected and published. As to the students in the control group, they basically received traditional instructions that could generally be characterized as teacher‐centred and involving little communication with peers as they covered the five topics. After receiving instruction regarding a given topic, they then had to practice and pass several traditional assessments. Most of the questions in the traditional assessments were well designed so to determine whether or not the students had acquired a body of knowledge. Hence, in comparison with the students in the experimental groups, the control group students had a learning environment with a minimal level of contextualization.

3.3 The OGSA and definitions of the three levels of contextualized media

The first part of the OGSA provides learner a brief introduction to target science concepts and figures, to engage them in problem scenarios. After that, the OGSA provided two types of game‐based argumentation for one topic of learning: find‐the‐fault and find‐the‐difference games. The first one consisted of “find fault” argumentation. In this type of argumentation, the participants had to find all the faults and unreasonable phenomena in the given images, for example, an upside‐down rainbow. The second type of argumentation consisted of “find difference” argumentation. Similar to the find fault argumentation, the find difference argumentation required the participants to spot the differences between two given images.

Three versions of the OGSA were developed on the basis of the three respective levels of contextualized media. A main difference among them involved the embedded media they utilized. In our definition, highly contextualized media consist of photos and videos that can effectively activate students' prior knowledge in order to solve the problems in the OGSA. The media created by the highly contextualized group students represented part of their direct experiences. It is thus reasonable to assume that they involved the best contextualization cues in terms of activating the students' prior knowledge. Second, the moderately contextualized media were defined as those media that can provide vicarious experiences. These media were created by our research team using a camera. Any contextualized factors such as backgrounds and voices were removed or limited as much as possible. The students did not participate in any filming tasks. So, the first time they saw the media was when they played the games in the OGSA. Third, as to the mildly contextualized media, they were created by our research team based on the media used in the moderately contextualized version by using drawing and animation software. The mildly contextualized media show science phenomena and experiments (such as experiments involving refraction and dispersion) with as few contextualization cues as possible. With the exception of the media used, the other elements embedded in the OGSA interface, such as the main question for gameplay, student discussion block, and templates for supporting argumentation, were the same. Appendix A shows examples of the three types of contextualized images embedded in the five topics of the OGSA.

Figure 1 shows an interface of the find‐the‐fault gameplay in the moderately contextualized version of the OGSA for the balance levers topic. The students were asked to point out any possible problems with a bamboo dragonfly body that could cause its failure to balance and then propose arguments accordingly. Students were allowed to view a video on the left‐hand side of the interface showing two different bamboo dragonflies. One of them could balance on a small area with its head, while the other one failed to do so. The two photos labelled as “figure 1” and “figure 2” in Figure 1 show the front side and reverse side of the bamboo dragonfly that failed to balance.

Figure 2 shows an example of a find‐the‐difference activity regarding homemade wine in the highly contextualized version of the OGSA. The two photos in Figure 2 allow for a comparison between samples of homemade wine on its first day after being made with the same sample after it had been aged for 10 weeks. During game playing in the OGSA, the students could use a magnifier to spot the differences between the wines in the two photos. Furthermore, the students were allowed to explore related information on the internet and were allowed to review their group members' statements and provide their own responses. The right panel of the OGSA interface was used by the students for the purpose of entering four different utterances. To help the students to use scientific language in generating their arguments, two layers of templates were provided. The first layer provided definitions (that is, definitions of “claim,” “warrant,” “rebuttal,” and “question”), whereas the second layer provided one to two templates for each argumentation component. As shown on left side of Figure 2, the interface also provided students with a video of the day during which they made homemade wine together. Appendix B provides examples of the three versions of the OGSA.

3.4 Instruments

3.5 Data analysis

The main data in the present study were SCT, SAT, and argumentation scores from the OGSA. All the students were requested to complete SCT and SAT before and after one topic learning. All students' arguments proposed in the OGSA were collected. We applied Toulmin's theory and developed a framework for coding student dialogue argumentations in the OGSA. Each student statement was read line‐by‐line by the first author and the two teachers. Any off‐task portions of the conversation were removed. Then, we sorted the utterances into the four categories of utterances and evaluated their quality/level based on the analytical framework (Table 1). The cross‐coder reliability was constructed by two researchers and the value was 0.87. For example, if a claim included the source of authority, a relevant explanation, or (theoretical or empirical) evidence, we treated it as a Level 2 claim. Otherwise, it was deemed a Level 1 claim. In order to get a complete picture combining both the quality and quantity of the students' argumentation, total scores were calculated. Level 1 arguments were coded as 1, and Level 2 arguments were coded as 2.

4.1 Scientific Concept Test

Table 2 presents a summary of descriptive measures, the one‐factor measure analysis of covariance and post hoc test results on SCT. The analysis was conducted with the pretest of SCT as a covariate and the contextualized level as indipendent variable. The results indicated that the contextualized level had a significant effect on students' science comprehension (f₍₃₎ = 12.69, p < 0.001). Due to the variance of the data that was homogenous (f_{(3, 144)} = .068, p > 0.05), the Sidak post hoc test was used, and it showed that the students in the highly contextualized group outperformed their peers in the mildly contextualized (mean difference (md) = 2.50, p < 0.05) and the control (md = 5.12, p < 0.001) groups; the moderately contextualized group outperformed the control group (md = 2.48, p < 0.001); the mildly contextualized group outperformed the control group (md = 2.62, p < 0.05).

4.2 Scientific Attitude Test

A one‐factor repeated measure analysis of variance (ANOVA) was conducted with the pretest and posttest of SAT as the repeated factor and the contextualized level as independent factor. The analysis helped us to see the the degree of students' improvements regarding the students' science attitude and to investigate the effect of the contextualized level on the students' science attitude (Table 3). The results revealed that all the four student groups had a significant improvement after the five topics of learning (f₍₁₎ = 541.2, p < 0.001). The contextualized level had a statistically significant effect on the students' science attitudes (f_{(3, 144)} = 42.8, p < 0.001). Due to the variances of the data that were homogenous for both pretest (f_{(3, 144)} = 2.47, p > 0.05) and posttest (f_{(3, 144)} = 2.45, p > .05), the Sidak post hoc test was used. The results showed the students in the highly contextualized group outperformed their peers in the mildly contextualized (md = 9.28, p < 0.001) and the control (md = 17.42, p < 0.001) groups. The moderately contextualized group outperformed the mildly contextualized group (md = 5.15, p < 0.05) and the control group (md = 13.29, p < 0.001); the mildly contextualized group outperformed the control group (md = 17.42, p < 0.01). There were significant interactions between the two variables (f₍₃₎ = 73.25, p < 0.001). Follow‐up simple main effect indicated that the contextualized level had a statistically significant effect for the posttest of the SAT (f₍₃₎ = 97.09, p < 0.001).

4.3 Online game‐based science argumentation

Figure 3 presents a clustered bar chart generated from descriptive statistics and ANOVA data for comparing the OGSA performance of students in the highly, moderately, and mildly contextualized groups. A one‐factor repeated measure ANOVA with the topic as repeated factor and the contextualized level as independent factor was conducted to investigate the students' improvements regarding the argumentation scores (claim, warrant, rebuttal, and quesiton). It revealed that in general, all the three student groups acquired a significant improvement (Wilks' Lambda ₍₄₎ = 104.1, p < 0.001). A one‐factor multivariate analysis of variance was then conducted to investigate the differences among the three groups across the five topics. The results showed in the contextualized level had a significant effect on argumentation score in Topic 1 (f₍₃₎ = 34.60, p < 0.0001), Topic 2 (f₍₃₎ = 33.9, p < 0.0001), Topic 3 (f₍₃₎ = 22.2, p < 0.0001), Topic 4 (f₍₃₎ = 42.0, p < 0.0001), Topic 5 (f₍₃₎ = 31.9, p < 0.0001). Since the homogeneity of variance of the data is invalid for the five topics. Follow up Games‐Howell post hoc for the five topics showed that the highly contextualized group outperformed their peers in the mildly contextualized group (Topic 1, md = 2.14, p < 0.001; Topic 2, md = 5.51, p < 0.001; Topic 3, md = 4.65, p < 0.001; Topic 4, md = 9.16, p < 0.001; Topic 5, md = 9.01, p < 0.001). The moderately group outperformed than the mildly contextualized groups (Topic 1, md = 1.62, p < 0.001; Topic 3, md = 4.80, p < 0.001; Topic 4, md = 7.87, p < 0.001; Topic 5, md = 7.10, p < 0.001). The highly contextualized group outperformed their peers in the moderately contextualized group (Topic 2, md = 4.52, p < 0.001).

Table 4 presents the results of a one‐factor repeated measure ANOVA that was conducted with the four categories of utterances scores generated by the students in the five topics as the repeated factors and the contextualized level as independent factor. The results showed that students in the three groups made significant progress in terms of producing claims (f₍₄₎ = 246.6, p < 0.001), warrants (f₍₄₎ = 236.7, p < 0.001), rebuttals (f₍₄₎ = 192.7, p < 0.001), and questions (f₍₄₎ = 196.3, p < 0.001). The contextualized level had a statistically significant effect on students' claim (f₍₂₎ = 49.67, p < 0.001), warrant (f₍₂₎ = 60.33, p < 0.001), rebuttal (f₍₂₎ = 10.09, p < 0.001), question (f₍₂₎ = 13.26, p < 0.001). There were significant interactions between the two factors in the category of claim (f₍₈₎ = 15.08, p < 0.001), warrant (f₍₈₎ = 10.41, p < 0.001), rebuttal (f₍₈₎ = 3.79, p < 0.01), question (f₍₈₎ = 5.08, p < 0.01). The simple main effect analyses further showed that for the category of claim, the highly contextualized group outperformed their peers in the moderately (md = 0.92, p < 0.01) and the mildly contextualized group (md = 2.5, p < 0.001); the moderately contextualized group outperformed than the mildly contextualized groups (md = 1.58, p < 0.001). For the category of warrant, the highly contextualized group outperformed their peers in the mildly contextualized group (md = 2.03, p < 0.001); the moderately group outperformed than the mildly contextualized groups (md = 1.67, p < 0.001). For the category of rebuttal, the highly contextualized group outperformed their peers in the mildly contextualized group (md = .78, p < 0.001); the moderately contextualized group outperformed than the mildly contextualized groups (md = .60, p < .01). For the category of question, the highly contextualized group outperformed their peers in the mildly contextualized group (md = .77, p < .001); the moderately contextualized group outperformed than the mildly contextualized groups (md = .62, p < 0.001).