Ethnically diverse students' knowledge structures in first-semester organic chemistry



Chemistry courses remain a challenge for many undergraduate students. In particular, first-semester organic chemistry has been labeled as a gatekeeper with high attrition rates, especially among students of color. Our study examines a key factor related to conceptual understanding in science and predictive of course outcomes—knowledge structures. Previous research on knowledge structures has focused on differences between experts and novices. Given the increasing ethnic diversity of college classrooms and research indicating unique differences in certain higher order cognitive processes associated cultural practices and ethnicity, it is important to investigate whether similar patterns exist with respect to knowledge structures. Our study utilized concept maps to measure knowledge structures. Two separate analyses where performed to determine whether or not ethnically diverse students' organize their knowledge of organic chemistry content in structurally different ways. The first analysis utilized concept map proposition scoring to examine the influence of prior science achievement and ethnicity on knowledge structures. The second analysis examined holistic map structures to determine whether or not ethnically diverse students show qualitatively distinct structures overall. Results show significant mean differences on concept map proposition scores related to both prior science achievement and ethnic group membership. However, examination of holistic structures revealed that students' qualitative holistic structures did not vary by ethnic group membership. Taken together, our findings suggest that variation in students' knowledge structures are related to prior science achievement across ethnic groups, not qualitative differences in the ways ethnically diverse students' structure knowledge. Implications for teaching and learning in organic chemistry are discussed. © 2014 Wiley Periodicals, Inc. J Res Sci Teach 51: 741–758, 2014

On April 23, 2009, President Obama announced the “Educate to Innovate” campaign for excellence in Science, Technology, Engineering, and Math (STEM) education (Office of the Press Secretary, 2009). This nationwide campaign is an effort to strengthen the United States' role in scientific discovery and technological innovation over the next decade, in part, by expanding STEM education opportunities for underrepresented minorities. More recently, the president reiterated his commitment to STEM education and underrepresented minorities (Office of the Press Secretary, 2013). Obama's campaign is particularly relevant considering the shifting ethnic diversity in colleges and universities. Groups that are underrepresented in STEM disciplines—African Americans and Latina/os, for example—are projected to continue increasing in the general population over the next 30 years (U.S. Census Bureau, 2006). More importantly, higher numbers of students of color are enrolling in college with the goal of pursuing STEM disciplines now more than at any time in the past decade (NSF, 2011), at times surpassing their White and Asian peers (Hurtado, Eagan, & Chang, 2010).

Despite increases in enrollment rates and aspirations to pursue STEM disciplines, substantial disparities remain among graduation rates between students of color and their peers (National Academy of Sciences, 2011). A recent report by the Board on Science Education of discipline-based education research states that the research base investigating how learning might vary by important characteristics such as gender, ethnicity, and socioeconomic status is severely limited (Singer, Nielsen, & Schweingruber, 2012). If the United States is to realize its true potential and leverage the knowledge of all citizens, it is important to examine factors that contribute to academic achievement in science, and ultimately STEM completion rates. In this paper, we investigate knowledge structures as one such factor.

Knowledge structures describe the mental structure of knowledge and are related to understanding and course outcomes, thus potentially influencing matriculation rates in science. We examine the influence of prior science achievement and ethnic group membership on Asian, Latina/o, and White students' knowledge structures using concept maps.1 Overall, quantitative results show significant mean differences on concept map performance scores related to both prior science achievement and ethnic group membership. However, a separate examination of students' holistic concept map structures revealed that students predominately construct similar types of structures, regardless of ethnic group affiliation. Variation in holistic structures was associated with prior achievement in science and not ethnic membership. Our results underscore the importance of access to quality learning resources in science that provide opportunities for students to draw connections between fundamental concepts (i.e., structure their knowledge).

We start with a review of the pertinent literature on knowledge structures and its role in learning, while highlighting methodological approaches used to assess knowledge structures. Next, we discuss prior science achievement (as measured by prior science GPA) as a facet of prior understanding. We then present literature that questions the universality of cognitive processes to highlight the need to explore knowledge structures across ethnic groups. This is followed by a description our study design and data collection procedure. Finally, we present our results and a discussion of our findings in light of identifying obstacles for student achievement and equal opportunities for academic success.

Background Literature

Knowledge Structure as a Facet of Understanding and Learning

The organization of knowledge is an important aspect in many cognitive theories. Anderson (1984), an eminent cognitive psychologist, stated that the “essence of knowledge is structure” (p. 5). Studies of expertise have noted that experts' knowledge “is not simply a list of facts and formulas […] instead, their knowledge is organized around core concepts or ‘big ideas’ that guide their thinking” (Bransford, Brown, & Cocking, 2000, p. 24). As individuals develop a progressively sophisticated understanding in a domain, knowledge is based on increasingly interconnected concepts (Chi, 2011; Hmelo-Silver & Pfeffer, 2004; Rottman, Gentner, & Goldwater, 2012). Thus, to be knowledgeable in a domain suggests that one's knowledge is increasingly structured and integrated.

Knowledge structures stem from a semantic theory of memory, which builds upon an associative model of learning and memory. In an associative model, knowledge is stored and organized as a network containing concepts, ideas, and patterns that are associated (i.e., linked) based on the individual's experience (Collins & Smith, 1988). For the sake of simplicity, we shall refer to concepts, ideas, and patterns as elements of knowledge (White & Gunstone, 1992). The strength of the link is dependent, for example, by the frequency with which the link is traversed (Shavelson, Ruiz-Primo, & Wiley, 2005). The stronger the links between elements of knowledge, the more likely an individual can recall a specific piece of information. A semantic theory of memory builds upon the associative perspective by making explicit the relationships (i.e., links) between elements of knowledge.

Both associative and semantic models of memory have been highly influenced by domains such as cognitive psychology and artificial intelligence. For example, Ausubel's (1968) learning theory proposes that learning takes place when the individual assimilates new information within their prior existing knowledge structure (also known as cognitive structure). Tapping an individual's knowledge structure is thought to provide a visual representation of the structure of their knowledge, and thus their understanding. More recently, Novak (2010) suggests that meaningful learning occurs when the learner chooses to relate new information to his/her prior knowledge. Novak goes on to state that the quality of meaningful learning is “dependent upon the conceptual richness of the new material to be learned and the quantity and quality of the organization of the relevant knowledge held by the learner” (p. 23). Knowledge structures, then, are an important aspect of conceptual understanding and meaningful learning.

In a groundbreaking study, Chi, Feltovich, and Glaser (1981) examined experts' and novices' categorizations of physics problems and knowledge structures. Advanced physics PhDs served as experts, while physics undergraduates served as novices. Participants' knowledge structures were assessed using a sorting procedure in which participants were given 20 cards containing physics problems where superficial and conceptual information was crossed. Findings revealed that novices' representations were complex and involved many labels; however, the large majority of labels were based on problem surface features (e.g., Incline Planes, Block, Surface Property). Conversely, experts' knowledge was organized around underlying conceptual principals (e.g., Conservation of Energy, Newton's Force Laws) and then on superficial features. This study illuminated key expert-novice distinctions in the structures of participants' knowledge and further increased interest in understanding cognitive processes related to learning.

These studies support knowledge structures as an important facet of understanding and meaningful learning. As students' understanding increases, so too does the structure of their knowledge. Therefore, developing tools to capture the structure of knowledge is important.

Tapping Knowledge Structures Through Concept Mapping

Various methods have been used to investigate knowledge structures such as problem solving using think-aloud protocols and/or explanatory essays, card-sorting exercises, and concept mapping (for a review, see Jonassen, Beissner, & Yacci, 1993). Problem solving with the use of think-aloud protocols and/or explanatory essays provides detailed information but is difficult to administer across multiple concepts, students, and time-points. Card-sorting techniques are relatively easy to administer but require that users infer knowledge structures from students' sortings. Although useful under specific conditions, card-sorting exercises are an indirect method of measuring knowledge structures.

Concept maps, a third commonly used technique, are thought to provide a visual snapshot of an individual's knowledge structure (Singer et al., 2012). A concept map is a graphical tool for organizing and representing one's knowledge and is intended to represent meaningful relationships between concepts in the form of propositions (i.e., semantic unit) (Novak & Gowin, 1984). Concept maps often include three components: concept terms, linking arrows, and linking phrases (see Figure 1). Concept terms are key ideas and/or concepts in a domain. Linking arrows provide a directional relationship between two concepts. Linking phrases represent the specific relationships between a pair of terms (for a review of the theoretical underpinnings, see Novak, 2010). For example, the student-generated map in Figure 1 shows a student's understanding of the relationships between an Sn1 reaction and the solvent in which the reaction occurs (Sn1 reaction → requires a polar protic → solvent). In this study, concept map terms were not limited to concepts, but also included factual and procedural terms. This was done to obtain a richer picture of a student's integration of factual, procedural, and conceptual terms in their knowledge structures.

Figure 1.

Student-Generated Concept Map for Learning Unit 3 (Perram).

Assessing Knowledge Structures in Science Education: The Use of Concept Maps

Concept maps have a long history in science education that spans over four decades (Novak & Cañas, 2007). Overall, research examining the use of concept maps to infer knowledge structures have shown that they can be a valid and reliable technique. In addition, more complex and integrated structures are related to deeper understanding.

Ruiz-Primo and Shavelson (1996) performed an in-depth review of the literature pertaining to the technical characteristics of concept maps. The authors reported several estimates of reliability such as inter-rater agreement, inter-rater reliability, and stability coefficients. Of particular interest here, this study clearly indicates that concept maps can be reliably scored, especially when using trained scorers (the majority of coefficients were greater than 0.8). McClure, Sonak, and Suen (1999) reported that the highest reliability coefficients were associated with scoring methods that examined individual propositions. With respect to validity, researchers have demonstrated that content validity can be sufficiently achieved when subject matter experts are consulted to judge the representativeness and accuracy of terms (Ruiz-Primo & Shavelson, 1996). Scholars have also shown evidence toward the concurrent validity of using concept maps as a measure of students' knowledge structures and conceptual understanding in science education (Markham, Mintzes, & Jones, 1994). More recently, researchers have indicated that construct-a-map type concept maps best reflect differences between students' knowledge structures and uncover students' partial knowledge as well as naïve conceptions, compared to partially completed maps (fill-in-the-map) (Yin, Vanides, Ruiz-Primo, Ayala, & Shavelson, 2005). Concept maps, therefore, can serve as a reliable and valid evaluation tool to infer knowledge structures, especially when employing proposition scoring and consulting subject matter experts.

Another benefit of concept maps is that they can be used to capture the rich set of relationships that exist in a students' mental space. Wilson (1998) used concept maps to examine the different degrees of conceptual knowledge representation among grade 12, undergraduate, and postgraduate chemistry students after completing a unit on acid/base equilibrium. Students constructed concept maps using 22 acid–base concept labels and maps were analyzed using proposition scoring. Results showed that students' level of differentiation, degree of connectivity among nodes, and the use of abstract concepts related to chemical processes increased with schooling. Wilson concluded that knowledge becomes more organized and interrelated with increasing expertise in a domain, corroborating previous expert-novice results in other domains such as biology (Dauer, Momsen, Speth, Makohon-Moore, & Long, 2013; Pearsall, Skipper, & Mintzes, 1997), chemistry (Francisco, Nakhleh, Nurrenbern, & Miller, 2002; Ruiz-Primo, Schultz, Li, & Shavelson, 2001), engineering (Segalàs, Ferrer-Balas, & Mulder, 2008), and medical education (Daley & Torre, 2010; West, Pomeroy, & Park, 2000).

Specifically in organic chemistry, Coppola, Ege, and Lawton (1997) have used concept maps as an assessment tool to gauge student learning between two instructional conditions. A randomly selected group of students enrolled in a novel instructional course (n = 22) and a traditional lecture course (n = 20) completed concept maps on the concepts and practices used to identify an unknown chemical material. Students' concept maps were compared to expert maps, which included three organic chemistry faculty and two upper-level graduate students. Of particular interest here, authors showed that concept maps were systematically able to distinguish between students' degrees of understanding in both conditions. Overall, concept maps produced by students in the novel condition more closely aligned with expert-generated maps with respect to concept and procedure integration and order.

More recently, researchers used concept maps to investigate changes in students' understanding of gene-to-evolution models in undergraduate biology (n = 368) (Dauer et al., 2013). Student-generated concept maps were studied for complexity and correctness, which the authors define as the mean scientific accuracy of propositions within students' maps. Using a linear mixed effects model, the authors found that students generated more sophisticated concept maps as the semester progressed. In addition, students increasingly generated more parsimonious maps that included scientifically accurate language to explain relationships between concept map terms. The authors attribute the positive changes in students' gene-to-evolution models to accretion, tuning, and reorganization of schemas captured through concept maps.

In sum, concept maps are a reliable and valid method to infer an individual's knowledge structures. Maps also serve as a powerful tool to differentiate between students' varying degrees of understanding and uncover partial knowledge. These studies also implicitly highlight the importance of prior domain knowledge through their comparisons of experts or advanced students and novices. Novak states that the “fundamental characteristic of meaningful learning is integration of new knowledge with the learners' previous concept and propositional frameworks […]” (Novak & Cañas, 2006, p. 28). Since students do not enter chemistry classes as empty vessels to be filled, investigations of knowledge structures need to consider the role of prior science understanding as a factor that can affect knowledge structures. Currently, little is known about the role of prior science achievement and knowledge structures across ethnically diverse students. We aim to contribute to the discussion of students' academic achievement in undergraduate chemistry by pursuing this line of inquiry.

Prior Science GPA as a Proxy for Prior Science Understanding

There is a saying that goes, “the best predictor of future achievement is prior achievement.” That is, we know that prior achievement is associated with future achievement. This is true in science and in most domains (Bransford et al., 2000). We view science achievement as a broad construct that not only includes performance on standardized measures but also affective factors, classroom outcomes, and opportunities to participate in activities related to science.

Although scholars have questioned the role of GPA as a measure of understanding, our own research using concept maps has demonstrated a significant relationship between concept map performance and course grades. For example, Szu et al. (2010) investigated factors related to student performance in undergraduate organic chemistry. Using a mixed methods approach, the authors' found that final course grades significantly predict concept map performance (R2 = 0.57). Since GPAs are composed of course grades, it seems logical that prior science GPA captures a facet of prior domain knowledge.

The Influence of Race/Ethnicity and Cultural Practices on Cognitive Processes

Although prior science understanding is thought to affect knowledge structures, it is unclear to what extent differences exist across racial and ethnically diverse students. To date, studies have predominately focused on differences between experts or advanced students and novices. The increasing ethnic diversity of college classrooms underscores the importance of examining knowledge structure patterns across ethnic groups. To our knowledge, this is the first study to do so.

Before proceeding, it is important to distinguish race/ethnicity and cultural practices. Although related, we do not conflate these constructs. We view race/ethnicity as a socially constructed concept. However, following Gutierrez and Rogoff, we also take the position that “group membership defined by ethnicity, race, and language use is relevant.” As they argue, “these categories have long-standing influences on the cultural practices in which people have the opportunity to participate, often yielding shared circumstances, practices, and beliefs that play important and varied roles for group members” (Gutiérrez and Rogoff, 2003, p. 21). We view cultural communities as groups of individuals that share a fluid set of common practices and shared history of participation. For our purposes, we attribute variation in science achievement across ethnic groups to historical opportunities to participate in science activities such as science courses, not individuals' biology. As Gutiérrez and Rogoff (2003) remind us, “variations reside not as traits of individuals or collections of individuals, but as proclivities of people with certain histories of engagement with specific cultural activities” (p. 19). Scholars continue to highlight the inequitable distribution of resources and opportunities to participate in scientific cultural activities for many students of color (Lee & Buxton, 2010), which can ultimately lead to differences in the acquisition of content knowledge and subsequent achievement.

To our knowledge, no studies have been published examining the relationship between ethnicity and knowledge structures making direct connections between our exploratory study and prior research challenging. Scholars interested in social and cultural influences on learning have noted disconnects between the culture of school science and students' out-of-school cultural practices (Aikenhead & Jegede, 1999; Costa, 1995). Although informative, this line of research does not necessarily focus on cognitive processes. Cross-cultural cognition scholars have found unique differences in cognitive processes such as attention, recall, and problem-solving reasoning across cultural groups (Chua, Boland, & Nisbett, 2005; Nisbett, Peng, Choi, & Norenzayan, 2001). It is important to note that this line of research has historically focused on Western and non-Western individuals. One of the foci of our study was ethnicity, not Western vs. non-Western contexts. Cross-cultural studies, however, do provide evidence to question the universality of cognitive processes and provide a reason to explore other important processes such as knowledge structures.

In order to provide a rationale for investigating potential knowledge structure differences across ethnic group, it is useful to step out of the realm of science education. We draw from two studies that have shown mixed results with respect to racial/ethnic and cultural differences on cognitive processes. Baxter, Shavelson, Herman, Brown, & Valadez (1993) studied the performance of racial and ethnically diverse sixth grade math students. The authors qualitatively compared students' written responses to mathematical problems and examined responses based on problem-solving approaches, errors made, and strategies used to solve problems. The authors found that White and Latina/o students approached problems similarly, made similar errors, and used similar strategies to solve math problems. Rather, authors attribute variation in mathematical performance to students' curricular experience (i.e., whether taught using a traditional lecture or hands-on instruction).

More recently, researchers have found unique differences in cognitive processes related to thinking among Chinese individuals residing in different regions. Talhelm et al. (2014) studied 1162 Han Chinese cultural thought patterns using a paired-items task, similar to a task used in Luria's classic study (1979). Tasks were used to infer holistic and analytic thinking patterns among participants. Using regression analyses, the authors found that southern Chinese displayed more holistic thinking patterns and northern Chinese more analytic. The authors attribute differences to rice versus wheat agricultural practices that influenced thinking and behavior patterns over time.

We present these studies to highlight the tentative nature of researchers' knowledge of individual's cognitive processes within U.S. and non-U.S. contexts. Given the positive relationships between knowledge structures and learning, course performance and achievement, it is important to explore potential differences across ethnically diverse students in science. We pursue this line of inquiry by exploring the following research questions:

  1. What is the influence of prior science achievement on knowledge structures?
  2. What is the influence of ethnic group membership on knowledge structures?
  3. Do differences exist among Asian, Latina/o, and White students' holistic knowledge structures?

We would like to explicitly state that the goal of our study is not to essentialize ethnic groups' ways of knowing and learning. This study is not designed to explore those inquiries. Rather, our goal is to examine whether (or not) ethnic groups construct different knowledge structures so that they can be better served with respect to learning.


Study Context and Participants

First-semester organic chemistry was selected as a point of examination because this course has been singled out as a “weeder” course with high attrition rates (Grove, Hershberger, & Bretz, 2008; Paulson, 1999) and has led students to question their ability to succeed in science (Barr, Gonzalez, & Wanat, 2008). Organic chemistry is also a mandatory course for students' aspiring to matriculate in a variety of science disciplines and is highly influential in acceptance to science-related professional schools. Thus, organic chemistry can create a bottleneck for students' interested in science disciplines and stand as a barrier to diversifying the STEM workforce.

Students were recruited during their initial organic chemistry lecture. Researchers gave a brief presentation discussing the purpose and requirements of the study. Interested students completed a questionnaire requesting information on gender, ethnicity, and consent to access official transcripts. This information was used to stratify students by gender and ethnicity in order to maintain proportional representation, as much as possible. In total, 90 students participated—41 Asian, 32 White, and 17 Latina/o; 48 female and 42 male.

Students were classified by their self-labeled ethnicities into one of three groups: Asian, Latina/o, or White. Categories were created based on the majority self-labeled ethnicity (e.g., 40 out of 41 students self-labeled as Asian). In order to double check the potential influence of individuals that did not label themselves specifically as Asian, Latina/o, or White (e.g., one student self-labeled as Vietnamese, rather than Asian), analyses were performed with and without these individuals to examine potential changes in our statistical results. No differences were observed.

Concept Map Development

Construct-a-map type concept maps were used to assess knowledge structures, which are a more sensitive technique for capturing differences among students' knowledge structures and naïve conceptions (Yin et al., 2005). Map terms were constructed over several iterations as follows. Six commonly used organic chemistry textbooks were used to identify key terms within each learning unit. We refer the reader to Supporting Information Table 1, for textbooks used to identify concept map terms. Separately, two organic chemistry professors compiled a list of terms they believed to be fundamental for the same learning unit. Textbook and professor-created terms were cross-tabulated and overlapping terms were included. The draft concept term list was sent to organic chemistry professors for feedback. Revisions were made as necessary until a minimum of 10 and a maximum of 14 terms were agreed upon. The same process was used to complete subsequent learning units. In total, learning units 1, 2, 3, and 4 contained 14, 10, 12, and 11 concept map terms, respectively. A list of final concept map terms by unit is presented in Table 1.

Table 1. Concept Map Terms by Learning Unit (in Alphabetical Order)
UnitTopicConcept Map Terms
1Structure and bondingBond Angle, Covalent Bond, Electronegativity, Formal Charge, Hybrid Orbital, Ionic Bond, Lewis Structure, Molecular Orbital, Octet Rule, Organic Molecule, Pi Bond, Polar Covalent Bond, Resonance Form, Sigma Bond
2StereochemistryChiral, Chirality Center, Cis-trans Isomer, Diastereomer, Enantiomer, Meso Compound, Optically Active, R/S Configuration, Racemic Mixture, Stereoisomer
3Alkyl halide reactionsAlkyl Halide, Base Strength, Carbocation, E1 Reaction, E2 Reaction, Leaving Group, Nucleophile, Steric Hindrance, Stereospecific, Solvent, Sn1 Reaction, Sn2 Reaction
4Reactions of alkenesAddition Reaction, Carbocation, Double Bond, Electrophile, Hydrogenation, Hydroboration, Markovnikov's Rule, Nucleophile, Oxymercuration, Regioselectivity, Stereospecific

Scoring and Coding Maps

Concept maps were analyzed using two separate approaches. The first analysis examined the influence of prior science achievement across ethnic groups on students' concept map performance by scoring the scientific accuracy of propositions within concept maps. Propositions are thought to be the smallest unit of meaning that can be used to judge the relationship between two terms and its linking phrase (Ruiz-Primo & Shavelson, 1996). Two organic chemistry PhD candidates scored student-generated concept map propositions on scientific accuracy using the following four-level ordinal scale (with underlying continuum): 0—incorrect or scientifically irrelevant, 1—partially incorrect, 2—technically correct, but scientifically “thin” or vague, and 3—scientifically correct and scientifically stated (cf. Yin et al., 2005).

Scorers were trained to accurately and reliably score student-generated propositions during the training phase. Scorers were given a subset of overlapping propositions (40% overlap) from student-generated practice maps to score. Next, researchers and scorers came together to discuss proposition scores. Once the scorers demonstrated that they could reliably score practice maps, they were provided with actual student-generated maps used in our analyses. Both organic chemistry PhD candidates scored every unique student-generated proposition. Scored propositions were then entered into a master Microsoft Excel list that was used to automatically score the same propositions in other maps within the learning unit. Scorers continued to mark unique student-generated propositions not already found, while the master list automatically scored previously identified propositions. After each learning unit was marked, scorers came together to discuss any discrepancies in scores. Scorers were asked to justify their scores and come to consensus, if possible. In the event that scorers could not come to a consensus a third organic chemistry PhD candidate was consulted as a tiebreaker. We calculated the inter-rater agreement between scorers to estimate the extent to which scorers provided similar marks on propositions. Inter-rater agreement ranged between 0.77 and 0.91 across learning units (averaged 0.84).

Although scoring proposition provides quantitative information about the scientific accuracy of students' knowledge structures, it does not allow the overall qualitative examination of students' holistic map structures. It is possible that quantitative differences may stem from qualitative structural differences. For example, concept maps can receive similar quantitative scores yet show distinct differences in their holistic structures. Without observing the actual structures it would be difficult to know whether possible ethnic group differences were related to quantitative scores, holistic structure, or both. Therefore, a second, and separate, analysis examined students' holistic map structures to determine whether or not ethnically diverse students show qualitatively distinct structures overall.

A similar methodology used by Kinchin, Hay, & Adams (2000) was used to classify concept maps into three categories. Concept map classifications included—linear, radial, and network structures (see Table 2). Linear maps display a linear connection from one proposition to the next. Radial maps are identified by the use of a central term (e.g., the use of Covalent Bond in Table 2) from which links emanate to other key terms. Finally, network maps display several interconnections between terms. Occasionally concept maps displayed the structural characteristics of two categories. In such cases, scorers were asked to categorize maps based on the predominate structure. Inter-rater agreement ranged between 0.79 and 0.86 (average 0.83) across the four learning units.

Table 2. Classifications and Definitions Used to Categorize Concept Maps
Concept Map Holistic CategoriesDefinitionConcept Map Holistic Structure Example
LinearPropositions are chained in a linear fashiontea21160-gra-0001
RadialPropositions emanate from a central concept map termtea21160-gra-0002
NetworkPropositions are connected via a complex set of interconnected termstea21160-gra-0003

Concept Map Analyses

The first step was to establish whether or not ethnic groups differed with respect to the scientific accuracy of their knowledge structures. A 2 × 3 × 4 (prior science GPA [high, low] × ethnicity [Asian, Latina/o, White] × learning unit [units 1, 2, 3, 4]) split-plot ANOVA with repeated measures on the learning unit variable was performed to address research questions 1 and 2. Students whose prior science GPA ≥3.0 were considered relatively high achievers and students with a prior science GPA ≤2.8 considered relatively low achievers. A gap was intentionally left between GPAs in order to take into account the difficulty of assigning a cut-off point. Finally, research question 3 was investigated by analyzing patterns of students' holistic structures (i.e., linear, radial, and network) by ethnic group membership through the use of chi-squared tests.


Data collection took place over two separate semesters (Fall 2009 and Spring 2010) and covered four strategic learning units previously and unanimously identified by instructors as important to the course: (1) Structure and Bonding, (2) Stereochemistry, (3) Alkyl Halide Reactions, and (4) Reactions of Alkenes. Students were assessed during out-of-class interview sessions. All sessions were procedurally similar, however, session 1 served as an introduction to the study and consequently was twice as long. Also, session 1 occurred 2 weeks into instruction, while the remaining interviews occurred up to 1 week after that topic was covered in class.

The interview began by introducing students to concept maps and describing the components of a concept map (e.g., key terms, linking arrows, and linking phrases). Students were given instruction on how to properly complete a concept map, while the interviewer talked through a completed example. Following this explanation, students were asked to complete a practice map on the water cycle with interviewer feedback. The practice map included seven terms that did not overlap with organic chemistry concept map terms. After this training period, students were presented with a laptop containing a list of concept map terms and asked to create a concept map that best reflected their understanding of the relationships between the terms. It should be noted that concept maps were not part of students' course assessments and completed for research purposes only. Although seldom reached, a 30-minute time limit was used to encourage completion of maps. In total, each student completed four concept maps (one per learning unit). Table 3 provides a breakdown of the total number of students completing concept maps across learning units.

Table 3. Students Completing Concept Maps by Ethnicity Across Learning Units
EthnicityUnit 1: Structure and BondingUnit 2 StereochemistryUnit 3 Alkyl Halide ReactionsUnit 4 Reactions of Alkenes


Our findings showed a significant difference on mean concept map scores among ethnic groups. Upon closer examination, both prior science achievement and ethnicity were related to concept map proposition scores. Analysis of holistic concept map classifications, however, showed no statistical differences among ethnic groups on three out of four learning units. These results are unpacked below.

Research Question 1: What Is the Influence of Prior Science Achievement on Knowledge Structures?

Given the significant mean differences on concept map scores among ethnic groups, the next step was to investigate whether prior science achievement and/or ethnic group membership was attributable to map differences. Some preliminary analytic work was needed before directly interpreting the ANOVA results. The issue of whether students enter organic chemistry with statistically different prior science GPAs had to be addressed. A one-way ANOVA using ethnicity as the independent variable and prior science GPA as the dependent variable showed a significant difference between groups, F(2,87) = 3.21, p < 0.05. Tukey's post hoc tests showed that White students entered organic chemistry with higher mean prior science GPAs (mean = 3.02, SE = 0.11) than Latina/o students (mean = 2.56, SE = 0.16). No significant differences were observed between Asian (mean = 2.74, SE = 0.10) and Latina/o or Asian and White students' mean prior science GPAs.

The 2 × 3 × 4 (prior science GPA [high, low] × ethnicity [Asian, Latina/o, White] × learning unit [units 1, 2, 3, 4]) split-plot ANOVA with repeated measures on unit showed two main effects: prior science GPA and ethnicity (not unexpectedly given results just presented). A significant effect of prior science GPA on mean concept map proposition scores was observed, F(1,57) = 4.29, p < 0.05. On average, this result shows that students entering organic chemistry with a high prior science GPA received higher concept map scores (mean = 19.25, SE = 1.97) compared to students with low prior science GPAs (mean = 13.86, SE = 1.71). This is particularly relevant considering the significant correlation between prior science GPA and concept map performance (r = 0.52).

Research Question 2: What Is the Influence of Ethnic Group Membership on Knowledge Structures?

Students' mean concept map scores across all units were 17.03 (SE = 1.02). The second main effect showed a significant difference on mean concept map proposition scores among ethnic groups, F(2,57) = 4.04, p < 0.05. Tukey's post hoc tests showed that White students' mean concept map proposition scores (mean = 21.30, SE = 2.03) were significantly higher than Latina/o students' scores (mean = 11.40, SE = 2.88). No statistical differences were observed between Asian (mean = 16.96, SE = 1.68) and Latina/o or Asian and White students' mean scores.

No significant mean differences were found across learning units. Also, no significant two- or three-way interactions were found between the variables (ethnicity, prior science GPA, and unit). The lack of a significant interaction between ethnicity and learning unit indicates that mean rank order of ethnic groups was maintained across the four learning units.

Taken together, our findings suggest that concept map performance (as scored by proposition accuracy) was attributable to prior science achievement and ethnic group membership. In addition, a pattern emerged with respect to mean differences for both prior science GPA and concept map proposition scores showing a consistent rank ordering of White, Asian, then Latina/o students across learning units.

Research Question 3: Do Differences Exist Among Asian, Latina/o, and White Students' Holistic Knowledge Structures?

Chi-squared tests showed no significant relationships between concept map holistic structures and students' ethnic group affiliation for units 1, 3, and 4 (math formula [4] = 6.75, p > 0.05; math formula [4] = 2.66, p > 0.05; math formula [4] = 3.59, p > 0.05), respectively. However, a significant difference among groups was found on unit 2 (Stereochemistry) (math formula [4] = 11.90, p < 0.05). On average, White students created significantly more network structures than Latina/o students.

In order to rule out the potential confounding affect of prior science achievement, a separate chi-squared test was conducted replacing ethnicity with prior science GPA as a predictor of students' holistic concept map structures on unit 2. High achievers were considered students whose prior science GPA ≥3.0 and low achievers were considered students whose prior science GPA ≤2.8. No significant differences were observed between holistic structures and prior science achievement, providing further support that ethnic groups structure their knowledge differently on unit 2. However, this finding should be interpreted with caution given the lack of differences on units 1, 3, and 4 and the relatively low sample size of Latina/o students in our study. Further data are needed to provide additional insight.

Unit 2 (Stereochemistry) was the only learning unit that displayed a significant difference in holistic structures among ethnic groups, even when taking into account prior science GPA. Given unit 2's focus on spatial visualization, an alternative explanation that may account for differences may be related to students' visual-spatial abilities. Prior investigations have noted the association between students' ability to mentally manipulate three-dimensional structures in chemistry and measures of understanding (Bunce & Gabel, 2002; Wu & Shah, 2004). Although not the focus of this paper, and hence not explained as part of the methodology, students completed the Purdue Visualization of Rotations Test as part of their initial interview (for details, see Bodner & Guay, 1997), a previously validated visual-spatial assessment (for sample, visual-rotation questions, see Supporting Information Figure 1). A one-way between subjects ANOVA was computed to investigate possible mean spatial visualization differences between ethnic groups. No significant differences were found: F(2,89) = 1.564, p > 0.05. Thus, spatial visualization is not likely to be the limiting factor accounting for the observed holistic concept map structure differences between ethnic groups on unit 2. Further research is needed to clarify the relationship between concept map structures and ethnic background in the domain of Stereochemistry.

In sum, our results show that ethnic groups differed in their mean performance on concept maps (when propositions where scored by scientific accuracy). Differences were attributable to both prior science achievement and ethnic group membership. A rank order pattern was observed in which White students' received significantly higher mean concept map scores compared to Latina/o students. Asian students' performance fell in between. This pattern remained consistent across all learning units. Analysis of concept map structures indicated that ethnic group membership is not a significant predictor of students' holistic structures, with the exception of unit 2—Stereochemistry. These findings highlight important theoretical and practical implications with respect to teaching and learning in first-semester organic chemistry.


Analyzing students' concept map propositions by scientific accuracy allowed us to observe differences among ethnic groups and the influence of prior science GPA on concept map performance. Although ethnic group membership and prior science GPA was investigated as separate factors they could not be disentangled statistically using analysis of variance. This is aligned with prior access and participation studies in education and science education (Darling-Hammond, 2004; Lee & Buxton, 2010). Examination of holistic concept map structures showed that Asian, Latina/o, and White students displayed similar counts of linear, radial, and network structures with the exception of unit 2. It is difficult to say why unit 2 (Stereochemistry) showed unique differences in holistic structures since none of our variables were attributable to the variation between groups. Further research is needed to explore this area. Our findings are particularly interesting because previous research investigating cognitive processes across ethnic and cultural groups has produced mixed results. We show that holistic structures are associated with prior science GPA (a proxy for prior science understanding), rather than ethnic group membership. Given the exploratory nature of our study, these findings are suggestive rather than conclusive.

It is well established that many students traditionally classified as underrepresented in science face unequal and inequitable science learning opportunities both at K-12 and post-secondary levels. Oakes (1990) highlighted the inequities in access to educational resources for students from underrepresented backgrounds at all levels. Oakes showed that African American and Latino students' predominately attend inner city schools where tax dollars are highly competitive. This leads to a lack of funding to purchase science equipment, less AP courses, and less financial resources to hire highly qualified teachers and counselors, to name a few. Over two decades later, Lee and Luykx (2007) revisit and extend this topic showing that the same patterns of inequities still exist and are related to underrepresented students' decisions to remain in science disciplines and pursue science careers.

For students who persist through these tough obstacles, this may translate into poorer chemistry content knowledge, and ultimately knowledge structures at post-secondary levels. This explanation is further supported when examining students' prior science course grades and concept map proposition scores. On average, White students entered organic chemistry with higher grades in prior science courses and were also more likely to receive higher marks on concept maps compared to Latina/o students, whereas Asian students fell in between.

Our results underscore the important relationship between prior science understanding (as measured by prior science GPA) and opportunities for students to structure their knowledge of relevant chemistry content. A key goal of discipline-centered post-secondary science education is to better understand the “nature and development of expertise” (Coppola & Krajcik, 2013; Singer et al., 2012). As we have pointed out, knowledge structures are an important facet in the development of a conceptual understanding, and thus expertise. As students develop a more conceptual understanding in a domain, the structure of their knowledge becomes more interrelated (Shavelson et al., 2005) and increasingly resembles the structure of experts' knowledge (Nash, Liotta, & Bravaco, 2000). Thus, explicit opportunities for students to engage in practices that help structure knowledge can aid all students in developing a deeper understanding, particularly important in high-stakes gatekeeper courses such as first-semester organic chemistry.

Recitation sections provide a valuable opportunity to engage students in developing connections between important chemistry concepts. One such strategy was highlighted here—concept mapping. Recitation instructors can identify key concepts in each learning unit and ask students to create a concept map that reflects their understanding of the relationships between concepts. Instructors can then analyze students' structures using various coding schemes depending on the level of detail of information sought (Kinchin et al., 2000; Yin et al., 2005). An added benefit of using concept maps is that they can also uncover gaps in students' knowledge. Although some concept map training is necessary upfront, the payoff of information can far outweigh the initial time investment. This is especially important early on considering evidence suggesting conceptual changes are more likely to occur within the first 4 weeks of class (Pearsall et al., 1997).

Whether scored by proposition or categorized by holistic structure concept maps provide an explicit opportunity for students to reflect and draw connections between important ideas/concepts in a domain, while making visible the structure of students' knowledge. The main goal is to assess what students know (or do not know) and adjust instruction as needed.

A final caveat: studies investigating differences in academic achievement typically label students as underrepresented minorities (URMs) and non-underrepresented minorities (non-URMs). The lack of statistical significance on concept map proposition scores between Asians and Latina/os suggests that including Asian and Caucasian students in the non-URM category can mask important differences in concept map performance and promote racial stereotypes (e.g., the model minority myth). For example, many Hmong, Cambodian, and Vietnamese students encounter similar barriers to rigorous science learning opportunities that African American and Latina/o students encounter. Dichotomizing students into these broad categories may have detrimental effects by justifying the exclusion of certain ethnicities to interventions and tutorials that help students' achieve academic success in science (Lee & Buxton, 2010). Thus, caution should be taken when interpreting results from studies categorize students as either URM or non-URM.

In conclusion, our findings highlight prior science GPA as a factor related to concept map performance, rather than unique differences associated with ethnicity. To our knowledge, this is the first study providing empirical evidence toward the examination of students' knowledge structures across ethnic groups. Our study also emphasizes the importance of structuring undergraduate chemistry education such that students are provided with opportunities to reflect and structure their prior knowledge with their current content knowledge. If we are serious about improving undergraduate education in science, it is more important than ever that instructors support opportunities for students to structure their knowledge, especially in high-stakes gatekeeper courses like first-semester organic chemistry.

Limitations and Directions for Future Research

As with all research, we attempt to convey the most reliable and valid data possible. In order to honor this commitment, we would like to acknowledge aspects of this study that can be improved upon in future research. First, although ethnicity can serve as a useful variable to understand general similarities and differences among groups, future studies should aim to examine socio-cultural factors and interactions that can provide a richer understanding of how students' environment potentially shapes cognitive processes, such as knowledge structures. Therefore, we situate our conclusions within our study context and student sample.

Second, methodologically, this study incorporated techniques common in cognitive research that focuses on the individual. Given the exploratory nature of our study, future work should leverage cognitive and social and cultural factors to provide a richer description of similarities and differences. Doing so can illuminate individual and environmental factors that are important to students' learning and academic achievement.

Lastly, our study used prior science GPA as a proxy for prior science achievement. GPA is a composite score that takes into account numerous variables. Some of these variables directly tap domain knowledge (e.g., mid-term and final exams), while others do so indirectly (e.g., course attendance and participation). Therefore, future work can benefit by incorporating direct measures of prior science knowledge whenever possible, such as concept inventories. Doing so may provide a more accurate assessment of impact of domain knowledge on students' knowledge structures. Future studies that build on these limitations are likely to produce more detailed information that can be used to improve teaching and learning in chemistry and science in general.


1Too few African American students were enrolled in organic chemistry during our data collection period (n = 3). Due to the likelihood of biasing statistical data because of a small sample size, African American student data were not included in our analyses.