Comprehension and navigation of networked hypertexts
Abstract
This study aims to investigate secondary school students' reading comprehension and navigation of networked hypertexts with and without a graphic overview compared to linear digital texts. Additionally, it was studied whether prior knowledge, vocabulary, verbal, and visual working memory moderated the relation between text design and comprehension. Therefore, 80 first‐year secondary school students read both a linear text and a networked hypertext with and without a graphical overview. Logfiles registered their navigation. After reading the text, students answered textbased multiple choice questions and drew mindmaps to assess their structural knowledge of each text content. It was found that both textbased and structural knowledge were lower after reading a networked hypertext than a linear text, especially in students with lower levels of vocabulary. Students took generally more time to read the hypertext than the linear text. We concluded that networked hypertexts are more challenging to read than linear texts and that students may benefit from explicit training on how to read hypertexts.
Lay description
What is already known about this topic:
- Reading comprehension of adults is lower in networked hypertexts than in linear texts.
- Less is known about comprehension and navigation of networked hypertexts in secondary school students.
- Individual skills (prior knowledge, vocabulary, verbal, and visuospatial working memory) are related to comprehension of hierarchical hypertexts.
- Less is known about the influence of individual skills on students' comprehension of networked hypertexts.
What this paper adds:
- Comprehension of networked hypertexts was lower than of linear digital texts.
- Secondary school students took more time to read a hypertext than a linear digital text.
- Secondary school students with lower vocabulary particularly were negatively affected by the networked structure.
Implications of study findings for practitioners:
- Teachers need to be aware that reading a networked hypertext is more challenging than a linear text.
- Explicit practice with reading a networked hypertext can be helpful for those students.
1 INTRODUCTION
Digital texts on the internet are being used widely to gain knowledge. The majority of these texts has hyperlinks that are interconnected as a network (networked hypertexts), whereas some have hyperlinks being connected in a hierarchy (hierarchical hypertexts). Research in adult readers has shown that networked hypertexts, but not hierarchical hypertexts, are more difficult to comprehend than linear digital texts (Destefano & Lefevre, 2007) and that navigation in networked hypertexts is related to higher feelings of disorientation (Müller‐Kalthoff & Möller, 2006). It has also been found that a graphic overview of the text structure can facilitate hypertext reading (Salmerón, Cañas, Kintsch, & Fajardo, 2005), particularly when the reader's prior knowledge is low (Amadieu, Tricot, & Mariné, 2010). Less is known about hypertext reading in secondary school students. The few studies that have been conducted on hierarchical hypertexts did not find any differences in comprehension from linear texts (Klois, Segers, & Verhoeven, 2013, 13‐year‐olds; Schwartz, Andersen, Hong, Howard, & McGee, 2004, 9–17 year‐olds) and evidenced vocabulary as main predictor. However, to our knowledge, no study has focused on secondary school students' comprehension and navigation of networked hypertexts with and without a graphic overview. The goal of this study was thus to compare reading comprehension and navigation of linear digital text versus networked hypertext with and without a graphical overview in proficient student readers and to explain their individual variation.
1.1 Hypertext comprehension
There is large variation in the structure of digital texts on the internet, ranging from linear digital texts that can be read from the computer screen in a linear way to hypertexts containing hyperlinks that can be read by navigating through its constituent hypertext parts. The underlying structure of a hypertext can be hierarchical with hypertext pages being linked in a hierarchy or networked with hypertext pages being connected by semantically based, and thus less structured, hyperlinks. Given its structure, a hypertext forces the reader to actively decide on the reading path to be followed and can thus be seen as highly self‐regulated (Charney, 1994).
Two opposing views on hypertext reading comprehension can be formulated. On the one hand, it is assumed that the features of hypertext are advantageous, because the reader has to play an active role in the reading process. This rationale can be linked to cognitive flexibility theory (Spiro, Coulson, Feltovich, & Anderson, 1988), with a focus on how learning in complex and ill‐structured domains takes place with continuous shifts between so‐called conceptual landscapes (Spiro & Jehng, 1990). Spiro, Feltovich, Jacobson, and Coulson (1992) suggested that hypertext structures increase opportunities to use our knowledge in a flexible way to increase comprehension. It is assumed that navigating through a hypertext helps expanding the connections between its information nodes leading to deeper‐level text comprehension (Müller‐Kalthoff & Möller, 2006). On the other hand, it is claimed that reading a hypertext involves many simultaneous activities including navigation, reading, meaning integration, and updating of the text which put a high demand on the reader's cognitive load (Sweller, 1988). According to the cognitive load theory, hypertext comprehension is highly dependent on prior knowledge and working memory capacity and may lead to extraneous overload and negative learning outcomes in cases of ineffective information presentation (Kester & Kirschner, 2009; Niederhauser, Reynolds, Salmen, & Skolmoski, 2000; Sweller, Van Merrienboer, & Paas, 1998). Recently, Scharinger, Kammerer, and Gerjets (2015) studied the pupil size and electroencephalogram (EEG) of university students while reading a hypertext and indeed found additional load when they had to decide to click on a hyperlink.
To date, most studies have pointed towards an advantage of comprehension of linear texts over hypertexts and thus seem to support the cognitive load theory (see, e.g., Destefano & Lefevre, 2007). However, this may at least partly be dependent on the hypertext structure. Son, Park, and Kim (2011) measured knowledge of adult readers after reading a linear text, a hierarchical hypertext, and a networked hypertext. Readers who read a linear text had more declarative knowledge, as measured with questions, than readers who read a networked hypertext. Readers of the hierarchical hypertext did not differ in declarative knowledge from readers of the other two text structures. Regarding structural knowledge, which was measured with concept maps, no differences were found between the text structures. Furthermore, it should be noted that the effect of text structure on comprehension may also be influenced by characteristics of the reader. In their review, Destefano and Lefevre (2007) showed that learning outcomes were higher after reading a linear text or hierarchical hypertext than after a networked hypertext but only for readers with low prior knowledge and/or low working memory (Amadieu et al., 2010; Lee & Tedder, 2004; Potelle & Rouet, 2003). It might be that the readers with low prior knowledge and/or working memory are less able to efficiently navigate through an unstructured text (Shapiro & Niederhauser, 2004) and have less space to construct a mental representation of the text without using additional cognitive load (Amadieu & Salmerón, 2014). However, these studies did not take into account the role of other general reading‐related individual skills such as visuospatial working memory and vocabulary.
1.2 The added value of a graphic overview
The information above does assume that text processing and linking information together are easier in more logically written texts than in less structured hypertexts. Adapting the presentation of the text can influence reading outcomes. Following the cognitive load theory, it can be assumed that using a tool that explicitly represents the connections between hypertext pages might decrease the negative load of the visual search. As for the cognitive flexibility theory, one could assume that an explicit drawing of the hypertext structure can support the reader in the flexible construction of the situation model. A way to visually represent the underlying text structure is by providing a graphic overview. It could help the reader by internalizing the structure and enhancing both the textbased (directly derived from the text) and deeper structural knowledge (integration of related concepts) of the text content. Stull and Mayer (2007) found in university students that when an intermediate complex overview was presented in linear texts, deeper understanding was higher than when no overview was presented, whereas textbased understanding was equal. De Jong and Van Der Hulst (2002) found that comprehension of hypertexts with a visual graphical overview in first‐year undergraduates was better than comprehension of hypertexts with so‐called hints or without any hints.
Salmerón et al. (2005) described in their literature review that the effect of an overview is still inconclusive and depends on the type of comprehension and on the reader's level of knowledge. Readers with high prior knowledge do not benefit from an overview, whereas the results are mixed in readers with low prior knowledge. Regarding the interaction of prior knowledge and the structure of a navigable overview, Amadieu, Van Gog, Paas, Tricot, and Mariné (2009) showed that textbased comprehension in readers with low prior knowledge was not affected by the structure of the overview. For readers with high prior knowledge, a hierarchically structured overview led to higher textbased comprehension than a network‐structured overview. For deeper knowledge, the reversed pattern was found. The structure of the overview did not affect deeper comprehension in readers with high prior knowledge. Readers with low prior knowledge had deeper comprehension when a hierarchically structured overview was present than when a network‐structured overview was present.
Next to prior knowledge, the individual's reading skills and visuospatial working memory affected the influence a graphic overview on comprehension (Amadieu & Salmerón, 2014). A graphic overview was helpful for both readers with lower reading skills and/or low visuospatial working memory but less for readers with higher reading skills and/or high visuospatial working memory. Whether the usefulness of a graphic overview is also affected by the reader's verbal working memory and vocabulary is yet unknown.
1.3 Hypertext navigation
To understand the process of hypertext comprehension, another line of hypertext research has been done on the readers' navigation through a hypertext. Hypertext navigation is the reader's path through a hypertext and involves the number of pages that the reader visits, the amount of time spent on each page, and to what extent the order of the pages visited follows a logical, linear order. In general, linear navigation through a hypertext is related to deeper structural knowledge, but not textbased comprehension, whereas nonlinear navigation is not related at all (Salmerón, Kintsch, & Cañas, 2006). Compared to a linear text, navigation through hypertext is suggested to be related to higher cognitive load and more feelings of disorientation. Indeed, McDonald and Stevenson (1996) found that adult readers of a linear digital text rated themselves lower on disorientation than readers of a hierarchical or an unstructured hypertext. Readers of the linear text also showed less navigation problems than readers of the unstructured hypertext. Miall and Dobson (2001) conducted two studies in which they compared reading times per node of a hypertext and a linear text. Young adults reported their reading experiences afterwards. The authors found longer reading times per node while reading the hypertext than the linear text. Contrarily to the linear text readers, the hypertext readers reported more confusion during reading, that they missed something, or that the text was incomplete.
The reader's navigation behaviour may not only depend on hypertext structure but also on the availability of an overview and on readers' characteristics. Learning outcomes are low when prior knowledge and/or visuospatial working memory is low (Müller‐Kalthoff & Möller, 2006; Rouet, Vörös, & Pléh, 2012), and this is suggested to be related to more navigation problems and disorientation. Furthermore, Rouet et al. (2012) found that the availability of a graphic overview did not facilitate hypertext navigation and content recall. They speculated that this is caused by the fact that the hypertext structure was hierarchical in itself and that a graphic overview was not necessary to facilitate navigation. However, a graphic overview may very well facilitate effective navigation in networked hypertexts. Amadieu and Salmerón (2014) suggested that an overview reduces the processing that is necessary for navigation, which diminishes cognitive load. The role of the readers' vocabulary and verbal working memory capacity on navigation behaviour has not been investigated yet.
1.4 Hypertext reading in secondary school students
The majority of the research described above was conducted within university students or adult readers. Only a few studies have focused on hypertext comprehension in primary and secondary school students (see Segers, 2017 for a review). Salmerón and García (2012) compared the influence of literacy skills on reading comprehension of linear text versus a hypertext with navigable overview in 11‐year olds and found higher information integration scores in the hypertext than the linear text, but no differences were found for textbased questions.
Klois et al. (2013) also compared reading comprehension of linear digital text with and without a graphical overview to hierarchical hypertext with and without overview in 13‐year‐old students. At the textbased level, these students learned more from linear texts containing an overview than from the other three text designs. However, the mindmap drawings, reflecting structural knowledge, were more complex in the hypertexts than in the linear texts. Regarding the text navigation, Klois et al. found no differences in total reading time among the text designs, but fewer pages were visited in the texts containing an overview. Another study by Fesel, Segers, and Verhoeven (2017) showed no difference in comprehension of a linear and hypertext, but the availability of an overview was beneficial in sixth graders with low prior knowledge. Level of vocabulary, but not working memory, influenced reading comprehension of digital texts.
In an eye‐tracking study among fifth‐grade students, Sung, Wu, Chen, and Chang (2015) found higher reading performances in linear texts than in hypertexts. The hyperlinks within these hypertexts contained the definitions of difficult words and were not interconnected with other pages. However, students were more likely to be disoriented and had higher cognitive overload when reading the hypertext. With respect to searching behaviour, Schwartz et al. (2004) investigated navigation through and recall after reading linear versus nonlinear hypermedia in children with a large age range from 9 to 17 years and found that more information was recalled after searching in the linear environment.
In all hypertext studies in students, the focus was on hierarchically structured, but not on networked structured hypertexts, and the evidence is mixed regarding advantages or disadvantages of hypertext versus linear text comprehension. Vocabulary clearly is pivotal in the comprehension of both linear and digital texts in secondary school students. Additionally, working memory and prior knowledge have been shown to relate to hypertext comprehension. They are important factors in both cognitive load theory and cognitive flexibility theory. Secondary school students are young, less experienced readers, with developing vocabulary and working memory, and often little prior knowledge. This could lead to a different reading pattern compared to adult readers. It is therefore important to gain information on individual variation in hypertext reading and comprehension in this particular group.
1.5 Current study
To summarize, research in adults and university students shows that level of prior knowledge, vocabulary, verbal, and visual working memory affects text comprehension and navigation of structured hypertexts and that no consensus has been reached regarding the effect of a graphical overview. Less is known about the role of these individual variables on networked hypertexts, and how these factors and the availability of an overview influence text comprehension of networked hypertexts in younger secondary school students. If networked hypertexts pose a challenge for these students, schools should be aware of this and adjust reading comprehension lessons accordingly. To our knowledge, studies on hypertext comprehension and navigation in secondary school students have focused on hierarchical hypertexts only. As networked hypertexts are more ecologically valid, the current study aimed to investigate text comprehension of and navigation through a linear versus networked hypertext with and without an overview in secondary school students. Furthermore, it was investigated to what extent individual factors influence hypertext comprehension. Textbased comprehension was measured with both implicit and explicit questions, and concept maps assessed deeper structural knowledge.
The research questions were the following:
- What is the effect of text structure (linear vs. networked), with and without a graphic overview on text comprehension in school‐aged students, and to what extent is this effect influenced by individual variation in prior knowledge, vocabulary, and verbal and visual working memory?
- What is the effect of text structure (linear vs. networked), with and without a graphic overview on text navigation in school‐aged students?
We hypothesized text comprehension of networked hypertexts to be lower than that of linear texts and comprehension of texts without an overview to be lower than texts with an overview. It was hypothesized that adding an overview would have a larger positive effect on the comprehension of a hypertext compared to a linear text. Additionally, it was hypothesized that the relation between text comprehension and text design would be influenced by individual skills. More specifically, the students with little prior knowledge, low vocabulary, and low verbal and visual working memory would comprehend more after reading a linear text than a networked hypertext and would benefit more from an overview. For students with high prior knowledge, high vocabulary, and high verbal and visual working memory, comprehension would be less influenced by text structure and benefit less from an overview. Regarding text navigation, we hypothesized that reading a networked hypertext involves reading more pages in total and a longer total reading time compared to a linear text and more total pages and longer reading time while reading a text without an overview than with an overview. Furthermore, the addition of an overview would have a larger effect on hypertext navigation compared to linear text navigation, resulting in a decrease of reading time and number of pages read.
2 METHOD
2.1 Participants
Three secondary school classrooms in the Netherlands were involved in this study. Participants were 80 secondary school students (Mage = 12.89, 61.3% boys) in the first grade at preuniversity level. In line with this higher educational track, the mean percentile score of 60.75 (SD = 28.31) on the Raven Standard Progressive Matrices (Raven, 1960) indicated that the students scored above average on nonverbal reasoning skills. Scores on a standardized paper‐and‐pencil lexical decision task (Van Bon, 2007) showed an average level of technical reading skills (M = 83.64, SD = 15.00). Both schools and parents gave permission for participation.
2.2 Materials
Four geographical texts from the Reader's Digest for 11‐ to 14‐year olds that were hierarchically structured in previous studies (Blom, Segers, Hermans, Knoors, & Verhoeven, 2017; Klois et al., 2013) were rewritten to construct both a linear text and a networked hypertext with 10 pages containing exact the same content. They only differed in the underlying connections between the paragraphs. A linear text could only be read in a linear way by clicking on the “previous” or “next” button on the webpage. The reader always started on the main page and finished on the last page. The hypertext was constructed in such a way that a page was linked to other pages by making a hyperlink of that keyword (one or two words) whenever there was an overlap of content, resulting in a networked hypertext with 30 hyperlinks of which six were cross‐sectional, meaning that they were hyperlinks outside of the original hierarchical structure. The first page contained four or five hyperlinks, the remaining pages had on average three hyperlinks per page. For both text structures, an overview was constructed that gave a graphical representation of the text structure with its interconnections. The overview represented the network structure in the networked hypertext and the hierarchical structure in the linear text. The latter is done so that the reader could get a clear overview of the deeper original structure of the text. The overview was present on each page within the text. Both text structures were constructed with and without overview resulting in four different text designs: linear text without overview, linear with overview, hypertext without overview, and hypertext with overview. All four geographical topics (Oceania: 669 words, Russia: 649 words, South America: 681 words, and Southeast Africa: 702 words) were constructed in all four text designs, resulting in 16 different texts. Logfiles registered how long a reader remained on each page, the order of the paragraphs read, and the total reading time.
2.3 Measures
2.3.1 Textbased reading comprehension
Textbased reading comprehension was measured with explicit and implicit multiple‐choice questions that were derived from the Klois et al.'s study (2013). Explicit questions were questions of which the answer could be found in the text literally. Answering implicit questions required making inferences in the text and using general world knowledge. Each text contained five explicit and five implicit questions with four answer possibilities, and a correct answer was one point.
2.3.2 Structural knowledge
Concept maps were used to assess deeper structural knowledge (Van Dijk & Kintsch, 1983). Measures of structural knowledge were the score on concept map complexity, the number of concepts, and the number of hierarchies written down in the concept map. Students received a paper with the text topic in the centre and were asked to complete the concept map by writing down all concepts they remembered from the text. The scoring system was based on Klois et al. (2013), which had high inter‐reliability scores on scoring both first‐ (k = .95) and second‐level (k = .84) concepts within the concept maps. The total number of concepts was counted, as well as the number of hierarchies within the concept map. All concepts that were directly related to the centre were first‐level concepts. Second‐level concepts were all concepts related to the first‐level concepts, and third‐level concepts were linked to the second‐level concepts. The number of first‐, second‐, and third‐level concepts were given two, four, and six points, respectively. The score on concept map complexity was calculated by the sum of these concept scores.
2.3.3 Prior knowledge
Twelve multiple choice questions with four answer possibilities were used to measure prior knowledge. The questionnaire contained three questions per text of which one explicit question, one implicit question, and one general question about the text topic. The questions differed from those that measured textbased comprehension.
2.3.4 Reading time
Logfiles registered the students' activities from the first page until they clicked the “finished” button. By subtracting begin time from the end time, total reading time could be calculated. Also, the reading time per unique page was calculated using the logfiles. The total number of pages read was also registered.
2.3.5 Linearity of navigation
The linearity of the reading sequence was calculated using the logfiles. The order of all pages visited was analysed and scored. Whenever a reader moved from a page to a page that was one level up or below the current level or moved to a same‐level page of the same topic, this transfer received one point. All other navigations received zero points. The score on linearity was calculated as the mean score of all transfers within the text and can range from 0 to 1 (perfectly linear).
2.3.6 Vocabulary
The Peabody Picture Vocabulary Test‐III‐NL (Dunn, Dunn, & Schlichting, 2005) measured the students' vocabulary level. Students were presented with four pictures and pointed to the correct picture that matched the word presented by the researcher. Each correct answer was given one point. A set consisted of 12 items, and all students started at Set 8. The task was finished when they reached the end of Set 15 or after 9 or more errors within a set. Reliability scores of the Peabody Picture Vocabulary Test ranged from .89 to .97.
2.3.7 Verbal working memory
Students' verbal working memory level was measured using the forward and backward digit span task (Kort et al., 2005). In the first part, the student was asked to repeat a sequence of digits, increasing in length (test–retest reliability = .91). In the second part, the sequence had to be recalled in the reversed order (test–retest reliability = .76). There were two items per sequence length, resulting in 16 items in the forward and 14 items in the backward task. The task was finished when two items of the same sequence length were recalled incorrectly. The sum score of the correctly recalled sequences on the forward and backward digit span task (Max = 30) indicated the score on verbal working memory.
2.3.8 Visual working memory
The Corsiblock task (reliability scores ranging from r = .81 to r =. 89) was used to measure visual working memory (Kessels, Van Zandvoort, Postma, Kappelle, & De Haan, 2000). A plate with nine blocks was placed in front of the student. The student had to tap the blocks in the same sequence as the researcher did, and one point was given for each correct sequence. There were two items per sequence length and 16 items in total. The sequence length started with tapping two blocks and increased as long as the student gave correct recalls.
2.4 Procedure
Prior to the test sessions, the students received a lesson about how to draw concept maps and practiced with drawing a concept map to ensure that everyone understood the concept and procedure. Reading comprehension was measured within two sessions on different days with at least 1 day in between. During the sessions, the students read one linear text and one hypertext. A student always read four different text types in a randomized order, so when the linear text was read without overview, the hypertext contained an overview, and when the hypertext was read without an overview, the linear text contained an overview. At the first session, the students received instructions and practiced with reading a text. This practice text contained three pages, of which one page with hyperlinks and one page on which the students could only click on back and forth, so that the students became familiar with the features of both text structures. The final page only stated that the students finished their practice session. Additionally, on the linear page, a graphic overview of the text structure was present to show them what an overview would look like. Then, the students read their first text, after which they answered the questions and then drew a concept map about the text topic. The same procedure followed with the second text. The second reading session was equal to the first session, but there was no practice text. Next to the reading sessions, each student was tested one‐on‐one for all individual measures.
2.5 Data analyses
To answer the first research question, repeated measures ANCOVAs were conducted with the four variables related to reading comprehension as outcome variables and text structure (hypertext and linear text) and the availability of an overview (available or not available) as within‐subject variables. Additionally, prior knowledge, vocabulary, and verbal and visual working memory were included as covariates. By including them as covariates, possible interaction effects could be determined. To assess the influence of text design on navigation, repeated measures ANOVAs were conducted with the four navigation measures as outcome variables and text structure and the availability of an overview as within‐subject variables. Correlation analyses were used to analyse the significant interaction effects.
3 RESULTS
Table 1 provides the means and standard deviations of all students on the individual measures. Students' scores on prior knowledge were above chance level, indicating that they had some prior knowledge about the text topics they had read. The students' text comprehension scores are shown in Table 2. Reliability analyses of the reading comprehension questions showed a sufficient Cronbach's alpha (α = .70). The results on the repeated measures are presented in Table 3. Only significant results will be discussed below.
| Mean | SD | Range | |
|---|---|---|---|
| Prior knowledge | 5.34 | 1.53 | 2–8 |
| Vocabulary | 137.09 | 12.70 | 104–159 |
| Verbal working memory | 14.56 | 2.26 | 10–22 |
| Visual working memory | 9.19 | 1.58 | 6–13 |
| LT M (SD) | LTO M (SD) | HT M (SD) | HTO M (SD) | |
|---|---|---|---|---|
| Comprehension | ||||
| Questions | 7.56 (1.47) | 7.29 (1.55) | 6.99 (1.83) | 6.85 (1.80) |
| Concept maps | ||||
| Complexity | 54.80 (30.95) | 53.83 (36.25) | 43.71 (25.83) | 46.68 (29.40) |
| # concepts | 14.20 (6.02) | 13.44 (6.80) | 11.56 (5.07) | 12.18 (5.64) |
| # hierarchies | 4.68 (1.72) | 4.05 (1.53) | 4.09 (1.39) | 4.11 (1.58) |
| Navigation behaviour | ||||
| N pages read | 17.89 (10.54) | 14.51 (5.42) | 24.84 (8.01) | 23.15 (10.08) |
| Total reading time | 367.01 (148.07) | 396.45 (149.17) | 417.38 (263.57) | 448.94 (288.76) |
| Reading time per page | 26.25 (11.94) | 32.28 (14.68) | 18.11 (8.48) | 21.39 (10.46) |
| Linearity | 1.00 (0.00) | 1.00 (0.00) | .76 (.11) | .72 (.13) |
- Note. LT = linear text; LTO = linear text with overview; HT = hypertext; HTO = hypertext with overview.
| ANCOVA effects (F) | ||||
|---|---|---|---|---|
| Variable | Main effect covariates | X text | X overview | Text X overview |
| Questions | 5.14* | 0.06 | 0.44 | |
| Vocabulary | 45.91** | 4.03* | 0.08 | 0.38 |
| Verbal working memory | 0.23 | 0.24 | 0.20 | 1.21 |
| Visual working memory | 2.92 | 0.01 | 0.08 | 0.55 |
| Prior knowledge | 0.24 | 0.11 | 0.01 | 0.27 |
| Complexity | 0.27 | 0.11 | 0.28 | |
| Vocabulary | 3.17 | 0.64 | 0.17 | 0.60 |
| Verbal working memory | 2.42 | 0.11 | 1.05 | 2.80 |
| Visual working memory | 0.21 | 5.46* | 0.29 | 0.51 |
| Prior knowledge | 0.09 | 1.05 | 0.24 | 2.90 |
| N concepts | 0.01 | 0.29 | 0.10 | |
| Vocabulary | 1.93 | 0.98 | 2.96 | 0.60 |
| Verbal working memory | 2.63 | 0.00 | 1.70 | 2.95 |
| Visual working memory | 0.24 | 4.35* | 1.34 | 0.26 |
| Prior knowledge | 0.08 | 1.86 | 0.87 | 2.33 |
| N hierarchies | 0.82 | 6.42* | 1.46 | |
| Vocabulary | 9.96** | 0.82 | 8.86** | 0.53 |
| Verbal working memory | 0.01 | 1.20 | 0.27 | 0.98 |
| Visual working memory | 0.08 | 0.56 | 0.65 | 0.29 |
| Prior knowledge | 0.11 | 1.04 | 1.98 | 0.17 |
| N pages read | 53.72** | 9.92** | 1.23 | |
| Total reading time (s) | 4.89* | 2.00 | 0.00 | |
| Reading time per page (s) | 69.65** | 20.85** | 1.56 | |
| Linearity | 736.58** | 3.20 | 3.20 | |
- Note. The value of 5.14 represents the main effect of text on questions, whereas the value of 4.03 represents the interaction of text and vocabulary on questions.
- * p < .05.
- ** p < .01.
3.1 Effect of text structure, overview, and individual skills on comprehension
3.1.1 Questions
To measure the effect of question type on reading comprehension, we included for this specific analysis question type (explicit or implicit) as within‐subject variable. Because there was no effect of question type, this variable was not included in Table 3 for clarity's sake. The effects of text structure (F(1, 75) = 5.14, p = .026, ŋp2 = .064), vocabulary (F(1, 75) = 45.91, p < .001, ŋp2 = .38), and the interaction between text structure and vocabulary (F(1, 75) = 4.03, p = .048, ŋp2 = .051) were significant. This interaction can be explained by the fact that the correlation between level of vocabulary and reading comprehension, as measured with textbased questions, was stronger for hypertext comprehension (r = .577, p < .001) than for linear text comprehension (r = .497, p < .001).
3.1.2 Concept maps
Regarding the concept map complexity, there was an interaction effect of text structure and visual working memory, F(1, 75) = 5.46, p = .022, ŋp2 = .068. However, correlation analyses showed no significant correlations of level of visual working memory with either the complexity of the concept map of regular texts (r = .094, p = .40) or hypertexts (r = −.104, p = .36).
There was an interaction between text structure and visual working memory on the number of concepts within the concept map, F(1, 75) = 4.35, p = .040 ŋp2 = .055. Again, level of visual working memory was not correlated with the number of concepts within the concept map of linear texts (r = .082, p = .47) and hypertexts (r = −.100, p = .38).
Next to main effects of availability of an overview (F(1, 75) = 6.42, p = .013, ŋp2 = .079) and vocabulary (F(1, 75) = 9.96, p = .002, ŋp2 = .117), there was an interaction effect of vocabulary and the availability of an overview on the number of hierarchies written down in the concept map, F(1, 75) = 8.86, p = .004, ŋp2 = .106. Correlation analyses showed a significant negative correlation of level of vocabulary with the number of hierarchies written down after reading a text without an overview (r = −.442, p < .001), indicating that students with lower vocabulary level wrote down more hierarchies than students with a higher vocabulary level when they read a text without an overview. There was no correlation of vocabulary level and number of hierarchies written down after reading a text with an overview (r = −.177, p = .12).
3.2 Effect of text structure and overview on navigation
Regarding the total number of pages that was read, there was a main effect of both text structure, F(1, 79) = 53.72, p < .001, ŋp2 = .405, and the availability of an overview, F(1, 79) = 9.92, p = .002, ŋp2 = .112. A larger number of pages was read in a hypertext than in a linear text, and a larger number of pages was read in a text without an overview compared to a text with an overview.
When the total reading time was analysed, there was a main effect of text structure, F(1, 79) = 4.89, p = .030, ŋp2 = .058. More reading time was spent on hypertexts than on linear texts.
Regarding the average reading time per page, there was a main effect of both text structure, F(1, 79) = 69.65, p < .001, ŋp2 = .469, and availability of an overview, F(1, 79) = 20.85, p < .001, ŋp2 = .209. Students spent more reading time per page on linear texts than on hypertexts, and they spent more time per page on texts with an overview compared to texts without an overview.
Regarding the linearity of the navigation path, there was an effect of text structure, F(1, 79) = 736.58, p < .001, ŋp2 = .903. Linear texts were read perfectly linear, whereas hypertexts were not.
4 DISCUSSION
The goal of this study was to investigate reading comprehension of networked hypertexts versus linear digital texts in secondary school students and whether the presence of an overview and the reader's characteristics affected reading comprehension. Furthermore, it was investigated whether text structure and availability of an overview affected navigation.
Regarding the first research question, the results showed that textbased comprehension was lower after reading a networked hypertext than a linear text, but this relation was influenced by vocabulary level. It seems that lower vocabulary levels especially impair comprehension of networked hypertexts compared to linear texts. The theory that this particular hypertext representation causes a high cognitive load that negatively affects reading outcomes (Kester & Kirschner, 2009; Niederhauser et al., 2000) holds only for students with lower vocabulary levels. Vocabulary is by definition related to general knowledge, and to construct deeper structural knowledge, both prior content‐related information and general world knowledge are needed (Perfetti & Stafura, 2014). Our finding that students with lower vocabulary are more affected by the hypertext structure can be explained by weaker word‐meaning connections compared to students with higher vocabulary. Consequently, it may have been more difficult for them to gain deeper knowledge when the text structure was less obvious, because they had more difficulties with understanding the semantically based relations between the pages.
Students with lower levels of vocabulary wrote down more hierarchies when no overview was present, suggesting that, for them, the presence of an overview did not decrease the cognitive load while reading. This is comparable with a study by Hofman and van Oostendorp (1999). They distinguished low and high knowledgeable readers and showed that an overview did not affect comprehension on textbased level. However, structural knowledge in low knowledgeable readers was better when no overview was present. The authors suggested that an overview of the text structure distracts less knowledgeable readers from local information that they need to construct deeper knowledge.
The effect of an overview on both cognitive load and hypertext comprehension remains unclear. This is also due to a variety of features of the overview that has been used in the current study and the studies reported in the overview of Amadieu and Salmerón (2014). The overview in the current study was a static overview, representing the exact underlying structure of the networked hypertext. If the overview had been a simplified, hierarchical representation of the networked hypertext, it could have resulted in a different outcome, as it would have given a clearer structure and perhaps lowered the reader's extraneous load. Other studies used a navigable overview (Amadieu et al., 2009; Amadieu et al., 2010; Potelle & Rouet, 2003) and found that static and dynamic overviews both relate to different comprehension outcomes and levels of extraneous load (Bezdan, Kester, & Kirschner, 2013).
Contrarily to what was expected, there was no influence of prior knowledge on the relation between text design and comprehension. Perhaps the measure of prior knowledge was too narrow, because the prequestions were text‐specific. Additionally, there was no effect of question type on textbased comprehension, and this was also not influenced by text design or readers' skills. This does not correspond to the study of Potelle and Rouet (2003) who found overall higher scores on explicit questions than implicit questions in psychology students and that readers with low prior knowledge scored higher on implicit questions when a hierarchical map was present compared to a network map or an alphabetic list. This effect was not found in readers with high prior knowledge. It is, however, in line with other studies in younger age groups (see Segers, 2017). More developmental research would be necessary to explain these differences.
The second research question focused on the effect of text design on navigation. As hypothesized, more pages in total were read in a hypertext versus a linear text and in a text without an overview versus with an overview, which seems to suggest that readers were went back and forth more often within the text when no clear text structure was available. This was also indicated by the finding that readers spent more time in total on reading a hypertext than a digital text. The study by Klois et al. (2013) did not show longer reading times in hierarchical hypertexts than in linear texts, indicating that the networked structure of the hypertext requires more time to navigate through than a hierarchical text.
Another result in this study was that the average time spent per page was lower in hypertexts than in linear texts and lower in texts without an overview than with an overview. The effect of text structure on reading time per page is not in accordance with the result of the Miall and Dobson's study (2001) who found higher reading times on average per page in hypertexts. Fitzsimmons, Weal, and Drieghe (2016) measured eye movement during hypertext reading and stated that young adult readers engage in three stages when reading a hypertext. They first scan the page very quickly at bottom and top, and they check whether there are hyperlinks present. Then they read the text in a linear way from top to bottom, after which they scan the page again to choose the hyperlink to click on. In a linear text, a reader only reads the text from top to bottom, which makes it more logical to find higher reading times per pages in a hypertext. However, as we found the reversed pattern, it seems that younger readers do not engage in these three stages. Eye‐tracking studies with the current age group are necessary to understand these differences. As we found that hypertext comprehension is lower than linear text comprehension, the question remains whether the readers' scanning behaviour was effective and efficient enough for good comprehension. According to Salmerón, Naumann, García, and Fajardo (2016), good reading comprehension depends on a balanced trade‐off between scanning and deeper processing. In their study, comprehension of a specific type of hypertext in less skilled readers was unaffected by scanning through the hypertext, whereas skilled readers showed less comprehension when they scanned a hypertext. The participants in the current study were proficient readers, for their age, at pre‐university level, and their comprehension may have been influenced by their navigation behaviour.
What should be mentioned regarding the difference in comprehension and navigation of hypertexts versus linear texts is the fact that the hypertext contained 15 hyperlinks in total, which was about three hyperlinks per page. The pages in the linear text only contained two buttons; one for “previous” and one for “next,” and in a previous study with hierarchical hypertexts, there were in total 18 hyperlinks (Blom et al., 2017). Although Madrid, Van Oostendorp, and Melguizo (2009) did not find an effect of number of links, this difference in features may play a role in the reduced comprehension and higher reading times in hypertexts.
This study had some limitations that should be mentioned at this point. The order effects of the repeated measures ANOVA were controlled for by the randomization of the texts within the participants. The use of multiple choice questions limits the types of knowledge assessed, and the texts that were used will impact the results, and thus limit the generalizability of the conclusions. Additionally, the participating students were a high‐achieving subgroup of the general population. Furthermore, this study did not contain a hierarchical hypertext, which could have been useful for the comparison between a linear digital text, a structured hypertext, and an unstructured hypertext. Only a few process measures were used to investigate comprehension and the navigation behaviour of the reader. Eye‐tracking could provide insights into where the readers look on the computer screen, their navigation paths, and actual fixation time per page. Future studies should also focus on the specific overview features that support reading comprehension of hypertexts, should consider how the quality of the concept map should be measured and whether the focus should be more on the structure, the interrelations between the concepts, and the centrality of those concepts instead of the quantity of the concepts and hierarchies.
The practical implications of this study are mainly focused on educational settings. Reading a networked hypertext poses a challenge for secondary school students, while this is often not as such recognized, as this age group is sometimes regarded as digital natives (Prensky, 2001). Especially students with lower vocabulary levels need additional help. Students may benefit from explicit training on how to read hypertexts, and on what they should pay attention to while navigating the networked hypertext (see e.g., Fesel, Segers, de Leeuw, & Verhoeven, 2017). In conclusion, we showed that for 13‐year olds, reading a networked hypertext negatively impacts comprehension and navigation behaviour, especially for those with lower vocabulary.
ACKNOWLEDGEMENTS
This research was funded by the Royal Dutch Kentalis, Sint‐Michielsgestel (BSI project nr. 24000809).
Number of times cited: 1
- Cindy Paans, Eliane Segers, Inge Molenaar and Ludo Verhoeven, Dyadic executive function effects in children's collaborative hypermedia learning, Learning and Instruction, 10.1016/j.learninstruc.2018.11.008, 60, (66-74), (2019).
