Current approaches define reading literacy as the ability to understand and use various types of texts for personal and social purposes (OECD, 2009, 2010). Using information that has been made available in a text or series of texts, readers are asked to answer a select number of questions in a printed or digital environment. This process resembles everyday situations where people engage in reading activities for diverse purposes such as reading a textbook to answer end-of-chapter questions, reading a brochure to obtain information about a language course, or reading forums to contrast different opinions about a conflicting issue. In these scenarios, the goal is no longer to understand the text but to comprehend and use information that is relevant to a particular task, which can determine the focus of a reader’s attention. Similar issues of relevance have been raised in the theory of relevance in reading (McCrudden & Schraw, Reference McCrudden and Schraw2007). According to this framework, relevance is the extent to which textual information can be germane to a specific task or goal. A second feature of the scenario defined above is that readers interact with texts during tasks (e.g., questions, summaries, etc.), moving between text and task until he or she thinks that the task has been properly completed. Many experts call this scenario task-oriented reading (Vidal-Abarca, Mañá, & Gil, Reference Vidal-Abarca, Mañá and Gil2010; Vidal-Abarca, Salmerón, & Mañá, Reference Vidal-Abarca, Martínez, Salmerón, Cerdán, Gilabert, Gil and Ferris2011).
Our study focused on educational settings that require high school students to read one or more texts for the purposes of learning and comprehension. Students in this scenario used texts to understand specific details within the material, answer questions that demanded the integration of information, and reflect on particular aspects of the text. Given the characteristics of task-oriented reading, students must make several decisions throughout the reading process. While some of these decisions influence a student’s focus and methodology for reading (e. g., “do I read the text in full or will skimming the material suffice”?), others refer to a student’s decisions to search the text (e. g., “do I search the text to answer this question or can I answer it from memory”?). The decision to read a certain text prior to answering its questions can have a strong impact on subsequent text search practices. For instance, if a reader decides to read an entire text, then he or she should encounter fewer instances where they must search the text for answers than if they had chosen to merely skim the material.
In contrast to the importance of task-oriented reading, research on this field is still in its infancy. Because task-oriented reading places specific demands on readers that are not explicitly addressed within comprehension processes that lack support from current theories and models of comprehension, an independent research agenda is required (McNamara & Magliano, Reference McNamara, Magliano and Ross2009). A research approach known as the standards of coherence investigates reader’s dispositions towards literacy tasks (van den Broek, Risden, & Husebye-Hartman, Reference van den Broek, Risden, Husebye-Hartmann, Lorch and O'Brien1995; van den Broek, Young, Tzeng, & Linderholm, Reference van den Broek, Young, Tzeng, Linderholm, van Oostendorp and Goldman1999; Tzeng, van den Broek, Kendeou, & Lee, Reference Tzeng, van den Broek, Kendeou and Lee2005). Standards of coherence refer to the level of comprehension that students deem acceptable in the context of their readings. The standards of coherence were conceptualized in relation to readers’ goals, strategies, and states of engagement, all of which are factors that can fluctuate due to situational factors over time. Conversely, van den Broek et al. (Reference van den Broek, Risden, Husebye-Hartmann, Lorch and O'Brien1995) suggests that standards of coherence can vary depending on the individual and impact their capacity for strategic processing. This idea has been largely unexplored.
Strategic processing’s role in task-oriented activities has previously been highlighted by Ozuru, Best, Bell, Witherspoon, and McNamara (Reference Ozuru, Best, Bell, Witherspoon and McNamara2007). These authors conducted two experiments that compared conditions with and without text. College students were tested for prior background knowledge that was vital to their understanding of the texts. They were required to read a collection of science texts and to answer inference questions that were based upon the materials. In the condition with text, Ozuru et al. (Reference Ozuru, Best, Bell, Witherspoon and McNamara2007) found that a reader’s prior background knowledge was highly predictive of his or her scores in comprehension; in the condition without text, however, a reader’s prior background knowledge proved to be irrelevant. The authors argued that each condition emphasized different aspects of the comprehension process. While the condition with text better measured readers’ abilities to comprehend, rapidly access relevant knowledge, and recognize content that was familiar to the content found within a given text, it was also better at engaging readers in strategic processing that allowed them to fulfil specific reading goals. However, Ozuru et al. (Reference Ozuru, Best, Bell, Witherspoon and McNamara2007) did not record students’ strategic behaviors during the study. Existing studies that have investigated this strategic processing have forced readers to read a text in full before allowing them to search the text for answers (Cataldo & Oakhill, Reference Cataldo and Oakhill2000; Cerdán, Gilabert, & Vidal-Abarca, Reference Cerdán, Gilabert and Vidal-Abarca2010; Mañá, Vidal-Abarca, Domínguez, Gil, & Cerdán, Reference Mañá, Vidal-Abarca, Domínguez, Gil and Cerdán2009; Vidal-Abarca et al., Reference Vidal-Abarca, Mañá and Gil2010). These studies, which were conducted with middle school students, concluded that the information readers accessed and retrieved was highly dependent on their individual comprehension skills. While monitoring decisions to search does not depend on these same skills, forcing readers to read a text in full can have impacts on their search behaviors and performances. Although we cannot determine whether the strategic decision to read a certain amount of the text initially is dependent on a reader’s general comprehension skills, our study has been designed to answer these questions.
In our study, we bring together two distinct research traditions: experimental research, where reading is considered a cognitive process, and assessment-related research, where reading is a latent skill that’s processes are assumed but never measured. No existing empirical research has ever attempted to link these two approaches to the field of task-oriented reading. In the following section, we analyze the potential effects that different strategic decisions can have on task-oriented reading performance. Using cognitive and assessment approaches, we then discuss the possible relationships between students’ strategic decisions and their reading comprehension skills. Strategic decisions about initial reading and performance Whether or not the strategic decision to read a text before answering its questions impacts task performance is unclear. This decision is particularly important in large-scale standardized tests that contain task-oriented reading comprehension sections such as the GRE (Graduate Record Examinations) and the TOEFL (Test of English as Foreign Language). As Rupp, Ferne, and Choi (Reference Rupp, Ferne and Choi2006) assert, test-taking guides that use test-wiseness strategies provided by third-party publishers are not consistent in their recommendations. Where some guides recommend reading test questions prior to reading the text (e.g. Green & Wolf, Reference Green and Wolf2000; Hinkel, Reference Hinkel2004; Rogers, Reference Rogers2005; Rymniak & Shanks, Reference Rymniak and Shanks2002), others suggest reading the text before answering the questions (e.g., Gallagher, Reference Gallagher2000; Sullivan, Brenner, & Zhong, Reference Sullivan, Brenner and Zhong2004).
The central argument for reading questions before reading the text includes the following: During task-oriented reading scenarios, students read with a specific goal in mind (e.g., to answer a particular question). Therefore, the only information that is relevant to the reader is text that is relevant to a task’s goal. If students read a set of questions prior to reading the text, the questions may provide cues that help them to identify information that is relevant to their task. Questions may act as signals of relevance, allowing readers to better process the value of specific sections within the text. In this regard, McCrudden, Schraw, and Kambe (Reference McCrudden, Schraw and Kambe2005) asked undergraduate students to begin by reading a group of questions that targeted one or more of the most important aspects in a given text, and to then read the text and do their best to recall it. Students recalled more of the information in the text that had been mentioned in their initial questions. In sum, questions provide cues of relevance that allow readers to pay close attention to information that is relevant to a specific task.
The main argument for reading the text prior to reading the questions includes the following: Through their initial reading of the text, students are able to construct a complete mental representation of the information that is described in the material. This strategy can prove useful in the following three ways: a) it provides a central source of information that has been designed to answer most of the questions (Rouet, Reference Rouet2006); b) it acts as a structural map that can guide a student’s review of the text when needed (Payne & Reader, Reference Payne and Reader2006); and c), it informs students as to whether or not they will need to search the text to answer a particular question (Vidal-Abarca et al., Reference Vidal-Abarca, Mañá and Gil2010).
In a benchmark study on this issue, Cerdán, Vidal-Abarca, Martínez, Gilabert, and Gil (Reference Cerdán, Vidal-Abarca, Martínez, Gilabert and Gil2009) compared the test performance of undergraduate students who were instructed to read a lengthy science text before accessing its questions, with another group that was told to read the questions before searching the text for every answer. While requiring students to read the text prior to reading the questions improved their test performance, it also decreased the number of times that they searched the text for their answers. This positive effect was independent of the types of questions that the students answered, in spite of difficulty or complexity. Furthermore, when recall and comprehension tests were given to test student retention of the readings two days later, those who had read the text initially proved more capable of answering inference questions (see also Payne & Reader, Reference Payne and Reader2006).
The aforementioned studies are limited because they manipulated students to either read entire texts before answering their relevant questions or to answer questions without reading the texts in advance. Therefore, the impact of a reader’s strategic decision to read a text prior to answering its questions during task-oriented reading scenarios remains unclear.
The beginning of Rupp et al.’s (Reference Rupp, Ferne and Choi2006) study illustrates the spontaneous strategic decision processes that readers go through as they determine how much of a specific text they should read before they answer its relevant questions. However, the study says nothing about the influences that these strategic decisions can have on student performance. The study provides data from 10 cognitive interviews with non-native, but fluent, English reading adults who were required to read three passages before answering multiple-choice (MC) questions. The authors found that while some readers read part or all of the texts before reading and answering the questions, others read the questions before reading the texts. Some readers even used the conditional strategy where they chose to read a certain amount of text initially based on the perceived difficulty of questions, passages, and other characteristics such as text and question length. It is therefore possible that strategic readers choose not to adopt the simple notion that reading a text prior to reading its questions is more effective than reading its questions initially. On the contrary, a skilled reader may apply the following conditional rules: if answering the question demands inferential comprehension, then reading the text prior to answering the questions will be more beneficial; if, however, a question demands locating explicitly stated information from the text, then reading the question first will prove more effective. Similarly, Rupp et al. (Reference Rupp, Ferne and Choi2006) found that readers’ experience with MC tests (eight out of ten had either minimal or extensive experience) played a major role in their ability to use these strategies. In a similar vein, it is possible that the strategic decision to read the text prior to answering its questions is linked to a reader’s comprehension skills.
Strategic decisions about how much text is read initially and comprehension skills.
The link between comprehension skill and the strategic decision to read a certain amount of text initially remains unclear. Skilled readers implement basic reading procedures that they have practiced extensively. These include decoding, identifying statements, and integrating ideas without explicit attention (e.g. Perfetti, Reference Perfetti2007). For task-oriented reading scenarios, prior studies have shown that readers skilled in comprehension reviewed only relevant information after an initial reading of a given text (Cataldo & Oakhill, Reference Cataldo and Oakhill2000; Vidal-Abarca et al., Reference Vidal-Abarca, Mañá and Gil2010). In addition, the benefits of initial readings have proven to be independent of comprehension skill, particularly with undergraduate students (Reynolds, Trathen, Sawyer, & Shepard, Reference Reynolds, Trathen, Sawyer and Shepard1993). Please note that varying reading skill levels among samples may not account for the total number of possible interactions (e.g., low-skilled readers are likely to be underrepresented at the undergraduate level). Such being the case, none of the aforementioned studies inform us about the relationship between strategic decisions and comprehension skills.
Based on previous experimental results (Cerdán et al., Reference Cerdán, Vidal-Abarca, Martínez, Gilabert and Gil2009) and our conclusions about the role of forming strong mental representations of a text prior to answering test questions (Kintsch, Reference Kintsch1998) and guide searching processes (Payne & Reader, Reference Payne and Reader2006), we expected that reading texts prior to answering questions would have a positive effect on performance (e.g., the percentage of test questions that are answered correctly). Given the importance of such strategic behavior, we specifically predicted that students skilled in comprehension would be aware of the value in building a complete mental representation of the text prior to making attempts to answer comprehensive questions about the material. In other words, a greater number of readers skilled in comprehension will initially read a higher percentage of the text than their low-skilled counterparts. This strategic behavior may also mediate student performances during task-oriented reading assignments. Because they initially read a greater amount of the text, those with high comprehension skills will likely perform better than those with low comprehension skills. We tested this mediation model in a study by asking a sample of adolescents to read three texts before answering questions that were related to the content. Their performance and on-line reading behaviors were recorded.
Method
Participants
One-hundred-seventy 7th–9th grade Valencian (Spanish) public school students participated in the study. The sample included nine classrooms, from which 44% of the students were female. Schools and children were recruited to participate under a specific agreement of collaboration between principals, the research team, and regional educational authorities.
Materials
Texts
We used three texts from a reading literacy test that was developed using a PISA (OECD, 2009) framework called CompLEC (Llorens et al., Reference Llorens, Gil, Vidal-Abarca, Martínez, Mañá and Gilabert2011). The main characteristics of these texts are summarized in Table 1. A complete example (Nuclear energy text) is reported in the Appendix. All texts were read on a computer and presented with the software Read&Answer in a way we will explain later.
Descriptive statistics of the texts used
Questions
Thirteen questions (between three and five questions per text) following PISA sub-scales, were administered to the participants. PISA sub-scales included access and retrieve (four questions), interpretation and integration (five questions), and reflect and evaluate (four questions) questions. The proportion of questions were similar to the proportion of the three question types used in PISA-2009. Access and retrieval involves locating and retrieving one or more pieces of information (e.g., finding a reason for a claim). Interpretation and integration questions require either global understanding of different sentences or linking of two or more sentences (e.g., cause and effect). In order to relate the information provided within the text to one’s own conceptual and experiential frames of reference (e.g., applying a text’s content to a novel situation), reflection and evaluation questions involve drawing upon ideas, attitudes, or knowledge beyond the text. Most of these questions were MC, with the exception of one open-ended question that required a short answer. The order of questions in a text did not correspond to any specific sequence of ideas within the text. The complete set of questions for text 2 (the nuclear energy text) can be found in the Appendix.
Results from a validation study that evaluated 1865 students (Llorens et al., Reference Llorens, Gil, Vidal-Abarca, Martínez, Mañá and Gilabert2011) indicated that question difficulty ranged from .25 to .74 (i.e., each question’s success rate). The item discrimination index ranged from .22 to .50, with only two items falling below .30. Finally, and according to the Cronbach alpha index, the reliability of the questions was .79. In sum, results from a previous study indicate that the materials used in this current study have proper psychometric characteristics.
Read&Answer
In order to record students’ strategic decisions to read a certain amount of text initially, students were asked to read a collection of texts and answer a group of questions using a software called Read&Answer (Vidal-Abarca et al., Reference Vidal-Abarca, Martínez, Salmerón, Cerdán, Gilabert, Gil and Ferris2011). Read&Answer presents its texts in a fashion that was inspired by the moving window paradigm, a system that blurs the texts initially, so students cannot read them, but can clearly identify structural cues (e.g., titles, indentations, etc.). Students must click on text segments (e.g., a sentence or a group of sentences) to make them readable. Text segments were designed to correspond to pieces of information that are relevant to the answers of the various questions. In other words, if a sentence or group of consecutive sentences contains information relevant to a question’s answer, then they are considered part of the same segment. Therefore, Read&Answer allows researchers to record student reading times and patterns during the entire experimental process. Read&Answer presents questions in a separate window where questions and alternative answers are masked (see an example of the Read&Answer screens in Figure 1). Students must click and hold the mouse over both the question and its answers to unmask them. Prior studies have demonstrated that the masking/blurring procedure does not interfere with normal reading strategies and performance (Bednarik & Tukiainen, Reference Bednarik and Tukiainen2007; Jansen, Blackwell, & Marriot, Reference Jansen, Blackwell and Marriot2003; Vidal-Abarca et al., Reference Vidal-Abarca, Martínez, Salmerón, Cerdán, Gilabert, Gil and Ferris2011). Therefore, Read&Answer is an appropriate tool that simplifies the process of recording readers’ strategic decisions to read a certain amount of the text initially, and does little to disturb the normal process of reading.
Captures from the Read&Answer interface. The left panel represents a situation in which a participant is reading the second paragraph, while the rest of information is blurred. The right panels display two screens with multiple choice and open ended questions.
Reading comprehension skills test
In order to measure reading comprehension skills, we used the Test of Comprehension Strategies (TPC; Vidal-Abarca et al., Reference Vidal-Abarca, Gilabert, Martínez, Sellés, Abad and Ferrer2007), a standardized, Spanish paper-and-pencil test composed of two expository texts and ten multiple-choice questions per section. As proposed by Kintsch (Reference Kintsch1998), questions target different comprehension processes such as the comprehension of explicit ideas within text, drawing inferences between two ideas located in different paragraphs of the text, and building inferences between ideas in the text and students’ background knowledge.
Procedure
In two sessions of approximately 50 minutes a piece, students performed each task individually. During the first session, students completed the TPC in their classroom. During the second session, students worked individually on computers in a classroom of 8–12. Participants were first trained to use Read&Answer. They were taught how to use the windows and instructed about the tool’s interface. They then practiced using the program with two short texts and four questions. Afterwards, they were instructed to use the information in the texts to answer a list of questions. They were told the following: “In order to perform this task, you must use the information in the texts to answer the questions. You are free to read the texts prior to answering the questions. You are also allowed to refer back to the text at any time.” During the experimental session, they were permitted only to ask questions that were related to using the tool. Students read the following three texts in the given order: Bees’ language, Nuclear energy, and Good chairs.
Results
Descriptive analysis
Descriptive statistics of the students’ behaviors and performances for each text are reported in Table 2. In addition, because no existing research has studied readers’ strategic decisions to read certain amounts of text initially during task-oriented reading assignments, we considered it necessary to conduct a series of descriptive analyses that characterized this strategic behavior. Specifically, we documented the number of students who initially read some or all of the text. We then measured this behavior’s degree of consistency for each text. The variable percentage of ideas read initially was computed as the percentage of Read&Answer text segments that were opened before reading any of questions. This variable was not normally distributed (W = .68, .74, .75, all ps < .01, for texts 1–3). This plot distribution suggests that most of the participants read most of the text before moving onto the questions. To analyze the distribution, we computed quartiles of this variable for each of the texts (Table 3). For the three texts, a majority of the students (50.6–59.4%) read at least 75% of each text segment initially (fourth quartile). On average, the percentage of students in the other quartiles was similar. This meant that the strategy to read the entire text initially was the most popular amongst the study’s participants, and that the strategy to not read the text at all was as popular as reading half or three quarters of the text initially. This behavior was moderately consistent across the texts. Specifically, 51.46% of students remained in the same quartile for all three texts, meaning that they initially read a similar amount of material across every text of the study.
Descriptive statistics of students’ behaviour and performance by text
Note: Mean values are reported together with SD (in brackets).
Percentage of participants as a function of percentage of initially read text segments by text
Path analyses
Using the percentage of text that was read initially, and following the bootstrapping procedure suggested by Preacher and Hayes (Reference Preacher and Hayes2004), we tested for the possible mediation effects of comprehension skill on performance scores. Bootstrapping is a nonparametric method that does not assume a normal distribution of the variables and that can be applied to samples of small size (Preacher & Hayes, Reference Preacher and Hayes2004; Preacher, Rucker, & Hayes, Reference Preacher, Rucker and Hayes2007). The effect of interest here consists of the product of the path from comprehension skills on initial readings multiplied by the path from initial readings to task performance (see Figure 2). This product represents the indirect effect of comprehension skill on task performance that occurs as a result of more frequent initial readings from readers with high levels of comprehension. The significance of this effect can be evaluated by estimating its standard error, and on this basis, the construction of a confidence interval. The results indicated that compression skill had a significant positive and indirect effect on performance, via its impact on initial readings in at least two of the texts (text 1 estimate: 0.03; CI95%: 0.01 to 0.09; text 3 estimate: 0.11; CI95%: 0.04 to 0.20), which accounted for 2–4% of the variance (Fairchild et al., Reference Fairchild, MacKinnon, Taborga and Taylor2009). Although the data moves in the same direction, results for text 2 were close to significance levels (text 2 estimate: 0.02; CI95%: –0.01 to 0.05). Taken together, these results are consistent with the assumption that better task performance in readers with high comprehension levels occur in part because they practice the functional strategy of reading a text initially at a high frequency. The sizes of these indirect effects, however, were not very large. For the inspection of the product’s point estimates, increases in standard deviations of task performance were obtained indirectly through initial readings of the text, with increases in comprehension skill amounting to 0.02 standard deviations for text 2 and 0.10 standard deviations for text 3. In addition, a significant direct effect on comprehension skill remained in each case. For two of the three texts, the path coefficient for direct effect was larger than the path coefficient for the mediator ‘initial reading’. Therefore, the effect of comprehension skill is only partially mediated by initial readings.
Partial mediation model for the indirect effect of comprehension skill on task performance that is mediated through initial reading (product of the paths a and b). In addition, a direct effect of comprehension skill on performance is assumed (path c). Together with the model the results for the three texts are presented. Note. + p < .10, * p < .05, ** p < .01, *** p < .001.
Discussion
By presenting new findings about student strategies to read either the text or the questions first during task-oriented reading assignments, this study contributes to existing literature that covers reading literacy skill. Although reading is a general-purpose activity, people often read texts to retrieve and understand information, or use it to follow specific instructions (e.g., to answer a question). While most readers in these scenarios pay attention to relevant information and discard or pay less attention to other parts of the text, they also interact with the material by going back and forth between the texts and their relevant questions. As our results reveal, readers need to approach task-oriented reading assignments strategically in order to succeed. Below, we will discuss several strategic patterns and the implications that our study will have on assessments and instructional practices.
Our results show that most of the students read the majority of a given text before moving onto the questions. A strategic pattern that is related to better performance, most people who have high levels of comprehension take up this strategy. Rather than skimming the text or moving on directly to its relevant questions, we have predicted that people skilled in comprehension tend to read the majority of a given text from its beginning. This impacts their task performance. Evidence for this view is revealed by the mediation effect of comprehension skill, via its influence on initial readings, on performance. Because skilled readers typically read a large amount of the text before moving onto its relevant questions, they generally achieve better performances than their low-skilled counterparts. This new evidence illustrates how skilled readers behave in complex and task-oriented scenarios, suggesting that they can search texts for answers in a highly efficient way (Cataldo & Oakhill, Reference Cataldo and Oakhill2000; Vidal-Abarca et al., Reference Vidal-Abarca, Mañá and Gil2010) and are likely to read a given text should they be given the option to do so.
Furthermore, while the influence of initial readings can vary due to an individual’s general comprehension skill, it can also be interpreted as a specific strategy for task-oriented-reading assignments. This result correlates with a prior study that experimentally manipulated the initial readings of a given text during its evaluation of college students (Cerdán et al., Reference Cerdán, Vidal-Abarca, Martínez, Gilabert and Gil2009). In that study, participants who were asked to read the text initially outperformed students who were asked to read the questions before reviewing the text for answers. In our study, students were given the choice to decide how much of the text they would first like to read. In sum, we can speculate that students who read text before reading questions can construct coherent and rich representations of the information within the text during task-oriented reading assignments, and can also allow them to correctly answer the questions without frequently reviewing the text (Cerdán et al., Reference Cerdán, Vidal-Abarca, Martínez, Gilabert and Gil2009; Ozuru et al., Reference Ozuru, Best, Bell, Witherspoon and McNamara2007). Future research should specifically address this issue.
In some ways, our results contrast with Rupp et al.’s (Reference Rupp, Ferne and Choi2006) claim that test takers tend to “segment a text into chunks that are aligned with individual questions and focus predominately on the microstructure of a text base rather than the macrostructure of the situation model” (p. 469). While this strategy may be useful when tests include questions about specific details, and when the order of the questions is aligned with the order of the information within the text, it would be inappropriate for our study since most of its questions either involved the evaluation and interpretation of multiple parts of the text (e.g. questions 2–5 in the appendix) or required a global understanding of the text as a whole (recall that our study’s participants practiced with two short texts before reading the experimental ones; this informed them about the types of questions they would need to answer beforehand).
Although increases in the predicted task performances after initial readings was substantial, the proportion of the explained variance was well below 100%. This means that initial readings, student skills other than comprehension, and text characteristics can contribute to task performance, as well. On the one hand, probable individual variables can include a student’s existing knowledge about a certain subject, and more basic cognitive variables such as working memory, or the automatization of basic reading processes (e.g. Just & Carpenter, Reference Just and Carpenter1992; Perfetti, Reference Perfetti2007), and the ways in which these variables can interact with reading strategies. For example, a careful initial reading may prove especially vital for students who lack general cognitive resources and are overtaxed when they search for the correct answer within a given text. Individual text characteristics can also explain variables in task performance. In this line, the path analysis for Text 2 (i.e., Nuclear energy) deserves special mention. Dissimilar to the cases of Text 1 and Text 3, the effect of comprehension skill on performance, via its impact on initial readings, does not reach a level of significance. A possible explanation to this pattern could be the distinct organization of Text 2, which is composed of two paragraphs written by different authors, and provides discrepant accounts about the consequences of nuclear energy. In this regard, the text is somewhat similar to reading two different texts. Reading multiple texts about the same issue demands integration mechanisms that go beyond the construction of a model about a single author’s description of a situation (e.g. Rouet, Reference Rouet2006) and may have reduced the impact of initial readings on student performances.
In addition, although only a fraction of comprehension skill’s impact on performance was mediated by initial readings, a large portion of this skill went unaffected. Future studies might strive to employ a more fine grained measurement of comprehension skill that is capable of disentangling various reading processes (e.g., Richter, Isberner, Naumann, & Neeb, Reference Richter, Isberner, Naumann and Neeb2013). This may help researchers to determine the specific processes of comprehension that influence initial reading strategies. Issues such as these should be addressed by future research, employing larger samples that can accommodate for a greater frequency of predictor variables.
Our findings have implications on several existing assessment and instructional practices. Previous studies have demonstrated the importance of considering on-line processes for the assessment of reading comprehension skills (Martínez, Vidal-Abarca, Gil, & Gilabert, Reference Martínez, Vidal-Abarca, Gil and Gilaber2009). The analyses of these on-line processes are even more relevant to the assessment of reading literacy skills when readers are free to use texts to answer questions, which is a common procedure for such assessment tools as the GRE, the TOEFL, the PISA, and the Program for International Assessment of Adult Competencies. These tools are typically applied to an electronic environment so that readers can perform entire tasks at the computer. The measurements considered in this study are easy to implement and can provide indices of reading literacy strategies that complement performance indices. They can also illuminate test-taking guides for large- scale standardized tests. From an instructional perspective, initial readings of text that are designed to help students form coherent mental representation of the material and review the text when needed is easy to teach and practice. It may also improve task-oriented reading performances.
Our study has clear limitations. Some important drawbacks derive from the texts, the tasks, and the whole context of the study. Maybe our findings cannot be applied to longer texts (e.g., reading a book chapter), simpler tasks (e.g., locating specific and simple information), or reading about familiar topics. In all these situations, students may find it more efficient to read the questions first, restricting the demands of a given task through the adoption of more specific reading purposes. Similarly, tasks that involve complex hyperlinked texts may demand for additional and more efficient strategies to cope with the scope of information that can be found on the internet (Salmerón, Cerdán, & Naumann, Reference Salmerón, Cerdán and Naumann2015; Salmerón, Naumann, García, & Fajardo, Reference Salmerón, Cerdán and Naumann2015). With tools such a Wikipedia, an initial reading of the text and its hyperlinked pages is impractical because they lack clear boundaries.
A second limitation is that our study is unable to establish causal relationships between reading processes and performances. As discussed above, however, several of our results are supported by experimental studies that have evaluated college students. In spite of the fact that parts of our study are correlational in nature, we believe that the on-line reading analysis provided by Read&Answer allows clear insights into student strategies that are predictive of their subsequent performances during task-oriented reading assignments.
This research was funded by the Ministerio de Ciencia e Innovación (Secretaría General de Universidades, Project EDU2008–03072). Johannes Naumann is currently at the Goethe-University Frankfurt am Main (Germany)
APPENDIX
Text ‘Nuclear energy’ and its corresponding questions. Correct response options are marked with an asterisk.
