Arithmetic word problem solving can be considered as a complex activity in which solvers create different levels of representation, both mathematical and nonmathematical. The aim of this set of studies is to analyze the role that nonmathematical representation play in problem comprehension. This representational level is related to informal comprehension of the situation described in the problem, the so-called situation model (van Dijk & Kintsch, Reference van Dijk and Kintsch1983) or mental model (Johnson-Laird, Reference Johnson-Laird1983). Although situation models have been studied mostly in the context of tasks related to reading comprehension, they carry implications for word problem solving as well. Solving a problem requires not only mathematical knowledge but also comprehension of the situation evoked by the text, since the problem reflects a real-world situation that specifies agents, actions, and relationship between the events of the problem in everyday context. There is considerable evidence that situation models are constructed on-line during text processing (see Zwaan & Radvansky, Reference Zwaan and Radvansky1998). In contrast, few empirical studies have been made to explore this issue in the context of word problems. Thus, our objective is to extend the findings found in text comprehension and to determine if solvers remain sensitive to situational information during problem solving, even when this information is not (strictly) necessary to solve the problem.
Several simulation models of understanding and solving arithmetic word problems have analyzed the knowledge structures and processes necessary to solve a problem. Most of these models, however, have not considered the importance of the construction of an adequate representation of the concrete situation described in the text of a word problem (e.g., Briars & Larkin, Reference Briars and Larkin1984; Riley, Greeno, & Heller, Reference Riley, Greeno, Heller and Ginsburg1983).
According to these models, solving a problem involves more or less direct assignation of problem’s numerical information to an activated problem schema. These schemata are considered formal structures that represent basic semantic relationships between the main quantities in the problem. For example, there are schemata involving a change from an initial to a final state through application of a transformation-Change schema-, the combination of two discrete sets or splitting of one set into two discrete sets -Combine schema-, and schemata involving the quantified comparison of two discrete sets of objects -Compare schema- (Riley et al., Reference Riley, Greeno, Heller and Ginsburg1983; Verschaffel & De Corte, 1997).
Kintsch and Greeno (Reference Kintsch and Greeno1985) were the first who used the term “situation model” in the context of word problem solving. Based on the general theory of text comprehension developed by van Dijk and Kintsch (Reference van Dijk and Kintsch1983) and Kintsch and Greeno (Reference Kintsch and Greeno1985) theorized that, during the reading of a word problem, a dual representation is constructed, including a propositional representation, termed text base, to capture the local meaning of the passage, and a representation for the situation and actions in a text, termed situation model or problem model, inferred from the text base and the reader’s knowledge in the domain of arithmetic problems. The problem model reflects only the mathematical information relevant and necessary to solve the problem. Thus, when a problem is presented, propositions are assigned to a set schema in the problem model. A set schema is a structure that represents properties and relations of sets. A set is represented with four attributes that correspond to the object, the quantity, the specification, and the role slots. Role holds a relational term that identifies the set’s role in a higher level structure (higher order schema) that includes other sets. The authors termed this representation as problem schema, which is triggered by specific linguistic terms, such as “give”, “altogether” or “more … than”, depending of the kind of problem.
A limitation of the Kintsch and Greeno model, as stated by Staub and Reusser (1995; see also Thevenot, Devidal, Barrouillet, & Fayol, Reference Thevenot, Devidal, Barroulillet and Fayol2007), is that it relies excessively on the schema theory, jumping directly from the propositional text base to a problem schema that is activated by specific textual clues. In this sense, and similar to the above-mentioned by Briars and Larkin (Reference Briars and Larkin1984) and Riley et al. (Reference Riley, Greeno, Heller and Ginsburg1983), the Kintsch and Greeno model still shows no real attempt to explicitly understand the situation described in text of the problem in the initial stage of the problem-solving process.
Extending the work of Kintsch and Greeno (Reference Kintsch and Greeno1985), Reusser (Reference Reusser, Mandl, De Corte, Bennett and Friedrich1990) developed a simulation model of understanding and solving word arithmetic problems, called “Situation Problem Solver”, that involves the generation of an episodic situation model, implying the “application of comprehension strategies to the text base, which generate an analysis of the temporal and functional structure of the situation and actions depicted in the problem texts” (Staub & Reusser, Reference Staub, Reusser, Weaver, Mannes and Fletcher1995, p. 293). The goal of this stage is to construct an overall representation of the events and actions depicted by the text of the problem in terms of everyday concepts. This nonmathematical representation would be the bridging element between the text base and the problem model as described by Kintsch and Greeno (Reference Kintsch and Greeno1985). Thus, the construction of the problem model depends not only on activation of the problem schema, but also on information provided by situation model.
Although the evolution of these successive models has been aimed at stressing the importance of the comprehension of the situation described in the problem, the role that this initial nonmathematical representation plays in understanding and solving arithmetic word problems is still put to test nowadays (Múñez, Orrantia, & Rosales, Reference Muñez, Orrantia and Rosales2013; Orrantia & Múñez, Reference Orrantia and Muñez2013; Orrantia, Tarín, & Vicente, Reference Orrantia, Tarín and Vicente2011; Thevenot, Reference Thevenot2010; Thevenot et al., Reference Thevenot, Devidal, Barroulillet and Fayol2007; Vicente, Orrantia, & Verschaffel, Reference Vicente, Orrantia and Verschaffel2007, Reference Vicente, Orrantia and Verschaffel2008a, Reference Vicente, Orrantia and Verschaffel2008b). A specific question that needs to be answered is whether inferences related to the construction of a situation model are generated on line when the word problems are presented in the context of a story, that is, when they are presented in a more enriched and elaborated way by highlighting the situation described by the problem. This is important because as textual entities, word problems are similar to other texts, such as narrative texts, especially when word problems are presented in the context of a rich story, given that both narrative text and word problem are basically narrative reflecting real-world situations. In both texts one may find characters with goals and motivations to achieve desired states and actions to attain a desired goal related to a particular state, activity, or object (Trabasso, van den Broek, & Suh, Reference Trabasso, van den Broek and Suh1989), even if numerical information is included in the text (i.e., a character with certain amount of money; the character wants to get her or his brother a present; the character goes to the department store; the character spends a certain amount of money…).
In this context, research in narrative comprehension has found that the creation of causal inferences guided by the goals of a story character is an essential aspect of comprehension (Albrecht & Myers, Reference Albrecht and Myers1995; Suh & Trabasso, Reference Suh and Trabasso1993; Trabasso et al., Reference Trabasso, van den Broek and Suh1989). Because goals provide reasons for characters’ actions, they allow readers to integrate actions with text information presented earlier, thus forming coherent causal networks (Graesser, Singer, & Trabasso, Reference Graesser, Singer and Trabasso1994; Magliano, Zwaan, & Graesser, Reference Magliano, Zwaan, Graesser, van Oostendorp and Goldman1998; Trabasso et al., Reference Trabasso, van den Broek and Suh1989). There is substantial evidence that such goals are highly accessible during comprehension (Albrecht & Myers, Reference Albrecht and Myers1995; Lutz & Radvansky, Reference Lutz and Radvansky1997; Suh & Trabasso, Reference Suh and Trabasso1993; van den Broek & Lorch, Reference van den Broek and Lorch1993). Much of this research has focused on how causal relations influence processing when the reader is confronted with actions that are inconsistent with the characters’ goals (Albrecht & Myers, Reference Albrecht and Myers1995; Huitema, Dopkins, Klin, & Myers, Reference Huitema, Dopkins, Klin and Myers1993; Poynor & Morris, Reference Poynor and Morris2003). For example, in Huitema et al.’s (Reference Huitema, Dopkins, Klin and Myers1993) study, participants read line by line passages that included a goal for the protagonist, such as “go to a place where he could swim and sunbathe”, and then a distant target sentence described the protagonist performing an action that was either consistent (e.g., asked for a plane ticket to Florida) or inconsistent (e.g., asked for a plane ticket to Alaska) with the previously stated goal. As predicted, subjects read the inconsistent version of the target sentence more slowly than the consistent version. A similar consistency effect emerged even when the goal was not explicitly stated in the text (e.g., had always been a real sun-worshipper; Poynor & Morris, Reference Poynor and Morris2003). Thus, readers draw inferences connecting the characters’ actions to goals stated earlier in the text. Moreover, it also has been shown that if a goal has not yet been completed, this goal information remains available (Lutz & Radvansky, Reference Lutz and Radvansky1997; Magliano & Radvansky, Reference Magliano and Radvansky2001; Suh & Trabasso, Reference Suh and Trabasso1993). In these studies, participants read texts in which a character’s goal was introduced (e.g., Betty wanted to give her mother a present). Then, later in the story, that goal was either completed (e.g., Betty bought her mother a purse) or failed (e.g., Betty could not buy anything for her mother). Looking at response times to a probe question that asked about the critical goal assessed availability of the goal. The results showed that response times were shorter for the failed goal condition than for the completed goal condition, indicating that the availability of the failed goal, relative to the completed goal, increased. Thus, unsatisfied goals are more accessible than completed or satisfied goals.
Results from these studies show that, during reading of narrative texts, readers generate inferences to match actions with particular goals. Since the generation of these inferences is part of constructing of a situation model of what the text describes (Zwaan & Radvansky, Reference Zwaan and Radvansky1998), the main goal of the present set of studies was to analyze to what extent this inferential processing also occurs when readers are confronted with word problems rather than narrative texts. Taking as starting point the above-mentioned studies that suggest that the characters’ goals are highly accessible during reading, we constructed “rich story problems”, that is, word arithmetic problems included in the context of a story, in which the characters propose goals, and then these goals are followed by actions to achieve it. If solving word problems involves a more or less direct assignation of the numerical information and relationships to an activated schema, as stated by schema theory (Kintsch & Greeno, Reference Kintsch and Greeno1985; Riley et al., Reference Riley, Greeno, Heller and Ginsburg1983), the mental representation constructed by reader (solver) would contain only the mathematical information relevant and necessary to solve the problem. This means that attention would be focused on the pieces of information that are specific to the problem to be solved, but not on situational information, such as the characters’ goals. In contrast, if the construction of the mental representation depends not only on specific arithmetic information, but also on information contained in the situation model (Reusser, Reference Reusser, Mandl, De Corte, Bennett and Friedrich1990; Staub & Reusser, Reference Staub, Reusser, Weaver, Mannes and Fletcher1995), readers would show a pattern of goal information availability as part of constructing their mental representation.
The purpose of our research was to demonstrate that readers are, in effect, sensitive to situational information related to the characters’ goals. To demonstrate such sensitivity, we designed three experiments in which the situational information was more or less related to the activation of the problem schema. In Experiment 1, the story problems were designed so that the character’s goal was closely related to the activation of a problem schema, e.g., the character wants to increase or to decrease a flock of sheep (see Appendix). In this case, the schema, which corresponds to a Change problem, could be triggered by information included in the goal statement. Experiment 2 was similar to Experiment 1 except that the goal of the protagonist was not explicitly stated; to activate the schema readers would need to infer the character’s goal. In these first two experiments, measuring participants’ reading times for action statements that were either consistent or inconsistent with the characters’ goals assessed goal information availability. In Experiment 3 the problem schema activation was clearly separated from the goal information. In this experiment, the availability of the goal was tested by measuring participants’ response times to target words selected from goals that were either completed or failed by the character/protagonist. In all three experiments, two different reading conditions were purposed: reading for understanding or reading for solving the story problem. In this way, the two groups of participants read the same set of texts under two different perspectives. While the first group read the texts as narratives, the second group read them as word problems. By introducing this variable, we also analyzed the extent to which the reading purpose has an influence on processing of story problems. There is considerable evidence that readers differentially allocate their processing resources according to reading purpose, text genre, reading tasks, and so on (e.g., McCrudden, Magliano, & Schraw, Reference McCrudden, Magliano and Schraw2010; van den Broek, Lorch, Linderholm, & Gustafson, Reference van den Broek, Lorch, Linderholm and Gustafson2001; Zwaan, Reference Zwaan1994). Whereas most of these studies have focused mainly on narrative and expositive texts, much less attention has been given to the possibility that text processing may also vary under different reading purposes or tasks when other kinds of texts are considered, such as word arithmetic problems.
Experiment 1
The purpose of the first experiment was to assess whether readers are responsive to the match between characters’ actions and the goals that precede those actions when they are confronted with story problems. Participants read story problems in which the characters propose goals to increase or decrease sets. Therefore, the goal statements elicited the activation of the problem schema. Later in the story problems, the characters carry out actions that are consistent or inconsistent with the goals. As stated earlier, two different reading tasks were purposed: reading for comprehending or reading for solving the story problems. We predict that reading under a narrative comprehension orientation, readers should emphasize the construction of the representation of the causal structure of the text. In this way, according to previous research (e.g., Huitema et al., Reference Huitema, Dopkins, Klin and Myers1993), more time would be spent on the actions that are inconsistent with the character’s goals than on consistent actions. However, we suggest a larger consistency effect under a solving orientation than under a comprehension orientation, due to the inconsistency being also arithmetic. Let us consider this prediction for the sample story problem given in Table 1. When readers under a solving orientation encode the goal information (i.e., the protagonist wants to decrease the size of the flock), that information would support the activation of a Change schema with the first quantity as the start set, together with the direction of transfer, so that the start set would be reduced. When the inconsistent target sentence is encountered (i.e., the protagonist decides to buy sheep), the direction of transfer implies increasing the start set. Thus, both the goal information (namely that the target sentence is inconsistent with the goal) and the arithmetic knowledge (namely that the target sentence is inconsistent with the Change schema activated) create a conflict with the information being processed so far. In this sense, the combined effects of goal-action inconsistency and arithmetic inconsistency would lengthen the reading times of the target sentence as compared to reading under a narrative comprehension orientation, wherein only there would be an effect of goal-action inconsistency.
Mean (and Standard Deviations) reading times (in milliseconds) for the target and post-target sentences in Experiment 1
In addition to reading times for consistent or inconsistent action sentences, reading times of other text segments, such us sentences containing numerical data were also measured. If readers allocate their processing resources according to reading orientation, then these segments that contain information that is necessary to solve the problem would be read for a longer time under a problem solving orientation than under a narrative comprehension orientation.
Method
Participants
Sixty Spanish psychology undergraduates, 48 women and 12 men from 18 to 35 years of age (M = 19.8. SD = 3.4) participated for course credit. The data from one participant were dropped because that individual made more than 15% errors in responding to questions presented after the story problem.
Materials
We created 12 experimental story problems. All the problems were constructed on the same model, and they all referred to a single protagonist who performed an action that was either consistent or inconsistent with his or her goal (see Table 1). Each story problem began with two sentences that served to introduce the character and to establish the situation, together with the first numeric information. The next two sentences stated the character’s goal and motivation to the goal. One of two goal versions was presented: consistent or inconsistent. The consistent version contained a goal of the protagonist that supported the execution of the action described in the critical sentence. The inconsistent version presented a goal that conflicted with the execution of the action. Both the consistent goal and the inconsistent goal contained the same number of words, with an average length of 76.5 and 76.3 characters, respectively.
Following the goal statement were four filler sentences that served to distance the character’s goal from the consistent or inconsistent action while maintaining local coherence. Thus, the goal was no longer in working memory when the action was read. After these filler sentences, the critical sentence appeared that described an action of the protagonist together with the second numeric information. This sentence was then followed by a spillover line that was a continuation of the character’s action, but it made no direct reference to the goal. The next sentence was added to resolve the contradiction in the inconsistent version and was the same in every passage regardless of condition. The last section of each story problem included a new episode, which was composed of three to four sentences containing another operation to make the story problem mathematically more complex.
Two sets of story problems were constructed; each set contained six problems in each of the two versions, consistent and inconsistent. Across the two problem sets, each problem occurred in each of the two versions. Each problem was followed by a yes-no comprehension or solving question to ensure that subjects were carefully reading each problem for comprehension or solving (see Appendix). Twelve filler story problems were included to vary the form of the problems and the arithmetic operations included. Most of the filler story problems did not contain a goal. When they did, the goals did not appear at the same point of the story at which they appeared in the experimental story problems.
Design
A 2 (condition: comprehension vs. solving) X 2 (version: consistent vs. inconsistent) factorial design was used. Version was a within-subjects factor, whereas condition was a between-subjects factor. Participants were randomly assigned to one of the two conditions. Regarding orienting instructions, in the narrative comprehension condition the participants were asked to read for comprehension of story, just as with any narrative text. In the problem solving condition, the participants were said that the problems would be presented in the context of a story, and they were asked to read for solution of the problems.
Procedure
Participants were randomly assigned to one of two stimulus sets. Problems were presented on a computer screen one line at a time. Participants controlled the presentation of the problem with a line advance key. Each key press caused the current line to be erased and the next line to be presented. After the last line of the problem was read, a comprehension or solving question was presented (depending on the condition to which the participant was assigned), to which the participant responded by pressing one of two keys (“yes” and “no”). If the participant replied incorrectly, the word “error” appeared on the screen for 500ms. Participants were given four practice trials before beginning the experiment.
Results and discussion
Data for reading times are presented in Table 1. Reading times that were three standard deviations from the mean were discarded. This eliminated no more than 1.5% of the data. In all analyses reported in this article, F1 represents the results of analyses in which the error terms were based on subject variability, and F2 the results of analyses in which the error terms were based on item variability. Results reported as significant are at the .05 level unless otherwise indicated.
As shown in Table 1, readers spent more time on the inconsistent target lines than on the consistent target lines in the comprehension condition. This replicates the results reported by other studies in which a similar consistency effect was found with narrative texts (Huitema et al., Reference Huitema, Dopkins, Klin and Myers1993; Poynor & Morris, Reference Poynor and Morris2003), and supports the validity of the texts used in our experiment. More interestingly, however, is that this effect was also found for the solving condition. There was a main effect of consistency, F1(1, 57) = 65.44, MSE = 64,117, p < .0001, η p 2 = .53; F2(1, 22) = 26.46, MSE = 45,204, p < .0001, η p 2 = .55. This effect was qualified by a significant interaction between condition and version, F1(1, 57) = 13.05, p < .001, η p 2 = .17; F2(1, 22) = 4.9, p < .05, η p 2 = .17. Planned comparisons between means using the Bonferroni adjustment with p < .05 indicated that though the consistency effect was significant for both the comprehension condition, F(1, 57) = 10.19, p = .002, and the solving condition, F(1, 57) = 67.33, p = .0001, as predicted this effect was larger in the solving condition (546 ms) than in the comprehension condition (209 ms).
The effect on the target line was also observed on the post-target line for both conditions. Again, there was a main effect of consistency, F1(1, 57) = 6.87, MSE = 77,343, p < .05, η p 2 = .11; F2(1, 22) = 4.49, , p < .05, η p 2 = .16; but no interaction effect, F1(1, 57) = 1.39, p = .24 F2 < 1. This spillover effect has been observed in several other experiments when the post-target line followed a target line that caused processing difficulty (e.g., Albrech & O’Brien, Reference Albrecht and O’Brien1993).
With regard to regions containing information necessary for solving the problems (i.e., information containing numerical data), it was expected that the participants in the solving condition would spend more time to these regions. According to this expectation, reading times for target sentences were longer in the solving condition than in the comprehension condition, F1(1, 57) = 9.18, MSE = 307,393, p < .005, η p 2 = .14; F2(1, 22) = 20.69, MSE = 40,214, p < .0001, η p 2 = .48. Similarly, numerical information 3 regions were read more slowly in the solving condition than in the comprehension condition, F1(1, 57) = 9.81, MSE = 870,505, p < .01, η p 2 = .14; F2(1, 22) = 11.35, MSE = 213,812, p < .005, η p 2 = .34. However, there were no differences between conditions respect to numerical information 1 regions. One possible explanation is that processing this numerical date leads no activation of the arithmetical schema yet. Because schema still has not been activated, participants in the solving condition did not need to spend more time to this numerical information. The rest of numerical information regions, however, were processed with a schema activated to be filled, which caused longer reading times to these regions. Despite reading times on numerical sentences were longer in the solving condition than in the comprehension condition, it is interesting to consider that total reading time for the story problem was longer in the comprehension condition than in the solving condition. F1(1, 57) = 8.43, MSE = 34,729,791, p < .01, η p 2 = .15; F2 (1, 22) = 12.98, MSE = 9,064,674, p < .01, η p 2 = .37. These results suggest that expectations generated by reading purpose influence how readers process texts.
The results of this first experiment support the idea that, in solving story problems, readers process situational information besides mathematical information relevant and necessary to solve the problem. Considering that generation of inferences to match actions with particular goals is part of constructing of the situation model, participants in Experiment 1 were slow to read a target line describing an action that was inconsistent with the goal stated earlier in the story problem. As expected, this slowdown was longer in the solving condition than in the comprehension condition, due to the goal information activated the problem schema creating a conflict between this schema and the one associated with the target sentence. Apparently the combined effects of goal-action inconsistency and arithmetic inconsistency slowed the reading times of the target sentence in the solving condition.
However, it is necessary to rule out an alternative explanation of the results of Experiment 1. Since the problem schema is triggered by specific linguistic terms that appear explicitly in the goal sentences (e.g., to increase or decrease), it is possible that only an effect of arithmetic inconsistency may have been responsible for the reading time differences observed in Experiment 1, rather than this effect combined with a inconsistency between the goal and the action. In other words, it is possible that participants in the solving condition only processed from the goal sentences the linguistic terms related to the activation of the Change schema. In this case, the arithmetic inconsistency effect in the solving condition would be larger than the goal-action inconsistency effect in the comprehension condition, given that, in the solving condition, there are clear-cut criteria for when a story problem is comprehended (Kintsch, Reference Kintsch1988) in comparison with approaching the story problem with a more general goal of comprehension. Experiment 2 was conducted to assess whether this explanation is true by using story problems in which the goal of the protagonist was not explicitly stated. In this way, the linguistic terms related to the activation of the problem schema were removed.
Experiment 2
Experiment 2 was similar to Experiment 1 except that the goal of the protagonist remained implicit. Thus, to activate the schema participants would need to infer the character’s goal, given that the linguistic terms related to the activation of the problem schema were not explicitly stated. Finding a consistency effect in Experiment 2 similar to that found in Experiment 1 would further demonstrate that participants are sensitive to situational information included in the story problems.
Method
Participants
Forty psychology undergraduates, 31 women and 9 men, participated for course credit. Their ages ranged from 19 to 34 years (M = 20.5, SD = 3.2).
Materials, design and procedure
The 12 experimental story problems from Experiment 1 were modified so that the character’s goal was not explicitly stated, presenting only the motivation to the goal (see Appendix). In all other respects, Experiment 2 was identical to Experiment 1.
Results and discussion
Table 2 presents the means of reading times for the target and post-target lines. As in Experiment 1, participants took longer to read the inconsistent target lines than the consistent target lines, F1(1, 38) = 23.39, MSE = 48,175, p < .0001, η p 2 = .40; F2(1, 22) = 6.72, MSE = 71,358, p < .02, η p 2 = .23. Although the interaction did not reach standard levels of significance, F1(1, 38) = 2.25, p = .15; F2(1, 22) = 2.15, p = .17 planned comparisons between means using the Bonferroni adjustment indicated that consistency was significant for both the comprehension condition, F(1, 38) = 5.69, p = .02, and the solving condition, F(1,38) = 19.82, p = .0001, but this effect was larger in the solving condition (317 ms) than in the comprehension condition (170 ms). As in Experiment 1, the post-target line was read more slowly after the inconsistent target line than after the consistent target line, F1(1, 38) = 6.49, MSE = 47,320, p < .02, η p 2 = .15; F2(1, 22) = 6.07, MSE = 50,547, p < .03, η p 2 = .22. In addition, numerical information 3 regions were read more slowly in the solving condition than in the comprehension condition, F1(1, 38) = 5.66, MSE = 394,963, p < .03, η p 2 = .16; F2(1, 22) = 5.21, MSE = 174,625, p < .03, η p 2 = .22, whereas there were no differences between conditions respect to numerical information 1 regions. However, total reading time for the story problem was longer in the comprehension condition than in the solving condition, F1(1, 38) = 4.16, MSE = 36,186,115, p < .03, η p 2 = .14; F2(1, 22) = 15.21, MSE = 11,997,439, p < .01, η p 2 = .20.
Mean (and Standard Deviations) reading times (in milliseconds) for the target and post-target sentences in Experiment 2
These results strengthen the findings of the first experiment by replicating the consistency effect when the goal was implicit. Participants inferred the goal, as evidenced by the increased reading time on inconsistent statements. Furthermore, inferential processing of the goal allowed participants to activate the problem schema in the solving condition, as evidenced by a larger consistency effect in this condition than in the comprehension condition. Taken together, the results of Experiment 1 and 2 provided evidence that, in the solving condition, participants were sensitive to the character’s goal that allowed the activation of the schema regardless of whether the goal was explicitly stated or had to be inferred by the reader. In addition, the reading time patterns showed by participants in both experiments suggest that reading purpose influences how readers process texts.
Experiment 3
Despite the results of Experiments 1 and 2 would support the idea that readers generate inferences to match actions with particular goals when they are confronted with story problems, the limits of goal availability still remain to be analyzed. Given that in Experiments 1 and 2 the character’s goal activated the problem schema, it could be that the weight of the goal-action inconsistency was overshadowed by the arithmetic inconsistency when inconsistent target lines were processed. In other words, it is possible that once processed the goal, even if the reader must infer it, the problem schema rather than the information related to the goal is what remains available in memory. To analyze this possibility, in Experiment 3 we designed story problems such that the character’s goal did not activate any problem schema. Given that it is difficult to include inconsistent actions in these new passages (e.g. the action would be inconsistent either with the goal or with the math schema), another paradigm was used to analyze whether readers make goal inferences while solving story problems. As we noted in the introduction, one way in which previous studies on reading comprehension have demonstrated the importance of goals is to show that if a goal has not been completed, this goal information remains available (Lutz & Radvansky, Reference Lutz and Radvansky1997; Magliano & Radvansky, Reference Magliano and Radvansky2001; Suh & Trabasso, Reference Suh and Trabasso1993). Accordingly, failed goals should remain available because they are unsatisfied, while successfully completed goals should not. This idea has been tested by having people read stories and then giving them memory probes later in the story to assess the availability of goal-related information.
According to this paradigm, we designed story problems in which a character’s goal was introduced. In contrast with Experiments 1 and 2, the goal information was presented so that problem schema activation was impossible. Later in the story problems, that goal was either completed or failed. In order to determine whether participants maintained the availability of the failed goal information, we recorded, in the context of a lexical decision task, responses to target words that were selected from the goal statement in the story problem. We opted for a lexical decision task due to, as in Experiments 1 and 2, two different reading tasks were purposed: reading for comprehending or reading for solving the story problems. In this sense, memory probes, as those used in previous studies, would drive attention of participants in the solving condition toward information related not only with the resolution of the problem, but with the story problem as a whole. In line with previous research in which narrative texts were used (e.g., Lutz & Radvansky, Reference Lutz and Radvansky1997), we predicted that in the comprehension condition decision latencies to words that refer to failed goal information would be faster than latencies to words that refer to completed goal information. Furthermore, if as expected, participants in the solving condition would process goal-related information even when the goal does not activate the problem schema, then the differences between the failed and completed goal versions should not differ with condition.
Again reading times of other text segments, such us sentences containing numerical data were also measured. According to Experiments 1 and 2, these segments that contain information that is necessary to solve the problem would be read for a longer time under a problem solving orientation than under a narrative comprehension orientation.
Method
Participants
Fifty-four psychology undergraduates, 41 women and 13 men, participated for course credit. Participants ranged in age from 19 to 29 years (M = 20.9, SD = 2.9).
Materials
There were 16 experimental story problems and 24 filler story problems. The experimental story problems had the same structure as those used in previous studies (e.g., Lutz & Radvansky, Reference Lutz and Radvansky1997; Suh & Trabasso, Reference Suh and Trabasso1993) but including numerical information. Each story problem had two versions: failed goal and completed goal (see Appendix). There were three episodes in each story problem. In Episode 1, goal 1 was introduced and it was not attained in the failed goal version but attained in the completed goal version. Note that Goal 1 information still does not allow activating a problem schema. In Episode 2, Goal 2, a subgoal of Goal 1, was introduced and attained. In Episode 3, the actions lead to a successful outcome, which was of goal 1 in the failed goal version, whereas it was of a goal other than Goal 1 or Goal 2 in the completed goal version.
Each story problem had a lexical decision task. For the experimental story problems, the lexical decision always occurred in the third episode, just before the successful outcome of Goal 1 in the failed goal version. This placement was chosen because prior studies have shown that Goal 1 is highly available at this point for the failed goal versions, but not for completed goal versions (e.g., Lutz & Radvansky, Reference Lutz and Radvansky1997; Suh & Trabasso, Reference Suh and Trabasso1993). The correct answer was always “yes”. For the filler and practice story problems, the lexical decision occurred at different locations of the story problem, so that participants could not anticipate when the lexical decision task would occur. Across all story problems there were an equal number of words and non-words. Words in the lexical decision task were selected from the Goal 1 information. Words were verbs (e.g., “to restore” [restaurar] from the goal “Pablo wanted to restore his classic car” [Pablo quería restaurar su coche antiguo]) or nouns (e.g., “present” from the goal “Maria wanted to give her mother a present”). Nonwords were constructed by selecting a word that was unrelated to the story problem and changing a vowel or consonant to create a pronounceable nonword.
Two sets of experimental story problems were constructed; each set contained eight problems in each of the two versions, failed goal and completed goal. Across the two sets, each problem occurred in each of the two versions. Each story problem was followed by a yes-no comprehension or solving question to ensure careful reading.
Design
A 2 (condition: comprehension vs. solving) X2 (version: failed goal vs. completed goal) factorial design was used. Version was a within-subjects factor, whereas condition was a between-subjects factor. Participants were randomly assigned to one of the two conditions. Instructions were the same that those in Experiment 1 and 2.
Procedure
Participants were randomly assigned to one of two stimulus sets. Problems were presented on a computer screen one line at a time. Participants controlled the presentation of the problem with a line advance key. Each key press caused the current line to be erased and the next line to be presented. The lexical decision task proceeded as follows. After reading a line, a row of asterisks was presented for 500 ms at the center of the screen and was then replaced by the target letter string. Participants decided whether the letter string was a word or a nonword by pressing one of two keys (“yes” and “no”). If the participant replied incorrectly, the word “error” appeared on the screen for 500 ms. After the last line of the problem was read, a comprehension or solving question was presented depending on the condition to which the participant was assigned. Participants responded by pressing the yes/no keys, and received feedback only when their response was wrong.
The participants went, first, through a training phase at the beginning of the experiment. A set of 10 words and 10 non-words were presented on the screen in random order. The participants had to perform a lexical decision for each of these items by pressing the yes/no keys. An appropriate warning message appeared on the screen when the answer was wrong. Then, four training story problems were presented using the same procedure as for the rest of the experiment.
Results and discussion
The overall error rate for positive lexical decision trials was 3.4%, with no significant difference between conditions or versions. All errors and decision latencies that were three standard deviations from the mean were discarded. The mean latencies are presented in Table 3.
Mean (and Standard Deviations) lexical decision latencies and reading times for numerical information sentences (in milliseconds) in Experiment 2
There was no main effect of condition, and no significant interaction between condition and version (Fs < 1). However, there was a significant effect of version by subjects, F1(1, 52) = 12.52, MSE = 3,209, p < .001, η p 2 = .19; and an effect that approached significance of version by items, F2(1, 30) = 3.4, MSE = 6,226, p = .07. Planned comparison between means using the Bonferroni adjustment with p < .05 indicated that the version effect was significant for both comprehension condition, F(1, 52) = 6.04, and solving condition, F(1, 52) = 6.47.
In the same way that in Experiments 1 and 2, participants in the solving condition read slower numerical information two sentences than participants in the comprehension condition; this effect was significant by subjects F1(1, 52) = 4.10, MSE = 330,988, but not by item F2(1, 30) = 1.86, p > .20. Similarly, numerical information three sentences were read slower by participants in the solving condition than those in the comprehension condition, F1(1, 52) = 13.43, MSE = 245962, p < .001, η p 2 = .20; F2(1, 30) = 4.30, MSE = 568,333, p < .05, η p 2 = .12. However, in the same way as in Experiments 1 and 2, there were no differences between conditions for numerical information 1 sentences. Again, total reading time for the story problem was longer in the comprehension condition than in the solving condition, F1 (1, 52) = 6,25, MSE = 23,453,533, p < .02, η p 2 = .11; F2 (1, 30) = 3.71, MSE = 25,738,172, p < .05, η p 2 = .11.
The results of this experiment confirm previous research by Lutz and Radvansky (Reference Lutz and Radvansky1997) by showing that failed goal information is more available to readers than completed goal information. But this experiment also extends their findings by revealing that this pattern of results is also observed when participants read under a problem solving rather than a narrative comprehension orientation. This result supports the idea that, in solving story problems, readers process situational information besides the mathematical information that is necessary to solve the problem. Furthermore, the reading time patters showed by participants differed as a function of condition, which again suggest that reading purpose influence how readers process texts.
Why do participants in the solving condition maintain more available a failed goal, even though that goal did not activate a problem schema? This effect can be accounted for within the framework of the theoretical models that stress the importance of the comprehension of the situation described in the story problem (e.g., Reusser, Reference Reusser, Mandl, De Corte, Bennett and Friedrich1990; Staub & Reusser, Reference Staub, Reusser, Weaver, Mannes and Fletcher1995). According to these models, the facts and events that constitute the world referred to in a story problem, such as the relations among the events of the story, would be the bridging element between the text base and the problem model. Considering the sample story problem given in Appendix, when participants in the solving condition processed that “Maria wanted to give her mother a present”, this goal sentence did not activate the problem schema. However, the problem schema was triggered by the subsequent numerical information 1 and 2 (“Maria got 175 euro” and “She spent 60 euro” would support the activation of a Change schema). In this context, once this schema is activated, an unsatisfied goal (“Maria spent money on a purse but did not seem to have enough”) not only is “foregrounded” until the goal is satisfied (Zwann & Radvansky, Reference Zwaan and Radvansky1998), but also would draw the reader’s attention to look for actions to increase or decrease quantities, since the problem schema also needs to be completed.
According to this reasoning, one could hypothesize an interaction effect between condition (comprehension vs. solving) and version (failed goal vs. completed goal) also in this experiment, since an activated schema draws the readers’ focus to satisfy the goal Footnote 1 ; that is to say, it would be feasible to predict combined effects of (unsatisfied) goals and (incomplete) arithmetic schemata in order to explain the goal availability, as in the previous experiments. Nevertheless, the role played by the goal in relation to the arithmetic scheme is different in this experiment. In the previous experiments, the goal processing explicitly or implicitly triggered the problem schema, which explains a larger consistency effect under a solving orientation than under a comprehension orientation, given that the inconsistency is not only situational but also arithmetic. In this experiment, however, the problem schema was separated from the goal information. Therefore, to maintain the goal available would help reader to link text being processed with the problem model, but combined effects of goals and schemata would not be predictable, since the availability of goal-related information was assessed apart from activated schema. Nonetheless, this vision of situational information as a bridging element between the text base and the problem model needs further research.
General Discussion
The aim of the present set of studies was to investigate whether people generate inferences related to construction of the situation model during solving story problems. To this end, we designed three experiments in which participants were asked to read story problems under two different instructions: reading for comprehending or reading for solving the story problems. The story problems were presented in the context of a story about realistic situations with goals for the characters and actions to achieve it. Whether readers build a situation model was measured by the extent to which they were sensitive to goal information, typically a critical component of a situation model. In all three experiments, the results showed that participants in the comprehension condition processed situational information by keeping track of the characters’ goals. This replicates the results reported by many other studies in which narrative texts were used. As such, this finding supports the validity of the texts used in our experiments. However, and more important to the purpose of this study, we showed for the first time that a similar pattern of goal information availability happened when participants had to solve the story problems. These results are interesting because they suggest that the generation of goal inferences occurs even when explicit instructions indicate it is not necessary.
The results of our experiments fit nicely with those studies that emphasize the role of situation model construction in word problem solving (Reusser, Reference Reusser, Mandl, De Corte, Bennett and Friedrich1990; Staub & Reusser, Reference Staub, Reusser, Weaver, Mannes and Fletcher1995; Thevenot, Reference Thevenot2010; Thevenot et al., Reference Thevenot, Devidal, Barroulillet and Fayol2007). According to these studies, to understand a problem it is important to pay attention not only to specific mathematical information (i.e., the problem schema) but also to the situation and the events described in the text. Although participants in the solving condition were instructed to adopt a problem-solving strategy that does not require the development of a situation model, we emphasize that situational information, at least information related to goals of characters, remained available during the processing of the problems. In this way, the construction of the problem model could be facilitated by situational information, since readers need to engage in some sort of inferential process to attempt to reestablish coherence when they detect an inconsistence (Experiments 1 and 2), and need to hold a goal active when it has not been completed (Experiment 3).
These findings add to a growing body of literature that suggests that, during reading, a situation model is created and constantly updated by the continual integration of incoming information within the evolving discourse representation in memory (Graesser et al., Reference Graesser, Singer and Trabasso1994; Myers & O’Brien, Reference Myers and O’Brien1998; O’Brien & Myers, Reference O’Brien, Myers, Goldman, Graesser and van den Broek1999). Part of that integration process involves the reactivation of “backgrounded” portions of text and general world knowledge related to the information currently active in memory. That is, when new information is encoded into working memory, readers search long-term memory (both episodic and general world knowledge) for information that is related to the information in working memory to develop a complete representation of the situation described by the text Footnote 2 .
In addition, the findings of this study also revealed that, similar to other texts, expectations generated by reading purpose influence how readers process story problems. This is consistent with the assumption that comprehension is guided by a control system that comprises information about the reading goals associated with a particular reading task (van den Broek et al., Reference van den Broek, Lorch, Linderholm and Gustafson2001; van Dijk & Kintsch, Reference van Dijk and Kintsch1983; Zwaan, Reference Zwaan1994). How can this occur? One possible explanation is that readers may engage in different strategies or hold different standards of coherence, depending on the context and/or purpose of the reading situation (e.g., van den Broek et al., Reference van den Broek, Lorch, Linderholm and Gustafson2001; van den Broek, Risden, & Husebye-Hartman, Reference van den Broek, Risden, Husebye-Hartmann, Lorch and O’Brien1995). Here, the notion of standards of coherence can be useful due to it reflects the degree of comprehension that a reader attempts to attain during the reading of a text, and therefore provide an explanatory account of how the reader’s goals influence the reading process (van den Broek et al., Reference van den Broek, Lorch, Linderholm and Gustafson2001), given that these goals induce readers to adopt particular standards to maintain as they proceed through the text. Therefore, it may be that participants of our study adopted different standards of coherence when they read to comprehend the story problems than when they read for solving them. This would explain the reading time patterns showed by participants in the comprehension and solving conditions. Thus, the goal of reading to solve the problems would draw the reader’s attention to the construction of a mental representation that emphasizes connections to arithmetic knowledge. These connections would be achieved at the situation model level rather than at the text-base level, since they are not directly represented by the explicit text information but rather supported by the reader’s prior knowledge. The activation of relevant prior arithmetic knowledge would facilitate the early construction of a mental model as a schema to be filled and, therefore, would reallocate cognitive resources to the processing of incoming information that is directly relevant to the model. In the same way, this would also reduce the allocation of cognitive resources to the processing of irrelevant information, which would then shorten the reading times of this information. These activation patterns would also explain why the goal availability in the solving condition was stronger when the character’s goal was closely related to the activation of the problem schema (Experiments 1 and 2) than when the character’s goal did not activate any problem schema (Experiment 3). In contrast, reading with the purpose of comprehend a narrative text would draw the reader’s attention to the creation of a coherent causal network of the narrated events and, therefore, to the identification of the causal path of events from beginning to end. In this case, readers would allocate more processing resources to maintain the causal coherence through the text, which would lengthen the reading times of those segments related to the causal structure (goals, actions, and outcomes).
There are several limitations to the current study that prompt further research. First, it could be argued that the story problems have much more of a narrative structure than one can expect from regular (typical) word problems, in which the situational information is usually not relevant to the solving process (Orrantia, González, & Vicente, Reference Orrantia, González and Vicente2005; Vicente & Orrantia, Reference Vicente and Orrantia2007) and, in many cases, this information can even be a distraction. However, it is necessary to consider the fact that many word problems describing real-world situation are embedded in events that take place within a particular time and space and with goals and motivations for the characters. Solving these word problems presumably requires understanding the situation described and constructing a mental model for the events. In fact, there is converging evidence demonstrating that students exhibit a strong tendency to exclude real-world knowledge from their solution of word problems presented in realistic situations (for a review see Verschaffel, Greer, & de Corte, Reference Verschaffel, Greer and de Corte2000). Because most of these studies have focused on off-line measures and learning outcomes, understanding to what extent solvers construct mental models for the underlying situation during problem solving is an important question to address. The present study is a step in that direction.
A second limitation of our study might be that the word problem set was easy-to-solve for the sample. Given that the focus of this study was on whether solvers process information related to situation model construction in word problem solving, the mathematical difficulty level was not considered. It could be possible, however, that if more complex problems were used, then solvers would potentially have more difficulties to process the situational information since these problems demand more cognitive resources; furthermore, focusing on the situational information could have negative implications on the ability to solve these complex problems. Alternatively, it is also possible that processing situational information could have positive effects on more complex problems, since situational information would be the bridging element between text being processed and the problem model (Reusser, Reference Reusser, Mandl, De Corte, Bennett and Friedrich1990). Although some studies support this second possibility (e.g., Orrantia et al., Reference Orrantia, Tarín and Vicente2011), this is an open question that needs further research.
Third, the present study did not assess the impact of reading goal on participants’ responses to the comprehension or solving questions, due to their nearly perfect performance. Thus, future research should examine to what extent the results found in this study using on-line measures are transferable to off-line measures, and then analyze whether the readers’ memory for the text is affected by what occurs during reading, as suggested by previous research (e.g., McCrudden et al., Reference McCrudden, Magliano and Schraw2010; van den Broek et al., Reference van den Broek, Lorch, Linderholm and Gustafson2001). For example, it is possible that memory for text portions including numerical information is better for the readers with a resolution purpose than for their counterparts with a comprehension purpose, as stated by reading times of sentences containing numerical data, whereas memory for situational information would be similar for both purposes, since both groups process situational information during reading. However, it is also possible that, in the solving condition, influence of situational information would affect during problem online processing, at a stage preceding the construction of mathematical problem representation. Once mathematical representation has been constructed, situational representation would lose strength in memory traces, given that the final representation would be more influenced by the mathematical model that participants created to solve the problem. In this case, memory for situational information would be better for readers in the comprehension condition. Future research could examine how reading goal affects readers/solvers’ memory in this type of texts.
Example of experimental passage (Experiments 1 and 2)
Episode 1
Introduction
John was a shepherd who take care a flock of Spanish sheep.
The shepherd had a flock of 67 sheep.
Explicit goal statement (consistent version in Experiment 1)
The shepherd wanted to increase the size of the flock because this year there was good fodder.
Explicit goal statement (inconsistent version in Experiment 1)
The shepherd wanted to decrease the size of the flock because this year there was few fodder.
Implicit goal statement (consistent version in Experiment 2)
As this year there was good fodder,
Implicit goal statement (inconsistent version in Experiment 2)
As this year there was few fodder,
Backgrounding sentences
The shepherd went to a cattle market
In the cattle market people traded with different kinds of sheep
There was Spanish sheep and sheep from other countries.
The shepherd saw the price of the Spanish sheep.
Target line
Then the shepherd decided to buy 20 sheep
Post-target line
and he put them into his truck
Follow-up line
There was a bargain for the sheep.
Episode 2
One evening the shepherd saw a pack of wolves in the area
The wolves were hungry.
The flock was cornered by the wolves.
Then the wolves devoured 8 sheep
Question
Comprehension condition
Did the shepherd sell any sheep in the cattle market?
Solving condition
Is the operation correct to solve the problem?
67 + 20 – 8 =
Example of experimental passage (Experiment 3).
Episode 1
Maria found that her mother’s birthday was coming soon.
She really wanted to give her mother a present. (Goal statement)
Maria got 175 euros from her bank account
She went to the department store.
She spent 60 euros on a pretty purse.
Completed goal
She gave her mother the purse
Her mother was very happy.
Failed goal
But she thought this present will not be enough
She was disgusted
Episode 2
Several days later, María saw her friend knitting.
Then she decided to knit a sweater
She went to a haberdashery
She bought some wool
Episode 3
She pressed the sweater.
Completed goal
She folded the sweater carefully.
Failed Goal
And she gave the sweater and the purse to her mother.
Now María has 25 euros
Comprehension condition
Did Maria find that her mother’s birthday was coming soon?
Solving condition
Is the following operation correct to solve the problem?
175 – 60 + 25 =
