2.1 Applying organizational strategies to compensate for the inadequacy of prose

Information is usually presented linearly via streams of words and sentences. Similarly, arguments are generally presented as “prose”, which is considered as “the medium of philosophical argumentation” (van Gelder, Reference Van Gelder2002: 85). However, students are likely to have difficulty cognitively processing the prose for the purpose of learning and later recall (Dwyer, Reference Dwyer2011). That is, to make the central point of the main argument explicit requires serious thinking and some reading (Davies, Reference Davies2009). Harrell (Reference Harrell2005) considers one reason to be the inability of students to recognize the argument presented in the text and therefore misunderstanding it as a story instead. Furthermore, argumentative prose “contains many more sentences than just the propositions that are part of the argument, but also … proceeding necessarily linearly, the prose obscures the inferential structure of the argument” (Harrell, Reference Harrell2004: 2). Accordingly, assimilating relevant information in argumentative prose (e.g. propositions supporting a claim in an argument that are often located in different paragraphs) is problematic (van Gelder, Reference Van Gelder2003), impeding successful comprehension and recall of materials. In terms of complex arguments, students are even engaged in “circular verbal dispute or maze-like structure of forbidding volumes of prose” (Monk, Reference Monk2001: 8). Likewise, Davies (Reference Davies2012) asserts that prose is not an efficient means of transmitting complex information as a result of memory limitations.

As Meyer, Brandt and Bluth (Reference Meyer, Brandt and Bluth1980) point out, analyzing the structure of the text by identifying its organizational structure facilitates memorization of the most important points. Moreover, “additional processing of superordinate propositions and their interrelationships increases the depth with which they are processed” (Craik & Lockhart, 1972, cited in Meyer et al., Reference Meyer, Brandt and Bluth1980: 78) because it is the structure of the text that dictates how the superordinate and subordinate propositions are logically linked. Therefore, knowledge of text structure simplifies text comprehension and assists later recall (Watson, Gable, Gear & Hughes, Reference Watson, Gable, Gear and Hughes2012).

Among organizational strategies necessary for exploring the text structure and thereby promoting recall and retention of textual materials such as hierarchical summarization, other evidence-based methods including graphic organizers have been also recommended (Watson et al., Reference Watson, Gable, Gear and Hughes2012). Graphical organization of textual information converts text into visual arrays that help learners by portraying the logical relationship between key concepts of the text to provide a mental plan for comprehending (Jiang & Grabe, Reference Jiang and Grabe2007). Graphic organizers, therefore, help students develop a schema (Wittrock, Reference Wittrock1992), connect prior information to the textual information they are reading (Mayer, Reference Mayer1984), and successfully recall and remember information (Griffin, Malone & Kameenui, Reference Griffin, Malone and Kameenui1995). However, they are believed to act more effectively in the post-reading than the pre-reading stage (Griffin & Tulbert, Reference Griffin and Tulbert1995). In the post-reading stage, they are used for the purpose of evaluating students’ level of comprehension in addition to improving their recall, retention, as well as the ability to summarize key ideas (Manoli & Papadopoulou, Reference Manoli and Papadopoulou2012).

Successful recall of information requires such essential memory processes as encoding, storage, and recall of information in memory (Bernstein, Reference Bernstein2016). In the process of recall, stored information such as stimulus and event is said to be recollected by responding to an external stimulus. Recall and retention are among the main processes of memory and are related to each other as well as factors such as learning and capacity limit of memory (Amin & Malik, Reference Amin and Malik2013) based on cognitive research results. They refer to remembering the same information just at different times; that is, immediate recall and delayed retention in the present study. According to Paivio’s (Reference Paivio1983) dual coding theory, verbal stimuli, which are stored in the form of propositions in the verbal code, and visual stimuli, stored in visual code, provide two representations that increase the probability of recalling information. These verbal and visual codes together promote retention of information (Schreiber & Verdi, Reference Schreiber and Verdi2003) that is often assessed by a cued recall test at a fixed interval (Lindsey, Mozer, Cepeda & Pashler, Reference Lindsey, Mozer, Cepeda and Pashler2009).

Although the length of the retention interval is usually taken as a robust factor in retention success (e.g. Bankó & Vidnyánszky, Reference Bankó and Vidnyánszky2010), it can be affected by factors other than time per se, including situational factors as well as the task itself (Naylor & Briggs, 1961, cited in Arthur, Bennett, Stanush & McNelly, Reference Arthur, Bennett, Stanush and McNelly1998). Moreover, increasing the length of retention intervals makes it more educationally realistic in comparison with short laboratory intervals of hours or a few days (Richland, Linn & Bjork, Reference Richland, Linn and Bjork2007). As Cepeda et al. (2006) recognized in their meta-analysis, most studies have examined retention intervals in laboratory learning environments and failed to assess performance on tests of recall after delays of weeks, months, and years; that is, “across educationally realistic retention intervals” (Richland et al., Reference Richland, Linn and Bjork2007: 264).

In addition to applying organizational strategies for rendering text to diagrams, Dwyer et al. (Reference Dwyer, Hogan and Stewart2010) suggest that if propositions within a complex argument are represented in reasonably small chunks, the subsequent encoding and recall of those arguments will turn out to be more successful. In essence, chunking is central to the concept of short-term memory capacity (Gobet & Clarkson, Reference Gobet and Clarkson2004). Anderson (Reference Anderson1983) claimed that working memory can sometimes include over 20 active chunks at one time. However, further research indicated that people cannot process and remember more than seven plus or minus two (a “magic number”) pieces of information, which is known as the chunking limit (Miller, Reference Miller1956). Revising Miller’s magical number, Cowan (Reference Cowan2001: 87) asserted that it “was meant more as a rough estimate and a rhetorical device than as a real capacity limit”. He estimated that the chunk size would probably include three to four items. Duration rate for the items in working memory was regarded as another factor affecting its content irretrievability capacity (Peterson & Peterson, 1959). Cognitive scientists, therefore, considered chunks (i.e. β=a number from two to six), items (i.e. γ=from three to four), and their retrievable duration (δ=measured by seconds) as three necessary quantities in calculating the working memory irretrievability capacity and arranged them in a formula called alpha value (i.e. α=(β * γ) * δ). Accordingly, in their research, Gobet and Clarkson (Reference Gobet and Clarkson2004) asserted that short-term memory capacity, as estimated by the number of recalled chunks, was between three and 15 chunks at all skill levels. However, for those with higher capacities, the number of chunks is less than three; the number of items per chunk can increase to 15. Thus they concluded that our memory capacity may consist of around 30 items based on the alpha-value formula (i.e. two chunks, each containing 15 items).

Notably, research on the effects of converting text-based information into diagrams and maps shows that diagramming not only reduces the cognitive load (Pollock, Chandler & Sweller, Reference Pollock, Chandler and Sweller2002) but also positively affects recall and comprehension (Berkowitz, Reference Berkowitz1986; Oliver, Reference Oliver2009). In a six-week study, Berkowitz (Reference Berkowitz1986) explored the effect of mapping instruction on reading comprehension of six-grade students. He assigned the participants (n=99) into four experimental groups (i.e. map construction, map study, question-answering, text rereading). A free recall test as well as a short-answer test were then administered immediately after the study and again two weeks later. The results showed that the map construction group gained higher scores compared to other groups, suggesting that map construction facilitates recall when students construct the maps themselves. In another study conducted by Suzuki, Sato and Awazu (Reference Suzuki, Sato and Awazu2008), the effectiveness of spatial graphical depiction of a sentence compared to linear sentential presentation was explored that suggested that graphical representation of textual information enhances ESL readers’ comprehension of sentences.

2.2 Argument mapping with pen and paper as well as computer

Due to difficulties learners have in making the highly abstract relationships in an argumentative prose explicit, argument mapping was introduced. Davies (Reference Davies2011) refers to the justification and purpose of using mapping tools in educational settings, suggesting that mapping increases the chance that meaningful learning (as opposed to rote learning) will occur. New information can be built on prior knowledge, thereby becoming more useable. Furthermore, mapping greatly improves recall and processing of information as well as meaningful engagement for learners. When students manage to represent complex textual information in a map, they are likely to have understood the relationship between ideas, as well as recalling and analyzing various propositions. Moreover, by asking students to construct maps, teachers are provided with helpful insights as to the students’ mental model of the given arguments (Butchart et al., Reference Butchart, Forster, Gold, Bigelow, Korb, Oppy and Serrenti2009).

As graphical organizers, argument maps provide a manageable representation of the structure of the sophisticated and ill-structured arguments that many people may find otherwise difficult to understand. Argument maps draw upon theories of visual and diagrammatic reasoning that explicitly depict the inferential structure of an argument (van Gelder, Reference Van Gelder2005). Arguments are suggested to be handled through argument mapping as they are demonstrated in “streams of words, whether written or spoken” (van Gelder, Reference Van Gelder2005: 44). Boxes and arrows are normally applied in diagramming arguments in which propositions such as reasons and objections are displayed within the boxes and the evidential relations among them are indicated via arrows (van Gelder, Reference Van Gelder2002; see Figure 1).

Figure 1

Example of an argument map created by Rationale (adapted from https://www.rationaleonline.com/)

Moreover, as a result of advances in computer technology, a number of software packages (e.g., Reason!Able, Araucaria, Athena, and recently Rationale) have recently been introduced in educational settings (van Gelder, Reference Van Gelder2007) in addition to traditional pen-and-paper methods of argument construction. CAAM is concerned with the use of a computerized program specifically designed to assist in the design of a structured argument through using concept structures and their relationships for the reasons of usability, complementation, and semi-formality (van Gelder, Reference Van Gelder2007). One such software is Rationale, developed by Austhink Company, which applies color coding to differentiate between various propositions of an argument (e.g. green for reasons and red for objections – the boxes labeled “because” and “but”, respectively, in Figure 1) as well as visual and spatial arrangement of textual information.

Applied research on the impact of CAAM on enhancing memory processes is limited. The only example at the time of writing this article includes Dwyer et al.’s (2010) experiment that compared the effect of prose reading versus argument mapping reading on psychology students’ memory and comprehension abilities. They assigned the psychology students (n=400) into six experimental groups including (1) a 30-proposition text, (2) a 30-proposition color argument map, (3) a 30-proposition monochrome argument map, (4) a 50-proposition text, (5) a 50-proposition color argument map, and (6) a 50-proposition monochrome argument map. The study took place in just three sessions, and the participants received no instruction on constructing maps of arguments but were just given a short lecture on argument mapping. Their comprehension was tested by asking whether a subset of propositions support or deny the main contention of a given claim. Moreover, they were asked to complete a fill-in-the blank test of recall. The results revealed that all experimental groups were similar in terms of comprehension; however, performance on the recall test was better for the 30-proposition group compared to the 50-proposition group. Students in both black-and-white and color map conditions had better memory performance than the text-only condition, although color map groups outperformed the monochrome groups significantly. The researchers therefore concluded that argument mapping enhances memory for arguments significantly when compared to pen-and-paper methods of argument presentation. However, their research took place within three weeks with no longitudinal CAAM training.

In such a context, several studies have explored the effects of applying graphic organizers such as semantic mapping and concept mapping on enhancing students’ reading comprehension (Asadollahfam & Shiri, Reference Asadollahfam and Shiri2013; Sadeghi & Taghavi, Reference Sadeghi and Taghavi2014). The results are promising, suggesting a positive correlation between training in mapping skills and the development of recall and comprehension. For example, Asadollahfam and Shiri (Reference Asadollahfam and Shiri2013) investigated the impact of training semantic mapping on EFL learners’ reading comprehension performance. Using a pre-/post-test control group design, their results revealed that semantic mapping instruction enhanced EFL learners’ reading comprehension of expository texts compared to the conventional reading group. However, at the time of writing, very few published studies concerning the impact of argument mapping instruction and reading in general and CAAM in particular on the development of EFL learners’ comprehension, recall, and retention has been reported. Therefore, this study contributes to the literature by addressing the impact of pen-and-paper versus computer-assisted argument mapping on EFL learners’ recall, retention, and comprehension skills after receiving a semester-long instruction on argument mapping.

This study investigated the following research questions:

1. 1. Does type of treatment (computer vs. pen and paper) have any significant effect on the performance of EFL majors on tests of comprehension, recall, and retention while controlling for possible effects of spatial and verbal intelligences?

2. 2. Does argument size (i.e. maps with differing numbers of propositions) have any significant effect on the performance of the participants on tests of comprehension, recall, and retention while controlling for possible effects of spatial and verbal intelligences?

3. 3. Do proficiency levels have any significant effect on the performance of the participants on tests of comprehension, recall, and retention while controlling for possible effects of spatial and verbal intelligences?

3.1 Participants

Participants included a sample of 120 EFL undergraduates, aged 18–27, who studied English translation at Payame Nur University in Aran and Bidgol (a city in the province of Isfahan), Iran. Almost one month before the commencement of the study, posters were distributed around the university campus, announcing and recruiting EFL majors for two extracurriculum reading classes. Initially, 139 students enrolled and took part in the classes, but 19 students were excluded because they failed to attend some of the training sessions (n=4), missed one of the tests (n=5), or were at an intermediate level of proficiency (n=10). They were randomly assigned to two experimental groups: the CAAM group were taught argument mapping via the Rationale software, and the conventional group practiced it via pen and paper. All 120 students were aware of the purpose of conducting the study and their contribution to the development of the project. Moreover, they were categorized into high and low levels of reading proficiency on the basis of their scores on a sample First Certificate in English (FCE) reading test.

3.2 Instruments

Several quantitative instruments were applied in the present study. First, a sample FCE reading test was used to assess the participants’ level of reading proficiency. The test includes three parts (multiple-choice, jumble-ordered, multiple-matching questions) comprising 30 questions and takes 60 minutes to complete. Since all the participants were to take part in a reading course infused with argument mapping or a pure reading course for the second experimental group, only the reading section of FCE was administered. As the distribution of FCE scores was not significantly different from the normal distribution, classifying the participants into high and low levels of proficiency was done based on the mean and standard deviation (SD) of their scores. The high proficiency group (n=60) included those scoring +1 SD above the mean, and the low proficiency group (n=60) consisted of those falling –1 SD below the mean; the intermediate group (n=10) was excluded from the study.

A software package for diagramming reasoning and mapping arguments, called Rationale (van Gelder Reference Van Gelder2007), developed by Austhink, was also used in this study. As van Gelder (Reference Van Gelder2007) suggests, it promotes reasoning because it is more useable for reasoning activities compared to prose. Furthermore, it is designed to enhance the brain’s ability to comprehend reasoning by complementing what it could already do imperfectly. It also aims to link the brain’s natural informality with the semi-formality of structured maps (Davies, Reference Davies2009). In this software package, information is displayed in color-coded boxes to distinguish among the central claim, reasons, objections, and rebuttals. Moreover, the strength of arguments can also be judged using the Evaluation section in the toolbar of the Rationale editor page. Some indicator phrases (e.g., because, however) are employed to determine the reasoning structure.

Another instrument included a sample of verbal and spatial subtests from the Differential Aptitude Tests (DAT; Bennett, Seashore & Wesman, Reference Bennett, Seashore and Wesman1986); the results entered the analysis as covariates. The reason for this was to see whether participants’ verbal and spatial reasoning abilities accounted for the differences in their comprehension, recall, and retention performances, as the reading and construction of argument maps is highly interrelated with the ability to reason both verbally and spatially (Dwyer et al., Reference Dwyer, Hogan and Stewart2010). Therefore, the chances are that students with various verbal and spatial reasoning ability levels approach argument map reading tasks in different ways. The verbal reasoning subtest of the DAT contains 40 tests that measure analogical reasoning by providing sentences with missing words completed through selecting from five word-pairs. The spatial reasoning subtest consists of 50 items that ask participants to visualize a three-dimensional object from a two-dimensional pattern. To make the object, they need to mentally fold cut-outs, and then choose the correct one among the four choices provided (Dwyer et al., Reference Dwyer, Hogan and Stewart2010).

The reading skill measures included tests of comprehension, recall, and retention. The cued recall test consisted of fill-in-the-blank questions aiming to assess participants’ recall for reasons and objections confirming or refuting the central claim of the arguments. It consisted of a set of 10 statements (regardless of the size of the argument selected) derived from the original study materials. The cue in each statement was specified by such conjunctions as because or but, followed by a set of blank lines. The comprehension test demanded understanding of the relationships between propositions in the argument structure. Participants were required to indicate if a subset of 10 of the propositions extracted from argument maps either supports (i.e. reason) or refutes (i.e. objection) the central claim. The same recall test developed by the researcher was applied as the retention test within a two-week interval after the first administration. This time interval was selected as Mackey and Gass (Reference Mackey and Gass2005: 149) recommend: “Often this is 1 week following the first posttest and then 2 weeks later and even 2 or 3 months later”.

3.3 Procedure

The study took place over 12 sessions, each lasting almost two hours. In session 1, 120 EFL majors took the reading section of a sample FCE test to be divided into low and high levels of reading proficiency based on their scores. They were also randomly assigned to the first experimental group (i.e. software group) or the second experimental group (i.e. pen-and-paper group), each having 60 students (i.e. 30 from the low and 30 from the high reading proficiency groups). In the second session, the spatial and verbal subtests of the DAT were administered. The course actually began in the third session, and lectures as well as classroom activities were presented to the two experimental groups. The content of the training materials was the same for the two groups except that in-class handouts and exercises differed based on the conditions of Rationale or pen-and-paper presentation. In this session, technical argument mapping terms were introduced, and the two groups were instructed how to use the software or pen and paper for making maps of the argument. A rudimentary familiarity with computers was adequate for working with the software. From sessions 4 to 10, mapping skills and the way to apply such skills through the examples and exercises when reading and interpreting arguments of various topics were explained. They were also shown how to extract the structure of the arguments, judge their logical strength, relevance, and credibility, and recognize the sources of arguments. In the 11th week, the study materials, which included two maps with differing numbers of propositions from an argument called “Advertising: Good or bad?”, were randomly presented to high and low proficiency students via computer for the software group and via paper for the pen-and-paper group. The small version of the article containing 30 propositions was developed based on Gobet and Clarkson’s (Reference Gobet and Clarkson2004) fresh working memory capacity estimate. The large version included the total number of constituent propositions of the study material (i.e. 45). As students in both experimental groups had already been classified into high (n=60) and low (n=60) reading proficiency levels, 30 students at high proficiency levels in both groups (n=15 in each group) received large argument maps, whereas 30 students at low proficiency levels in both groups were randomly given small argument maps. Students were allotted almost 15 minutes to read the maps and were told that they would be tested. Then, the argument maps were collected and relevant cued-recall tests were administered with 15 minutes given for completion. Both groups were then provided with the comprehension tests, which were administered similar to the recall test, again with 15 minutes allotted to the test completion. To check retention, the same cued-recall test was administered to the students in both experimental groups 14 days after the immediate post-test.