1. Introduction
Research on composing processes has led to rich descriptions of L1 and L2 composing, highlighting important differences between less proficient and more proficient L1 and L2 writers (Skibniewski & Skibniewska Reference Skibniewski and Skibniewska1986; Jones & Tetroe Reference Jones, Tetroe and Matsuhashi1987; Skibniewski Reference Skibniewski1988; Whalen & Menard Reference Whalen and Menard1995; Chenoweth & Hayes Reference Chenoweth and Hayes2001; Roca de Larios, Marín & Murphy Reference Roca de Larios, Marín and Murphy2001; Roca de Larios, Manchόn & Murphy Reference Roca de Larios, Manchόn and Murphy2006; Manchόn & Roca de Larios Reference Manchόn and Roca de Larios2007; Roca de Larios et al. Reference Roca de Larios, Manchόn, Murphy and Marín2008). One such difference is the allocation of planning time during the composing process: more proficient writers—both L1 and L2—devote a greater portion of total composing time to online planning (i.e., planning that occurs during the composing process), typically in the initial stages of the composing process (van den Bergh & Rijlaarsdam Reference van den Bergh, Rijlaarsdam, Torrance and Galbraith1999; van der Hoeven Reference van der Hoeven, Torrance and Galbraith1999; Rijlaarsdam & van den Bergh Reference Rijlaarsdam, van den Bergh, MacArthur, Graham and Fitzgerald2006; Manchόn & Roca de Larios Reference Manchόn and Roca de Larios2007). However, online planning is a highly individual process and may not be open to common types of planning used as instructional interventions. In contrast, pre-task planning (i.e., planning that occurs prior to composing) can be more easily manipulated through instructional intervention, but has received little research attention, particularly among L2 writing researchers.
A handful of recent L2 writing studies (Ellis & Yuan Reference Ellis and Yuan2004; Ong & Zhang Reference Ong and Zhang2010; Johnson et al. Reference Johnson, Mercado and Acevedo2012) have investigated pre-task planning and its impact on L2 writers’ language production, framing such studies as attempts to unify Kellogg's model of working memory in L1 writing (Reference Kellogg, Levy and Ransdell1996) with the Cognition Hypothesis (Robinson Reference Robinson2001, Reference Robinson2005, Reference Robinson and Robinson2011a, Reference Robinson2011b) and/or the Limited Attentional Capacity Model (Skehan Reference Skehan1998; Skehan & Foster Reference Skehan, Foster and Robinson2001). These studies have achieved very different results, making it unclear what impact—if any—pre-task planning may have on L2 writers’ language production. For example, Ellis & Yuan (Reference Ellis and Yuan2004) found pre-task planning to have a strong positive effect on L2 writing fluency (d = 1.45) and grammatical complexity (d = 4.42), whereas Ong & Zhang (Reference Ong and Zhang2010) found increases in pre-task planning time to be associated with decreases in L2 writing fluency and lexical complexity.Footnote 1 The difference in findings between Ellis & Yuan (Reference Ellis and Yuan2004) and Ong & Zhang (Reference Ong and Zhang2010) motivated a large scale study (Johnson et al. Reference Johnson, Mercado and Acevedo2012) on pre-task planning and its impact on the fluency and complexity of L2 written production.
In a study on the English-language writing of 914 learners of English as a foreign language, Johnson et al. (Reference Johnson, Mercado and Acevedo2012) found pre-task planning to have no effect on grammatical and lexical complexity and a minimal effect (d = 0.26) on writing fluency. Specifically, organization pre-task planning was found to significantly reduce sentence length but was associated with non-significant increases in total words produced. In an effort to explain the difference between their findings and the findings of previous research on pre-task planning and its impact on L2 writing fluency, grammatical complexity, and lexical complexity, Johnson et al. (Reference Johnson, Mercado and Acevedo2012) suggested the following:
1. Written language production differs fundamentally from oral language production. For this reason, pre-task planning may not impact the fluency or complexity of written L2 production, contrary to the predictions of the Limited Attentional Capacity Model (Skehan Reference Skehan1998; Skehan & Foster Reference Skehan, Foster and Robinson2001) and the Cognition Hypothesis (Robinson Reference Robinson2001, Reference Robinson2005, Reference Robinson and Robinson2011a, Reference Robinson2011b).
2. The results of previous L1 and L2 writing research which found that pre-task planning impacted writing fluency and/or the complexity of written language production were, perhaps, affected by the participants' educational experience and/or the participants' experience with the written genre elicited by the research materials.
3. The working memory resources of less proficient L2 learners may be overtaxed by the translation process of composing. For these learners, pre-task planning may have no measurable impact on their written language production because they have not achieved a threshold of general L2 proficiency. Pre-task planning may impact the fluency and/or complexity of written language production only among writers who have achieved such a threshold.Footnote 2
This paper investigates the third explanation—a hypothesized threshold—by replicating the study conducted by Johnson et al. (Reference Johnson, Mercado and Acevedo2012) among a group of L1 writers. From a theoretical standpoint, if replication of the original study demonstrates some effect of pre-task planning on the fluency, lexical complexity, and/or grammatical complexity of language production among writers who have presumably achieved a threshold of proficiency in the target language, Kellogg's model of working memory in L1 writing (Reference Kellogg, Levy and Ransdell1996) may require revision in order to more accurately describe L2 writing processes and systems and their interaction with working memory resources.
The following sections describe the theoretical underpinnings of the original study as well as the current replication study—specifically Kellogg's model of working memory in L1 writing (Reference Kellogg, Levy and Ransdell1996). The original study is then presented in some detail before describing (a) the nature of the hypothesized threshold of target language proficiency, (b) theorized differences in the composing process of L1 and L2 writers, and (c) a replication study which set forth to better understand whether pre-task planning and its impact on written language production may be mediated by a threshold level of general proficiency in the target language.
2. Working memory in L1 writing (Kellogg Reference Kellogg, Levy and Ransdell1996)
Based on Baddeley's model of working memory (Reference Baddeley1986, Reference Baddeley2007), the work of Flower & Hayes (Reference Flower, Hayes, Gregg and Steinberg1980), as well as on Kellogg's own research (Reference Kellogg1987a, Reference Kellogg1987b, Reference Kellogg1988, Reference Kellogg1990), Kellogg's model of working memory and its involvement in L1 writing describes the demands of various composing systems and processes on working memory resources. The writing systems of formulation, execution, and monitoring, are thought to place very different demands on limited working memory resources. Execution—the coordination of muscle movements necessary to encode language on the page—is thought to place almost no demand on the adult L1 writer's working memory capacity.Footnote 3 In contrast, monitoring—the assessment of the developing text against rhetorical goals—is theorized to place greater demand on working memory capacity. The third system—formulation—is thought to place the greatest demands on working memory resources, as the formulation system requires the coordination of two further processes: (a) translation—the encoding of ideas into language—and (b) planning.
The planning process comprises three planning sub-processes which describe the writer's varying attention to (a) idea generation—thinking of ideas to include in the text, (b) organization—arranging ideas in the text, and (c) goal setting—attending to the audience and the purpose for writing. If working memory is preoccupied by any one of these planning sub-processes, translation must be managed by the remaining working memory resources and may suffer as a result. Kellogg's model implicitly suggests that relieving working memory load through pre-task planning should result in increases in writing fluency as well as the use of more complex language.
Because the effect of pre-task planning on L1 written production is theorized to be similar to the effect of pre-task planning on L2 oral production (Skehan Reference Skehan1998; Skehan & Foster Reference Skehan, Foster and Robinson2001; Robinson Reference Robinson2001, Reference Robinson2005, Reference Robinson2011a, Reference Robinson2011b), L2 writing researchers have invoked Kellogg's (Reference Kellogg, Levy and Ransdell1996) model to describe the involvement of working memory in L2 composing processes (Ellis & Yuan Reference Ellis and Yuan2004; Ong & Zhang Reference Ong and Zhang2010; Kormos Reference Kormos2011; Johnson et al. Reference Johnson, Mercado and Acevedo2012). However, prior to the study by Johnson et al. (Reference Johnson, Mercado and Acevedo2012), empirical L2 writing research in the area had not addressed the individual effect of each pre-task planning sub-process.
The specific impact of each pre-task planning sub-process on language production has been only slightly better isolated and examined in L1 writing research. L1 writing research among children has examined the impact of instructional interventions on L1 writing strategies—among them idea generation, organization, and goal setting—to determine that such interventions improve writing fluency and holistic writing quality (Sexton, Harris & Graham Reference Sexton, Harris and Graham1998; Page-Voth & Graham Reference Page-Voth and Graham1999; De La Paz & Graham Reference De La Paz and Graham2002; Troia & Graham Reference Troia and Graham2002; Graham & Perin Reference Graham and Perin2007). However, because pre-task planning is addressed in such interventions as one ingredient in a cocktail of strategies, the specific effects of idea generation, organization, and goal setting cannot be singled out.
In contrast, Kellogg (Reference Kellogg1987b, Reference Kellogg1988) has examined organization pre-task planning—particularly outlining—among L1 writers in some depth and has found organization pre-task planning to have some impact on L1 writing fluency.Footnote 4 Furthermore, Kellogg (Reference Kellogg1987b, Reference Kellogg1988) has noted significant effects of organization pre-task planning on analytic assessments of L1 writers’ language use, suggesting—albeit indirectly—that organization pre-task planning may impact the grammatical and lexical complexity of L1 writers’ language production.
The results of the L1 writing studies noted thus far suggest that pre-task planning in general, and specifically organization pre-task planning, positively impact writing fluency and, indirectly, the complexity of written language production. However, as noted previously, L2 writing studies examining the impact of pre-task planning on the fluency and complexity of L2 written production have achieved very different results, suggesting strong positive effects of pre-task planning on fluency and grammatical complexity (Ellis & Yuan Reference Ellis and Yuan2004) or negative effects of pre-task planning on fluency and lexical complexity (Ong & Zhang Reference Ong and Zhang2010). However, a recent large scale study by Johnson et al. (Reference Johnson, Mercado and Acevedo2012) suggests that the impact of pre-task planning on the language production of L2 writers may be minimal at best.
3. The original study
3.1 Method
In order to better understand the specific impact of each of the pre-task planning sub-processes, Johnson et al. (Reference Johnson, Mercado and Acevedo2012) examined the L2 writing of a homogeneous group of 914 Spanish-speaking learners of English as a foreign language at a large bi-national center located in Lima, Peru.
3.1.1 Materials and procedures
Table 1 provides a brief overview of the materials used in the original study. Interested readers are referred to the original study (Johnson et al. Reference Johnson, Mercado and Acevedo2012) where the materials are presented in full. What follows is a brief description of the materials, their development, and purpose.
A brief description of the materials and treatment conditions in Johnson, Mercado & Acevedo (Reference Johnson, Mercado and Acevedo2012)
3.1.2 The writing prompts
In order to control for the potential effect of topic knowledge on the lexical features of the participants' texts, two writing prompts were counterbalanced such that roughly half of the participants wrote in response to each of the prompts (see Appendix).
3.1.3 Pre-task materials
Materials designed to elicit focus on a specific sub-process of pre-task planning (idea generation, organization, and goal setting) were developed through a series of small-scale pilot studies. In order to keep time on task and transcription time constant, a control task was created that was thematically related to each of the writing prompts.
In order to counterbalance the two writing prompts, ten packets were created to combine each of the writing prompts with each of the pre-task planning activities. Participants were presented with one of the two counterbalanced writing prompts, followed by one of the following pre-task activities.
3.1.3.1 Control condition
Participants in the control condition were presented with the writing prompt, followed by a vocabulary matching task which was thematically related to the writing prompt. The participants were instructed to take ten minutes to match each vocabulary item to its definition.
3.1.3.2 Idea generation condition
The materials for the idea generation condition presented the writing prompt, then instructed the participants to take ten minutes to list as many ideas as possible to help them write an essay in response to the prompt.
3.1.3.3 Organization condition
The materials for the organization pre-task planning condition presented the writing prompt, then asked the participants to take ten minutes to organize an essay in response to the prompt. The remainder of the page was a worksheet which provided the participants with a template for a hierarchical outline, which the participants were asked to complete.
3.1.3.4 Goal setting condition
The materials for the goal setting pre-task planning condition presented the writing prompt, then instructed the participants to take ten minutes to answer questions which elicited the participants' attention to their purpose for writing, their audience, and how best to achieve their purpose.
3.1.3.5 Goal setting plus organization condition
Because previous pilot efforts had suggested that a combination of goal setting and organization pre-task planning increased grammatical complexity, additional materials were designed to focus participants' attention on a combination of these two pre-task planning sub-processes. The materials for the goal setting plus organization pre-task planning condition were a hybrid of the materials used in the goal setting and organization pre-task planning conditions. Instructions asked the participants to first read the prompt, then take ten minutes to answer questions about their purpose for writing and their audience and to organize their essay in response to the writing prompt.
The final two pages of each packet were identical across all five pre-task conditions. The writing prompt was presented a second time, followed by lined paper provided for the participants to compose their essays.
3.1.2 Procedures
The procedures were administered by the participants' regular classroom teachers who were provided with explicit written instructions from the researchers. The materials packets were allotted to each teacher to encourage random assignment to the pre-task planning conditions. As can be seen in Table 1, participants were relatively evenly distributed among the pre-task planning conditions.
After handing out the materials, the teachers instructed the participants in the procedures and time limits for the pre-task activities and for composing. The teachers were asked to allow a moment or two for questions before asking the participants to begin. They were then asked to give their students ten minutes to complete the pre-task activity assigned to them. After ten minutes, the teachers were asked to stop their students and to instruct them to begin working on their essays. The teachers were asked to monitor their students carefully to assure that their students used the full composing time and did not use dictionaries. After 30 minutes, the teachers asked their students to stop working on their essays and to turn them in. The teachers collected the pre-task worksheets and essays then answered a fidelity questionnaire which asked the teachers how closely they had followed the procedures and time limits. All materials were then delivered to the authors.
3.1.3 Text analysis
Each of the handwritten essays was then transcribed into a digital file and analyzed for features of writing fluency, grammatical complexity, and lexical complexity noted in previous L1 and L2 writing research to be associated with (a) academic prose, (b) writing quality, and (c) lexical development over time (see Table 2).
Measures of writing fluency, grammatical complexity, and lexical complexity examined in Johnson, Mercado & Acevedo (Reference Johnson, Mercado and Acevedo2012)
a Measured using Coh-Metrix (McNamara et al. Reference McNamara, Louwerse, Cai and Graesser2005).
b Measured using MTLD (McCarthy Reference McCarthy2005).
c Measured using Range (Heatley, Nation & Coxhead Reference Heatley, Nation and Coxhead2002).
3.1.3.1 Fluency
Fluency measures were selected from previous L1 writing research (Ransdell & Levy Reference Ransdell, Levy, Levy and Ransdell1996; Ransdell, Levy & Kellogg Reference Ransdell, Levy and Kellogg2002; Kellogg Reference Kellogg2004). Because writing time was held constant, total words produced and average sentence length were thought to be suitable indicators of writing fluency for this large group of L2 writers.
3.1.3.2 Lexical complexity
Measures of lexical complexity were chosen from studies examining the relationship between lexical diversity and holistic measures of L2 writing (Engber Reference Engber1995; Grant & Ginther Reference Grant and Ginther2000; Jarvis Reference Jarvis2002; Jarvis et al. Reference Jarvis, Grant, Bikowski and Ferris2003). Specifically, the measure of textual lexical diversity—or MTLD (McCarthy Reference McCarthy2005)—was chosen as a measure of lexical diversity because it has been demonstrated to be less affected by text length than other, more traditional measures (McCarthy & Jarvis Reference McCarthy and Jarvis2007, Reference McCarthy and Jarvis2010). An additional measure of lexical diversity was taken as the ratio of pronouns to noun phrases.Footnote 5
Measures of lexical complexity were chosen based on the results of studies which found that the use of lexis which appears less frequently in the English language was associated with the development of L2 proficiency (Laufer Reference Laufer1994; Laufer & Nation Reference Laufer and Nation1995) and/or holistic L2 writing quality (Coniam Reference Coniam1999; Lemmouh Reference Lemmouh2008; Crossley & McNamara Reference Crossley and McNamara2012; Johnson, Acevedo & Mercado Reference Johnson, Acevedo and Mercado2013). Specifically, three measures were examined: (a) the frequency of word types from the fourth 1,000 (4K) most frequent word families (normed to 100 words), (b) the frequency of word types from the fifth 1,000 (5K) most frequent word families (normed to 100 words), and (c) the COBUILD frequency rating—a measure generated by the online text analysis tool Coh-Metrix (McNamara et al. Reference McNamara, Louwerse, Cai and Graesser2005). The COBUILD frequency rating compares the vocabulary in an entered text against frequency information in the COBUILD corpus to generate a mean frequency rating for the whole text.
3.1.3.3 Grammatical complexity
Measures of grammatical complexity were chosen from previous L1 writing research and previous research on features of complexity associated with academic prose. Specifically, Flesch reading ease was chosen to facilitate comparison to the findings of Galbraith et al. (Reference Galbraith, Ford, Walker and Ford2005) and Rau & Sebrechts (Reference Rau and Sebrechts1996). Number of words preceding the main verb of clauses was chosen based on the findings of McNamara, Crossley & McCarthy (Reference McNamara, Crossley and McCarthy2010) and was thought to be associated with subordination—a traditional measure of grammatical complexity—or with the use of extended nominal phrases in the subject position of verbs—a feature of complexity associated with academic prose (Unsworth Reference Unsworth1997; Schleppegrell Reference Schleppegrell2001; Fang Reference Fang2006; Fang, Schleppegrell & Cox Reference Fang, Schleppegrell and Cox2006; Rimmer Reference Rimmer2008; Biber & Gray Reference Biber and Gray2010).
3.1.4 Data analysis
Prior to the main statistical analysis, the participating teachers’ answers to the fidelity questionnaire were examined to determine whether the teachers had given their students any additional time to compose. Many had, in fact, given their students additional composing time. However, a statistical comparison of the variables examined in the study indicated there was no significant effect of additional composing time on any of the variables examined.
To determine the effect of pre-task planning on the fluency, grammatical complexity, and lexical complexity of the participants' writing, two statistical tests were conducted:
a. a MANOVA—multivariate analysis of variance—(α = 0.05) was calculated to compare total words, average words preceding the main verb of clauses, MTLD, ratio of pronouns to noun phrases, the normed frequency of 4K word types, and COBUILD frequency rating;
b. a Kruskal-Wallis test (α = 0.05) was calculated to compare average sentence length, Flesch reading ease, and the normed frequency of 5K word types.
3.2 Results
For reference, descriptive statistics from the original study are presented in Table 3. The results of the MANOVA indicated no significant difference in any of the variables entered into the test. In contrast, the results of the Kruskal-Wallis test indicated a significant difference in average sentence length among the five groups. A series of Mann-Whitney U-tests (α = 0.01) located a significant difference between the control condition and the organization pre-task planning condition. As can be seen in Table 3, participants in the organization pre-task planning condition produced significantly shorter sentences than participants in the control condition. Such a result suggests that organization pre-task planning impeded L2 writing fluency. However, it should be noted that significant reductions in average sentence length were associated with non-significant increases in total words produced. Furthermore, the effect size—calculated as Cohen's d—indicated that the impact of organization pre-task planning on sentence length was minimal (d = 0.26). Taken together, these findings suggested that pre-task planning had no impact on grammatical and lexical complexity and a modest impact on writing fluency.
Means (standard deviations) of variables examined in Johnson, Mercado & Acevedo (Reference Johnson, Mercado and Acevedo2012)
* Comparison between the organization condition and the control condition was significant, p < 0.01.
4. A threshold
Based on these results, the authors came to the conclusions that pre-task planning (a) does not impact the lexical and grammatical complexity of L2 written language production and (b) has a minimal impact on the fluency of L2 written language production. Such a conclusion runs counter to the results noted in previous L1 writing research on pre-task planning (Kellogg Reference Kellogg1987a, Reference Kellogg1988, Reference Kellogg1990; Sexton et al. Reference Sexton, Harris and Graham1998; Page-Voth & Graham Reference Page-Voth and Graham1999; De La Paz & Graham Reference De La Paz and Graham2002; Troia & Graham Reference Troia and Graham2002; Graham & Perin Reference Graham and Perin2007) and previous L2 writing research on pre-task planning (Ellis & Yuan Reference Ellis and Yuan2004; Ong & Zhang Reference Ong and Zhang2010). To explain the difference between their findings and the findings of previous L1 and L2 writing researchers, Johnson et al. (Reference Johnson, Mercado and Acevedo2012) proposed a number of suggestions noted previously, the most relevant to the current study being the following:
The working memory resources of less proficient L2 learners may be overtaxed by the translation process of composing. For these learners, pre-task planning may have no measurable impact on their written language production because they have not achieved a threshold of general L2 proficiency. Pre-task planning may impact the fluency and/or complexity of written language production only among writers who have achieved such a threshold.
Recent research on L1 and L2 composing processes suggests evidence for such a threshold by demonstrating differences not only in the online planning and metacognitive processes of L2 writers (Whalen & Menard Reference Whalen and Menard1995; Chenoweth & Hayes Reference Chenoweth and Hayes2001; Ong Reference Ong2014), but also in the translation processes of L1 and L2 writers (Roca de Larios, Marín & Murphy Reference Roca de Larios, Marín and Murphy2001; Roca de Larios, Manchón & Murphy Reference Roca de Larios, Manchόn and Murphy2006). Studies using think aloud protocols have found the amount of composing time devoted to online planning to be similar when the same participants are observed composing in the L1 and the L2 (Jones & Tetroe Reference Jones, Tetroe and Matsuhashi1987; Manchόn & Roca de Larios Reference Manchόn and Roca de Larios2007; Skibniewski & Skibniewska Reference Skibniewski and Skibniewska1986). However, there is some research evidence to suggest that online planning in the L2 differs from online planning in the L1 (Whalen & Menard Reference Whalen and Menard1995; Chenoweth & Hayes Reference Chenoweth and Hayes2001) in that online planning in the L2 may focus primarily on the surface-level linguistic features of the text. Further, recent research on the effect of pre-task planning on the metacognitive processes of L2 writers (Ong Reference Ong2014) suggests that pre-task planning may further focus L2 writers’ attention on features of the language, a focus that may likely be the result of the additional demands of the translation process in L2 composing (Roca de Larios et al. Reference Roca de Larios, Marín and Murphy2001, Reference Roca de Larios, Manchόn and Murphy2006).
Using think aloud protocols and retrospective interviews, Roca de Larios et al. (Reference Roca de Larios, Marín and Murphy2001) examined the translation process of composing—termed ‘formulation’ in their study—as the participants composed in their L1 (Spanish) and in an L2 (English).Footnote 6 In a comparison of the distribution of the instances of the translation process over time, Roca de Larios et al. (Reference Roca de Larios, Marín and Murphy2001) noted that in all cases, the majority of composing time was devoted to the translation process. However, Roca de Larios et al. (Reference Roca de Larios, Marín and Murphy2001) noted a difference in the translation processes of participants composing in the L1 and the same participants composing in the L2. Based on their observations, Roca de Larios et al. (Reference Roca de Larios, Marín and Murphy2001) distinguished between two types of translation: (a) problem-solving translation—in which the writer struggles with the difference between his/her intended message and the lexical and syntactic forms needed to express it—and (b) fluent translation—in which difficulties with the language are circumvented through the use of known lexical and syntactic forms.
According to Roca de Larios et al. (Reference Roca de Larios, Marín and Murphy2001), the translation process of the same writers composing in the L1 and the L2, while similar across L2 proficiency levels, differed when composing processes were compared between the L1 and the L2. When composing in the L1, the ratio of fluent translation episodes to problem-solving translation episodes was five to one. However, when composing in the L2, the ratio of fluent translation episodes to problem-solving translation episodes was two to one, suggesting that the translation process of L2 writers (a) is fundamentally different from the translation process of L1 writers and (b) may make greater demands on the working memory resources of L2 writers. For these reasons, pre-task planning may not impact L2 written language production if a general level of proficiency in the L2—a threshold—has not been achieved by the L2 writer.
In a more recent examination of the translation process of L1 and L2 writers, Roca de Larios et al. (Reference Roca de Larios, Manchόn and Murphy2006) further noted two types of problem-solving translation: (a) compensatory problem solving—in which the writer uses existing lexical and syntactic resources to make up for a lack of automatic access to the necessary lexical/syntactic resources—and (b) upgrading problem solving—in which the writer attempts to better express his/her intended meaning. According to Roca de Larios et al. (Reference Roca de Larios, Manchόn and Murphy2006), the participants devoted a greater percentage of composing time to problem-solving translation when composing in the L2, as was the case in their earlier study (Roca de Larios et al. Reference Roca de Larios, Marín and Murphy2001). However, Roca de Larios et al. (Reference Roca de Larios, Manchόn and Murphy2006) noted differences in the proportion of upgrading problem solving to compensatory problem solving when they compared the participants across L2 proficiency levels. As L2 proficiency level increased, so too did the proportion of upgrading problem solving to compensatory problem solving, suggesting that the nature of translation changes as proficiency in the target language increases.
Intuitively, such a difference in translation processes is obvious. However, the difference is particularly salient to L2 writing research attempting to apply Kellogg's (Reference Kellogg, Levy and Ransdell1996) model of working memory in writing to the L2 composing process. Kellogg (Reference Kellogg, Levy and Ransdell1996) assumes a native—or native-like—ability to quickly retrieve lexical and syntactic forms from long-term memory. As the results of Roca de Larios et al. (Reference Roca de Larios, Marín and Murphy2001, Reference Roca de Larios, Manchόn and Murphy2006) suggest, the translation process in the L2 may place large demands on working memory capacity, presumably because L2 writers have not reached a threshold of general proficiency in the L2 which would allow for automatic retrieval of lexical and syntactic forms needed to express a given proposition. If Kellogg's model of working memory in writing is to be applied to L2 writing, two conclusions follow from the difference between L1 and L2 translation processes noted in Roca de Larios et al. (Reference Roca de Larios, Marín and Murphy2001, Reference Roca de Larios, Manchόn and Murphy2006):
a. In L2 writing, the translation process of composing taxes working memory resources such that pre-task planning may have no measurable effect on the writing fluency, grammatical complexity, or lexical complexity of lower proficiency L2 writers.
b. Because they have reached the hypothesized threshold of general proficiency in the target language—in this case English—L1 writers’ working memory resources will not be overtaxed by the translation process of composing.
Thus, pre-task planning should result in increases in writing fluency, grammatical complexity, and/or lexical complexity among L1 writers and L2 writers who have achieved the proposed threshold of general target language proficiency. Pre-task planning would allow writers who have achieved such a threshold of proficiency in the target language to devote reserve working memory resources to the fluent production of complex written language.
The hypothesized threshold may explain the difference in findings between Ellis & Yuan (Reference Ellis and Yuan2004), Ong & Zhang (Reference Ong and Zhang2010), and Johnson et al. (Reference Johnson, Mercado and Acevedo2012). The participants in Ellis & Yuan (Reference Ellis and Yuan2004) may have achieved such a threshold, and for this reason, pre-task planning improved their writing fluency as well as the complexity of their written language. On the other hand, the target language proficiency of the participants in Ong & Zhang's (Reference Ong and Zhang2010) study may have been below the hypothesized threshold, and for this reason, their writing fluency and the complexity of their written language was negatively affected by pre-task planning.Footnote 7
The hypothesized threshold might also explain the difference in findings between previous L1 writing research (Kellogg Reference Kellogg1987a, Reference Kellogg1987b, Reference Kellogg1988, Reference Kellogg1990; Sexton et al. Reference Sexton, Harris and Graham1998; Page-Voth & Graham Reference Page-Voth and Graham1999; De La Paz & Graham Reference De La Paz and Graham2002; Troia & Graham Reference Troia and Graham2002; Graham & Perin Reference Graham and Perin2007) and recent L2 writing research (Ellis & Yuan Reference Ellis and Yuan2004; Ong & Zhang Reference Ong and Zhang2010; Johnson et al. Reference Johnson, Mercado and Acevedo2012). It is likely that participants composing in their L1 have achieved such a level of proficiency.
In order to determine whether a threshold of general proficiency in the target language mediates the effects of pre-task planning, this study, an approximate replication of a previous study on pre-task planning and L2 writing (Johnson et al. Reference Johnson, Mercado and Acevedo2012), examined the writing of a small group of participants writing in their L1 in order to address the following research questions:
1. To what extent do pre-task planning sub-processes (idea generation, organization, and goal setting) affect the fluency, grammatical complexity, and lexical complexity of L1 writers’ texts?
2. To what extent can the findings noted in Johnson et al. (Reference Johnson, Mercado and Acevedo2012) be replicated among a group of L1 writers?
If, in fact, the effects of pre-task planning on L2 writing fluency and complexity are mediated by the hypothesized threshold, the effects of pre-task planning on the fluency and complexity of L1 written production should (a) be more pronounced than those noted among L2 writers by Johnson et al. (Reference Johnson, Mercado and Acevedo2012) and (b) be more consistent with those noted by Ellis & Yuan (Reference Ellis and Yuan2004).
5. The replication study
5.1 Method
5.1.1 Setting and participants
Unlike the original study, which examined the L2 writing of large, homogeneous groups of students of English as a foreign language, the current study examined the L1 writing of 90 students enrolled in mainstream English Language Arts (ELA) classes at a middle school in the Midwest of the United States. Because the participants in the study were minors, no biographical information was collected about the individual participants in order to safeguard their anonymity. As a group, the participants were enrolled in four, eighth-grade, mainstream ELA classes taught by the second author. Based on the participants' placement in mainstream ELA classes, it was assumed that the participants had native—or near-native—access to the lexical and syntactic forms necessary to have met the hypothesized threshold of target language proficiency.
5.1.2 Materials and procedures
The materials and procedures in the current study were identical to those of Johnson et al. (Reference Johnson, Mercado and Acevedo2012) with one exception. Because many of the participants in Johnson et al. (Reference Johnson, Mercado and Acevedo2012) were not able to complete the combination of organization and goal setting pre-task planning in the time allotted, the condition was eliminated from the current study.
Essays were collected in the fourth week of instruction in the semester. The writing prompts used in the current study were identical to those used in the original study (see Appendix) and were counterbalanced in order to control for the potential impact of topic knowledge on the lexical variables examined in the study. Participants in two of the four classes wrote in response to prompt A, and participants in the remaining two classes wrote in response to prompt B.
Prior to composing, participants in each of the four classes were randomly assigned to one of four pre-task planning conditions as described in Table 4. As in the original study, the participants in each condition were given ten minutes to read the writing prompt and then complete their assigned pre-task planning activity. After ten minutes, the participants were asked to stop working on their pre-task planning activities and asked to begin composing their essays. They were then given 30 minutes to compose their essays. After 30 minutes, the participants were asked to stop working on their essays and to turn in their pre-task planning activities and essays.
A brief description of the materials and treatment conditions in the current study
The procedures were incorporated into regular classroom instruction and were administered by the second author—the participants' regular classroom teacher—following the first author's explicit written instructions. The second author's answers to a fidelity questionnaire indicated that (a) the procedures were followed exactly as they were described in the instructions and (b) the time limits for the procedures were followed exactly.
5.1.3 Data preparation and coding
Each of the handwritten essays was word processed and saved in two digital formats for further analysis. In an effort to provide results directly comparable to those of Johnson et al. (Reference Johnson, Mercado and Acevedo2012), each of the texts was analyzed for the same measures of fluency, grammatical complexity, and lexical complexity used in that study (see Table 2).
5.1.4 Statistical analyses
Data were grouped and screened for normality of distribution using a two-stage process. Data were first tested using Shapiro-Wilk tests of normality (α = 0.05). If the result of the test was significant, z-scores of skewness and kurtosis were calculated to determine whether the distribution of the variable was within acceptable limits (≤ |3.29|). This analysis found that the distribution of four variables was not within acceptable limits: (a) average sentence length, (b) mean words preceding the main verb of clauses, (c) normed frequency of 4K word types, and (d) normed frequency of 5K word types. The distribution of three of these variables was improved through square root transformation.Footnote 8 However, the distribution of the variable 5K word types could not be improved through transformation.
Two statistical tests were conducted to evaluate the effect of pre-task planning condition on the fluency, grammatical complexity, and lexical complexity of the participants' written language production: (a) A Kruskal-Wallis test (α = 0.05) was conducted to compare the dependent variable normed frequency of 5K word types among the four pre-task planning conditions; and (b) A one-way multivariate analysis of variance (α = 0.05) was conducted to compare the remaining dependent variables among the four pre-task planning conditions.
Prior to conducting the one-way multivariate analysis of variance, the data were further screened for multivariate outliers and multicollinearity through multiple regression analysis. Mahalanobis distance values indicated that two of the participants were multivariate outliers. Data from these two participants were excluded from further analyses. Assessment of collinearity diagnostics indicated that tolerance values and variance inflation factors were well within acceptable limits. All variables were retained for further analysis.
5.2 Results
The results of both the Kruskal-Wallis test [χ2(3, 88) = 7.44, p = 0.06] and the multivariate analysis of variance [F(21, 240) = 1.51, p = 0.07] indicated no significant difference in any of the variables examined in this study. However, examination of means and standard deviations suggest a trend noted in the original study (Johnson et al. Reference Johnson, Mercado and Acevedo2012). As can be seen in Table 5, participants in the organization pre-task planning condition produced shorter sentences than participants in the control condition. Participants in the organization pre-task planning condition also produced more total words than participants in the control condition. This result is consistent with Johnson et al. (Reference Johnson, Mercado and Acevedo2012), who found organization pre-task planning to be associated with significant reductions in sentence length and non-significant increases in total words produced. However, because each of the statistical tests in the current study delivered non-significant results, the authors wish to err on the side of caution and conclude no effect of pre-task planning condition on the grammatical complexity, lexical complexity, and fluency of written language production of L1 writers.
Means (standard deviations) of variables examined in the current study
6. Discussion
Through an approximate replication of a previous study examining the impact of pre-task planning sub-processes on the written language production of L2 writers, this study sought to extend research on pre-task planning and written language production by determining whether a threshold of general proficiency in the target language is necessary in order for pre-task planning to have any measurable effect on the fluency and/or complexity of written language production (Johnson et al. Reference Johnson, Mercado and Acevedo2012). By examining the effect of pre-task planning sub-processes on the written language production of L1 writers of English, who had presumably achieved the hypothesized threshold of proficiency in English, this study used measures of fluency, grammatical complexity, and lexical complexity identical to those in the original study (Johnson et al. Reference Johnson, Mercado and Acevedo2012).
Unlike the study conducted by Johnson et al. (Reference Johnson, Mercado and Acevedo2012), which found no impact of pre-task planning on grammatical and lexical complexity and a minimal impact of organization pre-task planning on L2 writing fluency (d = 0.26), this study found no impact of pre-task planning on the participants' written language production, suggesting no support for the hypothesized threshold of target language proficiency. This result leaves L1 and L2 writing researchers with a number of questions about the impact of pre-task planning on written language production, the first of which is whether pre-task planning has any impact at all on written language production.
A comparison of the findings of the current study to those of Johnson et al. (Reference Johnson, Mercado and Acevedo2012) suggests that pre-task planning has no significant impact on L1 writers’ translation processes and little impact on L2 writers’ translation processes. A comparison of the translation process of composing in the L1 and L2 (Roca de Larios et al. Reference Roca de Larios, Marín and Murphy2001, Reference Roca de Larios, Manchόn and Murphy2006) may explain the differential effect of pre-task planning on writing fluency.
As noted previously, through a comparison of the composing processes of participants writing in their L1 and in an L2, Roca de Larios et al. (Reference Roca de Larios, Marín and Murphy2001, Reference Roca de Larios, Manchόn and Murphy2006) found qualitative differences in the translation process of composing, depending on whether participants were composing in their L1 or in an L2. According to Roca de Larios et al. (Reference Roca de Larios, Marín and Murphy2001), when composing in the L1, participants engaged in fluent translation at a rate two and half times higher than when they composed in the L2. Further, L2 writers in Roca de Larios et al. (Reference Roca de Larios, Manchόn and Murphy2006) engaged in compensatory problem solving—using existing lexical and grammatical resources to circumvent difficulty in expressing a proposition—at a higher rate than when composing in the L1. These qualitative differences in the translation process of L1 and L2 writing may explain the difference in results between the current replication study among L1 writers and the original study (Johnson et al. Reference Johnson, Mercado and Acevedo2012) among L2 writers.
Among L1 writers, the results of the current study suggest that pre-task planning has no impact on written language production. In contrast, the results of the original study (Johnson et al. Reference Johnson, Mercado and Acevedo2012) suggest that organization pre-task planning may facilitate fluent translation and/or compensatory problem-solving translation (Roca de Larios et al. Reference Roca de Larios, Marín and Murphy2001, Reference Roca de Larios, Manchόn and Murphy2006), resulting in increases—however small—in L2 writing fluency. Such a difference suggests slight revisions to Kellogg's model of working memory in L1 writing (Reference Kellogg, Levy and Ransdell1996) if it is to more accurately describe the demands of writing in an L2 on the working memory resources of the L2 writer. Further, the demands placed on specific components of working memory by the retrieval of lexical and grammatical forms may be further described in future L2 writing process research. However, further research is needed to determine how pre-task planning may affect the translation processes of L1 and L2 writers and how pre-task planning may affect online planning among L1 and L2 writers.
Comparisons to previous L2 writing research on pre-task planning and its impact on written language production indicate a number of intervening and/or mediating variables. Contrary to the findings of previous L1 writing research (Kellogg Reference Kellogg1987a, Reference Kellogg1988, Reference Kellogg1990; Sexton et al. Reference Sexton, Harris and Graham1998; Page-Voth & Graham Reference Page-Voth and Graham1999; De La Paz & Graham Reference De La Paz and Graham2002; Troia & Graham Reference Troia and Graham2002; Graham & Perin Reference Graham and Perin2007), the current study found no effect of pre-task planning on L1 writers’ fluency. This difference in findings highlights a likely explanation for the results of previous L1 writing studies examining pre-task planning.
a. In the case of Kellogg's research among adults (Reference Kellogg1987a, Reference Kellogg1987b, Reference Kellogg1988, Reference Kellogg1990), the participants were very likely experienced in composing the kinds of writing tasks which were elicited in the studies.
b. In the case of L1 writing research among children (Sexton et al. Reference Sexton, Harris and Graham1998; Page-Voth & Graham Reference Page-Voth and Graham1999; De La Paz & Graham Reference De La Paz and Graham2002; Troia & Graham Reference Troia and Graham2002; Graham & Perin Reference Graham and Perin2007), the effects of pre-task planning could very well have been the result of instructional interventions in the use of multiple writing strategies in addition to pre-task planning.
In the case of previous L2 writing research on the effects of pre-task planning on written language production, it seems likely that participants in each study had varying levels of experience with the writing tasks and genres used to elicit writing samples from the participants. For example, the writing task used in Ellis & Yuan (Reference Ellis and Yuan2004) was a narrative writing task, a task with which most L2 writers have a good deal of experience. Given that familiar genres place fewer demands on working memory capacity (Kellogg Reference Kellogg2001), it is possible that the increases in fluency and grammatical complexity noted among Ellis & Yuan's (Reference Ellis and Yuan2004) participants were attributable to an interaction of genre familiarity/experience and pre-task planning.
In contrast, the writing tasks in Ong & Zhang (Reference Ong and Zhang2010) and Johnson et al. (Reference Johnson, Mercado and Acevedo2012) were argumentative and may have made additional demands on the participants' working memory capacity. Thus, participants' writing fluency and lexical complexity were impeded in Ong & Zhang (Reference Ong and Zhang2010)Footnote 9 and writing fluency was minimally impacted in Johnson et al. (Reference Johnson, Mercado and Acevedo2012).
The importance of genre knowledge and its potential interaction with pre-task planning highlights the importance of input to L2 writing instruction—and to L1 writing instruction for that matter. Any measures that L1 and L2 writing instructors can implement to provide their students with rich exposure to the types of genres they expect their student to produce will go far toward improving their students’ ability to complete assigned writing tasks. Further, explicit instruction in the use of pre-task planning activities, particularly when the use of such activities is directly related to features of the targeted genre, may assist L1 and L2 writers in producing fluent, complex language when they compose in such genres.
7. Limitations
The results of the current study found no impact of pre-task planning on L1 writers’ language production, indicating no support for the threshold of general target language proficiency hypothesized in Johnson et al. (Reference Johnson, Mercado and Acevedo2012). However, when evaluating the impact of pre-task planning on writing in general and L2 writing in specific, readers should bear the following limitations in mind.
As was previously noted, a non-significant trend in the data appeared to mirror a trend noted in the original study (Johnson et al. Reference Johnson, Mercado and Acevedo2012). However, in order to avoid Type I error, the more conservative Pillai's trace was used to evaluate the multivariate effect of pre-task planning condition, further increasing the possibility of Type II error. Less conservative criteria further indicated no significant effect of pre-task planning on writing fluency, grammatical complexity, and lexical complexity, though it is possible that a larger sample using less conservative criteria may reveal a significant effect. Based on the results of the current study, however, the authors conclude no support for the hypothesized threshold of proficiency.
Previous instruction and the participants' experience with composing argumentative essays of the kind elicited in the current study may have impacted the participants' writing performance. As speculated above, and in Johnson et al. (Reference Johnson, Mercado and Acevedo2012), pre-task planning may interact with knowledge of, and experience with, the targeted writing genre. According to the participants' regular classroom teacher, the participants in the current study did not receive instruction in argumentative writing in their ELA classes until after data collection for the current study. However, it is possible that some participants had received explicit instruction in argumentative writing instruction in their other classes or in previous years. Thus, it is not clear whether the results of the current study were influenced by participants' previous instruction (or lack thereof).
8. Conclusion and future directions
With the exception of the caveats noted above, the results of the current study provide no support for the threshold of target language proficiency hypothesized in Johnson et al. (Reference Johnson, Mercado and Acevedo2012). However, when compared to the results of Johnson et al. (Reference Johnson, Mercado and Acevedo2012), the results of the current study suggest that pre-task planning may affect the translation processes of L1 and L2 writers differently by facilitating fluent translation and compensatory problem-solving translation (Roca de Larios et al. Reference Roca de Larios, Marín and Murphy2001, Reference Roca de Larios, Manchόn and Murphy2006). In order to examine such a possibility, future studies on pre-task planning in L1 and L2 writing might combine manipulation of pre-task planning with examination of think- aloud protocols and retrospective interviews of L1 and L2 writers in order to determine how—or whether—pre-task planning sub-processes affect the translation process of composing. Such studies would examine the composing processes of L1 and L2 writers to determine whether pre-task planning affects L2 writers’ focus on fluent translation versus problem-solving translation (Roca de Larios et al. Reference Roca de Larios, Marín and Murphy2001), as well as their attention to compensatory or upgrading problem-solving translation (Roca de Larios et al. Reference Roca de Larios, Manchόn and Murphy2006). Such studies may lead to a modification of Kellogg's model of working memory in L1 writing to more accurately describe the L2 composing process, specifically the demands placed on working memory resources by accessing/retrieving lexical and grammatical forms from long-term memory.
Such a modification would describe the translation process to account for the translation sub-processes of fluent translation and compensatory problem-solving translation noted in Roca de Larios et al. (Reference Roca de Larios, Marín and Murphy2001, Reference Roca de Larios, Manchόn and Murphy2006). Further, Kellogg's model (Reference Kellogg, Levy and Ransdell1996) would require revision in order to account for the distinction between the translation sub-processes of compensatory problem-solving and upgrading problem-solving (Roca de Larios et al. Reference Roca de Larios, Manchόn and Murphy2006). The planning process—and its sub-processes (idea generation, organization, and goal setting)—might then interact differently with each translation sub-process by focusing the writer's attention on compensatory problem-solving translation, upgrading problem-solving translation, or fluent translation.
Future research might also examine the differential effects of pre-task planning on oral and written L2 language production in order to determine whether the predictions of the Cognition Hypothesis (Robinson Reference Robinson2001, Reference Robinson2005, Reference Robinson and Robinson2011a, Reference Robinson2011b) and the Limited Attentional Capacity Model (Skehan Reference Skehan1998; Skehan & Foster Reference Skehan, Foster and Robinson2001) may be applied differently to the two productive modes. In addition, future research might also investigate the interaction of genre knowledge/experience and pre-task planning in order to determine whether this is, in fact, a mediating variable. If genre knowledge/experience is found to be a mediating variable, future models of L2 composing processes would need to account for the interaction between working memory and long-term memory to account for the differential impact of genre knowledge/experience and pre-task planning sub-processes on the fluency, grammatical complexity, and lexical complexity of L2 writers’ language production.
Acknowledgements
The authors express their deepest gratitude to Ms. Cassandra Schug, Mr. Kent Jacobson, and Ms. Lindsay Bremberger for their cooperation in this study. The authors are also thankful for the kind support of Drs. Sidney McPhee and Brad Bartel of Middle Tennessee State University. Their cooperation and support was instrumental in realizing this project and is greatly appreciated. The authors also express their thanks and appreciation to Dr. Graeme Porte and the three anonymous Language Teaching reviewers. Their input has decidedly contributed to the quality of the manuscript you see here. Any shortcomings, however, remain the sole responsibility of the authors.
Mark Johnson is Assistant Professor of TESOL and Applied Linguistics in the Department of English at East Carolina University where he works with pre-service and in-service teachers of English as a second language. His research interests and publications address the role of planning and working memory in L2 writing as well as the role of productive vocabulary knowledge in L2 writing proficiency.
Christine Nicodemus is currently an instructor at East Carolina University, teaching composition courses in the Writing Foundations Program. She has also enjoyed instructing mainstream and ESL students in K-12 classrooms.
Mark D. Johnson & Christine L. Nicodemus
Department of English
East Carolina University
Mail Stop 555
Greenville, NC 27858
USA
Appendix
Prompts used in data collectionFootnote 10
Prompt A: Some people believe that the Earth is being harmed (damaged) by human activity. Others feel that human activity makes the Earth a better place to live. What is your opinion? Use specific reasons and examples to support your answer.
Prompt B: Some people believe that high school and university students should be required to attend classes. Others believe that going to classes should be optional for students. Which point of view do you agree with? Use specific reasons and details to explain your answer.
