In second language (L2) writing classes, teachers generally give corrective feedback (CF) on their learners’ writing, particularly on errors in grammar and lexis. The underlying assumption for giving feedback is that it will help learners to notice their errors and, subsequently, to produce the correct forms. However, although some recent studies (e.g., Bitchener, Reference Bitchener2008; Bitchener & Knoch, Reference Bitchener and Knoch2008; Sheen, Reference Sheen2007) show that targeted CF can be effective, extensive reviews of the available empirical research (e.g., F. Hyland & K. Hyland, 2006; K. Hyland & F. Hyland, 2006; Goldstein, Reference Goldstein2004, Reference Goldstein2005) conclude that the findings about the efficacy of CF are mixed and thus inconclusive (see also the debate between Ferris, Reference Ferris1999, and Truscott, Reference Truscott2007). A number of factors may explain the lack of definitive findings about the efficacy of CF, including the research methods employed in studies on CF, and a host of contextual and affective factors that relate to both teachers and learners (see Goldstein, Reference Goldstein2005).
A review of the literature on CF exemplifies these mixed findings and briefly discusses the different methodological approaches that have been used to assess the impact of CF, before focusing on a small number of studies that have investigated an aspect of CF that has not been extensively explored: how learners process the feedback they receive. Adopting a sociocultural theoretical perspective, it is argued that research that analyzes actual instances of learners engaging with feedback and revising their texts as well as research that looks more closely at how learners’ beliefs and goals impact their decisions is needed to understand how and why learners respond to different forms of CF.
Corrective feedback can be distinguished in terms of its directness (for a comprehensive description, see Guénette, Reference Guénette2007), which ranges from direct (e.g., writing the correct form above the incorrect form) to indirect (e.g., using editing symbols to signal an error). Research on which form of feedback is most effective has produced mixed results. For example, Lalande (Reference Lalande1982) found that students who received indirect feedback (editing codes) showed greater accuracy on subsequent writing than students who received direct feedback—but the differences were not statistically significant. Chandler (Reference Chandler2003), in contrast, found direct feedback to be more effective than three different types of indirect feedback (with and without codes that explain the type of error). However, these effects were found only on immediate revisions; texts written later showed no statistically significant differences in grammatical accuracy in relation to type of feedback.
One factor that may explain these mixed results is whether the impact of CF on a revised text or on a subsequently written new text is considered; another is the measure used to assess the impact of the feedback. For example, Chandler (Reference Chandler2003) used mean accuracy scores (mean number of errors per 100 words produced) on both revised and new texts but did not take into account the different types of errors or their relative frequency. Bitchener (Reference Bitchener2008) used accuracy scores of new texts but only measured the accurate use of structures targeted by the feedback. Sachs and Polio (Reference Sachs and Polio2007) considered revised texts and used a scale of revisions of T-units, defined as an independent clause and any subordinate clause attached to it or embedded in it, noting what proportion of these units were amended in the direction of the CF and distinguishing among partial, full, no revision, or not applicable. The use of different measures makes comparison across studies difficult. However, it is not necessarily the case that using uniform measures will produce conclusive research findings.
The bulk of research on feedback has investigated the type of revisions that students make (or do not make) in response to different types of feedback (see Goldstein, Reference Goldstein2004, for a review) rather than how learners actually engage with and process the feedback, and why they use (or fail to use) the feedback received. Processing of feedback is perhaps less well researched and understood because it is difficult to access such learner-internal cognitive processes. Studies that have collected feedback processing data through think-aloud protocols (e.g., Qi & Lapkin, Reference Qi and Lapkin2001), retrospective interviews (e.g., Hyland, Reference Hyland1998), or pair discussions of the feedback received on a jointly produced text (e.g., Swain & Lapkin, Reference Swain and Lapkin2003) have suggested that two additional factors may affect the impact of feedback—namely, depth of processing and learners’ attitudes toward the feedback provided.
In a small study (n = 2), Qi and Lapkin (Reference Qi and Lapkin2001) used think-aloud protocols, during which learners were asked to verbalize their processing of the feedback received (reformulation), to investigate the effect of depth of processing on uptake of feedback. The researchers analyzed the protocols and distinguished two types of noticing: substantive and perfunctory. The main difference between these two types of noticing was that substantive noticing episodes were those in which the learners articulated reasons for the feedback received. This articulation was taken as evidence that the learners understood why the CF was provided. Qi and Lapkin found that substantive noticing led to greater improvements in the revised text than perfunctory noticing. Similarly, Sachs and Polio’s (Reference Sachs and Polio2007) larger study (n = 54), which also used think-aloud protocols, found that when learners noticed and understood why a linguistic item was reformulated (as evident in the think-aloud protocols), they were more likely to revise the item on subsequent drafts. However, the researchers point out the highly inferential nature of coding for depth of engagement.
Other studies (e.g., Hyland, Reference Hyland1998, Reference Hyland2003; Swain, Reference Swain and Byrnes2006; Swain & Lapkin, Reference Swain and Lapkin2003) have shown that learners’ goals, attitudes, and beliefs may also affect the uptake of feedback. Hyland (Reference Hyland1998, Reference Hyland2003) used retrospective interviews and case studies to investigate students’ use and reactions to the feedback received. Hyland found that whether learners respond to the feedback and the strategies they adopt may depend on the importance they attribute to the grammatical accuracy of their writing. Swain and Swain and Lapkin showed, using pair work, that learners may reject teacher feedback because it is perceived as violating their own beliefs about language conventions or as altering their intended meaning.
The importance attributed to learners’ beliefs in explaining how and why learners process feedback is in line with sociocultural theoretical perspectives on learning. Sociocultural theorists view learners (particularly adult learners) as intentional agents in their language learning activity who assign relevance and significance to certain events and whose behavior is guided by their own goals (Lantolf & Pavlenko, Reference Lantolf, Pavlenko and Breen2001; Lantolf & Thorne, Reference Lantolf and Thorne2006). These beliefs and goals may affect what learners notice, whether they accept or reject the feedback provided, and how much of the feedback they retain.
THE CURRENT STUDY
Aims
The few studies that have investigated the nature of learners’ engagement with the feedback provided on their writing have considered only direct forms of feedback (e.g., Qi & Lapkin, Reference Qi and Lapkin2001; Sachs & Polio, Reference Sachs and Polio2007; Swain & Lapkin, Reference Swain and Lapkin2003). The current study compared the processing, uptake, and retention data of learners who received direct and indirect forms of feedback. Furthermore, taking a sociocultural approach to data analysis (microgenesis), the present study investigated activity as it occurred and attempted to link outcomes (uptake and retention) with processes (nature of engagement) and the learners’ underlying beliefs and goals. This study set out to explore (a) whether learners process direct feedback differently from indirect feedback, (b) what factors affect learners’ uptake of feedback when revising their text, and (c) what factors affect learners’ retention of feedback when writing a new text.
Design
A case study approach was adopted, with the cases selected from a larger research project that investigated the efficacy of different forms of written CF using an experimental design. This larger project involved two groups of English-as-a-second-language (ESL) learners, each group composed of 12 pairs. One group received feedback in the form of reformulation and the other received feedback in the form of editing symbols. Assignment of pairs to the different feedback condition groups was done randomly.
Participants
All participants in the project were volunteers recruited from advertisements displayed on notice boards in a large Australian research university. The advertisement stated that the aim of the project was to investigate the efficacy of different types of feedback on writing. Participants were invited to attend the project sessions in self-selected pairs. At the end of the study, each participant received $100 for their participation.
The participants (30 females, 18 males) were predominantly graduate students (n = 40) pursuing a master’s degree. Their average age was 25. Over half of the participants (n = 28) were enrolled in a commerce degree program (e.g., master of applied commerce, master of applied finance). The majority came from Asia, from countries such as China (n = 24), Indonesia (n = 11), and Thailand (n = 4), which reflects the typical native language background of international students at Australian universities. Participants were of advanced L2 proficiency and provided documentation that they had met the English language requirement for university entrance: an International English Language Testing System (IELTS) score of 6.5, with 6.0 in writing, or a TOEFL Internet-Based Test score above 240, with a Test of Written English score of 4.5 or above. The majority had learned English as a foreign language in their home countries, on average for 8 years at secondary school and a university; most participants (n = 45) had been in Australia for less than 1 year at the time of data collection. Participant pairs generally came from the same course of study, where they had met each other. The period of acquaintance with each other ranged from 1 to 8 months.
Implementation
Participants attended three separate sessions. In session 1, all pairs composed a data commentary text based on a graphic prompt (see Appendix A). They were given 30 min to complete the task and their pair talk was audio-recorded. In session 2 (day 5), the pairs were provided with feedback on their writing, either in the form of reformulations or editing symbols. The pairs were given 15 min to discuss the feedback (processing session), which was then removed by the researcher. The pairs were given the unmarked and original version of their text (written in session 1) and had 30 min to rewrite it (rewriting session). Pair talk in both the feedback processing and rewriting sessions was recorded to provide data of how learners process feedback. In session 3 (day 28), the learners individually composed a data commentary text using the same prompt as in session 1.
Direct feedback was provided in the form of reformulation and involved rewriting the learners’ text, attending to grammatical and lexical errors but preserving the original meaning as much as possible (Thornbury, Reference Thornbury1997). Editing, the indirect form of feedback, involved marking the text with codes that corresponded to certain types of errors and that were explained and exemplified in a handout (see Appendix B). Both forms of feedback were provided by the same native speaker who was an experienced ESL teacher. Feedback focused on errors in grammar (morphology and syntax), lexis (word choice), and mechanics (spelling and punctuation).
The following examples illustrate the two forms of feedback. In (1), the reformulated version contains five reformulations: insertion of a phrase (the graph), a change in phrase order (for all cities), correction of the verb form (occur), adjunction of the second sentence to the first, and deletion of a verb (being). The example in (2) contains two editing codes: parentheses around the entire sentence, which signals that a word or words need to be deleted, and the code (C) placed above a word, which indicates an error in word choice.
(1) Reformulations
a. Original
Analysing seasonally, the highest rainfalls constantly occurs in summer for all cities. And the lowest being in winter.
b. Reformulated version
Analysing the graph seasonally for all cities, the highest rainfalls constantly occur in summer and the lowest in winter.
(2)
Data
Three sources of data were used in this study: the feedback received (reformulations and editing codes on the day 1 texts); the texts written by the pairs on day 1, revised on day 5, and written individually on day 28; and the transcribed pair talk elicited on day 5 during the feedback processing and the rewriting sessions.
Data Analysis
For the first source of data, all editing codes and reformulations were counted. When counting the number of reformulations, each word or phrase that was reformulated, deleted, or reordered was counted as a single reformulation. Following researchers who work within a sociocultural theoretical framework (Aljaafreh & Lantolf, Reference Aljaafreh and Lantolf1994; Nassaji & Swain, Reference Nassaji and Swain2000), the data were analyzed microgenetically for evidence of uptake and retention across the three sessions. To trace for evidence of uptake of feedback, texts written on day 1 were compared with those rewritten on day 5 along with the feedback comments. Revisions were analyzed for whether they were correct or incorrect, whether they were made in response to the feedback given or unsolicited, and whether the changes were made at the word, clause or phrase, or sentence level. Revisions made in response to feedback were counted to provide a measure of uptake. The texts written on days 1, 5, and 28 were then compared to trace for retention of the feedback.
The example in (3) illustrates the procedure used to analyze evidence of uptake and retention of the feedback received. The data come from a pair (Bing and Lina) who received editing feedback and include relevant excerpts from the original and revised texts composed jointly as well as relevant excerpts from the students’ individually produced texts on day 28. In the investigation of evidence of uptake, revisions that were consistent with either the reformulation or the intent of the editing symbol were coded as correct and indicated by a double check (✓✓). The symbol (✓✗) indicated an incorrect change and (✗) no change. In (3), two editing symbols were provided in the feedback (to indicate an error in word choice and in spelling). The revised text contains two changes, both clearly in response to the feedback provided. The changes, deemed correct in this instance, were considered as evidence of uptake. However, it soon became apparent that employing the same approach to analyze the new texts produced individually in session 3 (day 28) was not always possible because these new texts were often very different from those produced and revised in sessions 1 and 2. As can be seen in (3), the text produced by Lina on day 28 is very different from that produced collaboratively on day 1 and revised on day 5. There is no relevant sentence to show evidence that the correction to the word countries was retained. Thus, the long-term impact of feedback (i.e., retention) was not quantified. Instead, a process-product analysis (Nassaji & Swain, Reference Nassaji and Swain2000) was used, which allowed for a closer focus on the data of four case study pairs.
(3)
The transcribed pair talk from the processing and the rewriting sessions were analyzed for language-related episodes (LREs; Swain & Lapkin, Reference Swain and Lapkin1998). LREs were defined as segments in the pair talk during which learners focused explicitly on language items. This included instances in which learners read the reformulated text aloud, deliberated over the reformulation, discussed how to revise in response to an editing symbol, or deliberated over language items that, although not targeted by the feedback, the learners felt needed amendment. Thus, all LREs were first identified, regardless of whether they dealt with language items targeted by the feedback. LREs varied in length: Some were composed of a short turn (e.g., a learner simply read aloud the reformulation received) and others of multiple turns (e.g., learners deliberated about word choice). All LREs were then further analyzed for focus to distinguish between episodes that dealt with form (i.e., morphosyntax; F-LRE), lexis (L-LRE), and mechanics (i.e., spelling and punctuation; M-LRE). It was also noted whether the LREs were resolved correctly (√), incorrectly (X), or left unresolved (?). Correct resolution in this study referred to resolutions that accorded with the feedback or to alternatives deemed as equally acceptable by the researchers. To link uptake and retention to how the feedback was processed, a process-product analysis (Nassaji & Swain, Reference Nassaji and Swain2000) was conducted. The LREs that dealt with language items that received feedback were identified and analyzed for the nature of engagement.
Based on the work of several researchers (Qi & Lapkin, Reference Qi and Lapkin2001; Storch, Reference Storch2008; Tocalli-Beller & Swain, Reference Tocalli-Beller and Swain2005), a distinction was made between LREs that showed extensive engagement and those that showed limited or no engagement. LREs that showed evidence of extensive engagement included episodes in which learners offered suggestions and countersuggestions, explanations, or any comments that showed evidence of meta-awareness of the feedback received (e.g., We don’t have to use being). LREs that showed evidence of limited engagement included episodes in which one member of the pair simply read the feedback and the other merely acknowledged or repeated it.
To code for LREs, one of the researchers, who has extensive experience in coding pair talk data for LREs, developed coding guidelines based on the data (see Appendix C). Using these guidelines, a second rater was trained and coded four transcripts independently. Reliability scores were calculated using simple percentage agreement. Interrater reliability scores for identifying LREs was 91%; disagreements seemed to be due mainly to oversights. Interrater reliability was the lowest (84%) when coding for the nature of engagement. Discussion between the two raters led to some modifications of the coding guidelines.
Figure 1 contains excerpts from the text (original and reformulated) and pair talk data to illustrate the coding procedure for LREs. The data come from a pair (Gus and Jon) who received reformulations. The reformulated version contains five reformulations (went, from, start, from, and to). The pair talk excerpt was coded as containing five LREs related to the feedback received: two F-LREs related to verb tenses and three L-LREs that dealt with the choice of prepositions. All were resolved correctly. The five LREs were coded as showing limited engagement; during the processing session, Gus read and acknowledged the reformulations, with no involvement from Jon. All five LREs were contained within a single turn.
Analyzing and coding pair talk for LREs.
Figure 2 provides an additional example that illustrates coding for LREs from Diana and Monica, who received editing feedback. The editing symbol (underlining of the phrase) signals that there is something wrong with the expression. The pair talk contained a segment coded as a L-LRE and directly related to the feedback provided. The LRE was coded as correctly resolved because the alternative the learners settled on (winter rainfall) was considered acceptable in this instance. However, unlike the LREs given in Figure 1, this LRE shows evidence of extensive engagement. The editing symbol initially challenges Diana’s previously held belief that this phrase is acceptable (Why I think rainfall of winter is correct?), but she then realizes that it should be reduced to a nominal phrase, and Monica agrees. Diana notes that this is something she has not previously learned (This is the first time I know this). Monica then suggests that the original expression is not necessarily wrong but that the alternative is a simpler or more direct expression.
Analysis of pair talk for LREs.
Finally, in analyzing the pair talk, any other salient features were also noted, particularly comments that reflected the learners’ attitudes toward the feedback provided and their beliefs about conventions of language use. Thus, in the example in Figure 2, it was noted that the feedback posed a challenge to Diana’s previously held knowledge but also provided a learning occasion and that Monica seemed to think that the revised phrase was not more grammatical but more direct.
FINDINGS
The quantitative findings from the larger research project (Storch & Wigglesworth, 2006) will be presented first: a comparison of the amount of feedback provided via editing and reformulations, the amount of uptake, and the number and nature of LREs. Case studies will then be used to report on the qualitative analysis, which attempted to link evidence of uptake (and retention) with the nature of the learners’ engagement with the feedback as well as their beliefs and goals.
Table 1 presents the amount of feedback provided via reformulations and editing as well as the uptake of this feedback as evident in the revised texts. The number of reformulations was almost double that of the editing symbols, but there was considerable variation in the number of feedback comments received by each pair, as evident by the large range. Table 1 also shows that there was more uptake following editing than following reformulations. However, it should be noted that analysis (and quantification) of uptake in revised texts following reformulations proved quite difficult. Revisions in response to reformulations were often made at the phrase and sentence level; thus, the revised texts contained new and completely rewritten sentences. In contrast, texts revised following editing feedback contained changes mainly at the word level, which were more easily traceable to the editing feedback received.
Feedback and uptake for the entire dataset
Table 2 summarizes the LREs for the entire dataset. Table 2 shows that despite the larger number of reformulations in relation to the editing symbols (as shown in Table 1), reformulations elicited fewer LREs than editing feedback during the processing session (day 5, session 1). In the rewriting sessions, the number of LREs was similar, regardless of feedback type. Most of the LREs in both sessions and in response to both types of feedback focused on grammar and lexis rather than on mechanics, and most were correctly resolved.
LREs during processing and rewriting sessions
In both the processing and rewriting sessions, the percentage of LREs directly related to the feedback was lower in the data of learners who received reformulations compared with those who received editing feedback. It should be noted that in the rewriting session, many of the LREs directly related to the feedback dealt with the same language items as those discussed during the processing session. There was more extensive engagement with feedback in response to editing than in response to reformulations, in both the processing and rewriting sessions.
The case studies illustrate how a number of factors, both linguistic and affective, impacted uptake and retention of feedback. The case study participants were fairly representative of the entire participant cohort in terms of language background, L2 proficiency, degree program, and length of acquaintance. Pairs who received a large amount of feedback but who showed contrastive patterns of engagement with the feedback were selected. For two of the pairs, the feedback elicited a large number of LREs, whereas for the two other pairs, it elicited few LREs.
FIRST CASE-STUDY PAIR
Background
The first pair consisted of one male (Eko) and one female (Sherry) graduate student, both Indonesian, with IELTS scores of 7.5 upon entry to the university. Both had studied English for several years at the high school and university level in Indonesia. They had known each other for 1 month. Eko was a graduate student in commerce and Sherry was a graduate student in cultural studies.
Analysis of Written Texts
The pair received a large number of reformulations (n = 40) on the text they wrote in the first session. However, a closer analysis of their errors and reformulations revealed that almost half of the errors (17/40) were errors in mechanics (spelling, capitalization). Word choice and expression errors made up the next largest category (n = 6). Errors in grammar varied and included errors in morphology (e.g., tense, agreement) and syntax (e.g., incomplete sentences, word order).
The revised text (day 5) was very similar to the original text, with revisions made mainly at the word level. Analysis for evidence of uptake showed that of the 40 reformulations, 30 were taken up. There were far fewer errors in mechanics in the revised text (n = 6) and in the texts produced individually on day 28 (four errors in Eko’s text and none in Sherry’s text). There was also a decrease in word choice errors in the revised text on day 5 (only one error). However, the texts produced individually on day 28 showed a number of errors in word choice or expression (three for Eko and five for Sherry), and some of these errors were the same as those found in the text produced jointly on day 1.
Analysis of Pair Talk
Eko and Sherry’s pair talk showed that despite the large number of reformulations (n = 40), only 14 LREs were generated by this pair in the processing session and 22 in the rewriting session. Although most were related to the feedback, these LREs focused mainly on lexical choices. Very few LREs (only two in the processing and one in the rewriting session) dealt with errors in mechanics. Furthermore, whereas L-LREs showed an extensive level of engagement, M-LREs showed a limited level of engagement. Thus, it seemed that although the pair did not deliberate over errors in mechanics, there was a high level of uptake and retention of the reformulated feedback on these errors. However, a high level of engagement with feedback on errors in lexis that led to uptake did not always result in retention. The examples given in (4)–(8), taken from the data of this pair, suggest possible explanations for these findings.
The example in (4) contains relevant excerpts that discuss the choice of where and when, an error that occurred on two occasions in the original text produced by this pair on day 1.
(4) Excerpts from texts and pair talk dealing with where versus when
a. Original version
Constantly for every season, Lagos has the highest rainfall except for winter where the highest rainfall being in Bucharest.
b. Reformulated version
For every season, Lagos constantly has the highest rainfall except for in winter when the highest rainfall is in Bucharest.
c. Relevant LREs
Eko: Mmm. Say it’s when instead of where. Which makes sense because we’re comparing time.
[…]
Sherry: except for winter when
Eko: when, yes. Remember that, when!
Sherry: when the highest rainfall
d. Revised version
For every season, lagos constantly has the highest rainfall, except for in winter when the highest rainfall is in Bucharest.
e. Eko’s new text
the highest rainfall in all seasons occurs in Lagos, except in winter when the highest rainfall occurs in Bucharest.
f. Sherry’s new text
Lagos constantly has the highest rainfall across all seasons except in winter when the highest is Bucharest.
As illustrated in (4), during the feedback processing session on day 5, the learners gained an understanding of the difference in the use of the two adverbs where and when. The texts produced on days 5 and 28 by both learners showed evidence of uptake and retention of the where-when distinction. These adverbs were used correctly in the revised text and in the individually produced texts (Eko: all four instances; Sherry: all three instances).
The example in (4) and the outcome in terms of uptake and retention contrasts with how the learners dealt with the use of the word fluctuative, as in example (5). Here, there is limited engagement with the feedback provided. Sherry merely reads the reformulated version and Eko does not respond, which may explain why no uptake or retention took place and the error persisted.
(5) Excerpts from texts and pair talk dealing with fluctuative
(a) Original version
whereas the more fluctuative level of rainfall
(b) Reformulation
whereas the more extreme levels of rainfall
(c) Relevant LRE
Sherry: whereas the most extreme, oh, extreme… most extreme level of rainfall
Eko: analysing the…
(d) Eko’s new text
although more fluctuative that Bucharest
(e) Sherry’s new text
Lagos, in the other hand, has the most fluctuative rainfall
Example (6) shows how the pair dealt with a stylistic choice of expression. Merely around was reformulated as around a mere.
(6) Excerpts from texts and pair talk dealing with merely around
(a) Original version
it fluctuates merely around 50mm
(b) Reformulation
it fluctuates around a mere 50mm
(c) Relevant LREs
Sherry: Don’t use merely. Oh, around a mere, oh, around a mere
Eko: A more sophisticated way of getting it.
[…]
Sherry: where it fluctuates… around… around
Eko: a mere
Sherry: Is that the how to put it? around a mere fifty millimetres?
Eko: That’s how she put it.
Sherry: I don’t know how to put merely there. Around a mere fifty millimetres
Eko: I think merely around fifty millimetres would be, you know, it equate.
Sherry: Oh, OK
(d) Revised version
it fluctuates around a mere 50mm
(e) Eko’s new text
It fluctuates merely around 50 mm
(f) Sherry’s new text
it only fluctuates a little around 50 mm
As can be seen in (6), in the processing session, there is extensive engagement with this reformulation. Sherry notes the reformulated phrase, and Eko comments that this is a more sophisticated way of expressing their idea. However, during the rewriting session, although Sherry recalls the reformulation after some hesitation and assistance from Eko, she expresses some doubts about her ability to use the expression. Eko suggests that both expressions are equivalent in meaning. Thus, although the revised version shows evidence of uptake, it is not retained at day 28. This may be because the learners do not feel comfortable using this expression or because they believe that the two expressions are equivalent in meaning.
The strategy the learners adopted was to memorize the reformulations, as shown in (7), and the goal driving their revisions was the desire to improve the accuracy of their text, as can be seen in (8).
(7) Excerpt from the pair talk in the processing session
Sherry: Maybe we ought to memorise, ah.
Eko: This
Sherry: So I memorise the first paragraph, you the second
(8) Excerpt from the pair talk in the rewriting session
Sherry: We’re not supposed to write anything totally different, right?
Eko: I’m not sure. I think, I think… we should, you know, make it better. You know, to improve it.
Summary
Eko and Sherry were very proficient (IELTS 7.5), and most of their errors were fairly superficial (i.e., mechanical) and easily corrected. Therefore, although the transcripts had only limited evidence of processing (i.e., few LREs), the learners clearly noticed the reformulations and were able to address their errors in their subsequent writing.
Other errors, particularly errors in word choice, required more overt attention. Here there was evidence of extended engagement and understanding (e.g., the when-where distinction), which led to uptake and retention. However, where there was lack of engagement (e.g., as with the word fluctuative) or resistance to the reformulation, there was no long-term retention, despite evidence of uptake (e.g., as with the expression around a mere) at day 5.
SECOND CASE-STUDY PAIR
Background
This pair was composed of two male Indonesian students, Gus and Jon. Both were graduate students in engineering and had an IELTS score of 6.5 upon entry to the university. Both had studied English for several years at the high school and university level in Indonesia. They had known each other for 6 months.
Analysis of Written Texts
As in the case of the first pair, the text produced by this pair showed a large number of errors and, consequently, a large number of reformulations (n = 43). Most of the errors were in verb use (n = 14), prepositions (n = 6), and sentence structure (n = 6). However, unlike the first pair, analysis of uptake revealed that of the 43 reformulations made to the original version, there were only eight instances of uptake in the revised text. The revised text (day 5) and the subsequent individual texts were very different from the text produced on day 1. The revisions made on day 5 were not always in response to the feedback provided and were at the sentence level, with deletions and additions of full sentences. Texts produced on day 28 bore little resemblance to the text produced jointly and contained new types of errors. This meant that there was no way to trace for evidence of retention.
Analysis of Pair Talk
Despite the large number of reformulations, there were few LREs (n = 13) found in the data of this pair during the processing session. Most dealt with verb tense choice and were thus related to the reformulations provided, but the level of engagement was limited. Most of the LREs consisted of single turns during which one learner read the reformulated text but made no comment. In the rewriting session, despite a large number of LREs (n = 40), only four related to the feedback provided.
The example in (9) illustrates the limited engagement with the reformulations and the impact of this limited engagement on the revised and new texts. The sentence produced in the original version had a number of errors and, consequently, a large number of reformulations (n = 6). In the pair talk, the learners focused only on the use of the phrase followed by instead of compared with and the verb tense. Engagement with these two reformulations was limited. Gus simply read the reformulations; Jon did not contribute at all. In the revised version, the sentence shows substantial revision; however, these revisions were not consistent with the suggested reformulations. The corresponding sentences in the texts produced on day 28 are very different from that of the original version. The sentence produced by Gus has a number of agreement errors; Jon’s sentence has errors in coherence.
(9) Excerpts from texts and pair talk dealing with a number of reformulations
(a) Original version
In detail, Bucharest and Lagos have the same pattern, that the rainfall in Spring is the second highest compared to autumn and winter, while for Beijing and Mexico City…
(b) Reformulated version
Specifically, Bucharest and Lagos had the same pattern: the rainfall in spring was the second highest followed by autumn and winter…
(c) Relevant LRE
Gus: okay, followed… oh wait was the second highest…however Beijing and Mexico City, autumn is the second… okay was the second highest followed by spring and winter so it’s the same.
(d) Revised version
Bucharest and Lagos falls into the first category. And Beijing and Mexico City are categorized in the second pattern.
(e) Gus’s new text
Bucharest and Lagos falls into the first category, for their rainfall patterns reaches the peak in summer, then the rainfall patterns decline...
(f) Jon’s new text
Bucharest and Lagos had the same order for the second and third highest rainfall, which were spring and autumn. While in Beijing and Mexico City,…
This pair’s lack of engagement with the feedback may be attributed to their attitude to the form of feedback (reformulations) and the aspects of language with which the feedback dealt. As shown in (10), Gus and Jon did not approve of this form of feedback.
(10) Excerpts from the processing session
Gus: huh? I don’t think this kind of feedback is good, because…
Jon: Yeah
Gus: people will tend to memorise this
Jon: yeah this still crap
[…]
Jon: yeah this not in good way to give a feedback
Gus: yeah a feedback should not just give away the answer. Yeah that’s…that’s my opinion. Okay so, are we supposed to memorise this?
Jon: yeah, you got paragraph one and two, I got paragraph three and four
Gus: okay, okay now you…you memorise paragraph three then four
Despite their disapproval, they elected to memorize the reformulated text, dividing the task between them. However, once the feedback was removed, they reconsidered their goals and felt that they should perhaps rewrite the text to improve it in any way that they saw fit.
(11) Excerpts from pair talk during the rewriting session
Jon: is it necessary that we have to write it in this style or…?
Gus: no, you change it in any way you want to
Jon: okay
Gus: any way that will make it better
[…]
Jon: We make our own improvements.
Summary
This pair, like the first, had a large number of errors and received a large number of reformulations (n = 43). However, unlike the first pair, this pair’s revised text showed little evidence of uptake. These learners showed limited engagement with the feedback. There were no instances of learning evident in the data, in contrast with the data of the first pair. This lack of engagement with the feedback could be attributed to the learners’ attitudes—their lack of approval of this type of feedback. Although the initial goal was to memorize the reformulated text, during the rewriting session, these learners’ goals changed. They decided to rewrite the text in the way they felt improved it and thus made substantial revisions to the text. Two pairs who received feedback in the form of editing will now be considered.
THIRD CASE-STUDY PAIR
Background
The third pair was composed of two female graduate students from China. Monica had an IELTS score of 6.5 and had studied English only at the university level (for 3 years prior to coming to Australia). Diana had studied English at the high school and university level and had a higher IELTS score of 7.0. Both were pursuing a master’s degree in human resource management (commerce) and had known each other for 7 months.
Analysis of Written Texts
The pair’s first version of the text elicited 17 editing symbols, mainly in the use of prepositions in phrases of time or location (n = 5), articles (n = 3), and word choice (n = 3). The revised text showed a high level of uptake (14/17), with errors in use of prepositions almost disappearing (one remaining error). There were also few errors in use of prepositions in the learners’ texts produced on day 28 (only one such error in Diana’s text and two in Monica’s). In contrast, errors in use of articles and in some word choices persisted, both in the revised text and in the new texts.
Analysis of Pair Talk
The feedback elicited 18 LREs in the processing session, of which 14 dealt directly with the feedback. In the rewriting sessions, of the 19 LREs, 11 dealt with aspects of language that received editing feedback. The majority of the LREs in the processing (14/18) and rewriting (12/19) sessions dealt with lexical choices—namely, the choice of prepositions. Engagement with this feedback was extensive. In contrast, few LREs dealt with feedback on articles (n = 2), and engagement was also limited.
The example in (12) shows the different levels of engagement with different types of errors and illustrates that engagement with the feedback on this preposition error led to an enhanced understanding about when to use in rather than of in temporal expressions. This, in turn, led to uptake and retention. Example (12) also shows that the feedback on the error in spelling was noticed in the processing stage but was not dealt with extensively. As in the case of the first pair, this limited engagement resulted in uptake and retention. However, in the case of articles, no attention was paid to the feedback provided in either the processing or rewriting session. This lack of attention may explain the persistence in errors in articles.
(12)
Example (13) illustrates the learners’ engagement with feedback on lexis. This example shows how the learners’ beliefs about the use of language—in this instance, the need to use linking phrases—based on previous language learning experience (IELTS training courses), resulted in resistance to the feedback, which may help explain instances of no uptake or no retention.
The phrase as can be seen that was underlined and put in parentheses to indicate that there was an error and that some words in this phrase should be deleted. However, the learners mistook the symbols to mean that the entire linking phrase was unnecessary and proceeded to delete this phrase in their revision. Similarly to the first pair with around a mere, these learners show uptake, but their resistance to this feedback based on their prior learning experience means that the linking phrase is used on day 28.Footnote 1
(13) Excerpts from texts and pair talk dealing with the use of linking phrases
(a) Original version
As we can see that the rainfall of most seasons in Lagos is the highest among the four cities.
(b) Editing feedback
(As we can see that)the rainfall of most seasons in Lagos is the highest among the four cities.
(c) Relevant LRE
Diana: I just don’t understand why we don’t need the as we can see that. I think this is…this is nice
Monica: mm hmm
Diana: I don’t know why
Monica: should be deleted
Diana: but I think as we can see that…you see in the IELTS book, they said um it can be seen, blah, blah, blah so I think that this is nice, I don’t know why
(d) Revised version
The rainfall in Lagos in most seasons is the highest among the four cities.
(e) Diana’s new text
It can be seen that the rainfall in summer in Beijing…
(f) Monica’s new text
As we can see, Bucharest…
Summary
Monica and Diana attended to most of the feedback received, particularly the feedback on their most frequent errors (use of prepositions in locative and temporal expressions). The LREs showed that these learners gained an understanding of this use of prepositions, which assisted them in using these prepositions correctly in the revised version and in subsequently produced new texts. Less attention was paid to the editing feedback on the use of articles, which may explain—along with the fact that articles are a renowned area of difficulty for L2 learners—the lack of retention of feedback on articles beyond the revised version. These learners seemed to show a higher level of uptake and retention when the feedback was consistent with their beliefs about language use. When the feedback contradicted those beliefs established in previous language learning courses, there was no retention.
FOURTH CASE-STUDY PAIR
Background
Unlike the other three pairs, the learners in this pair were undergraduate students in commerce. Bing was a male from Malaysia and Lina was a female from Indonesia. Both had studied English in high school and both had high IELTS scores of 7.0. They had known each other for 6 weeks.
Analysis of Written Texts
The original version of their text had 15 editing symbols. The most common errors were in the use of prepositions (n = 4) and verbs (n = 3). The revised text showed complete uptake (100%), with revisions made mainly at the word level. All errors in prepositions were amended, but new errors in the use of verbs were introduced. On day 28, the texts produced individually had no errors in verb use; Bing had only one error and Lina had three errors in the use of prepositions, which suggests overall high levels of retention.
Analysis of Pair Talk
In processing the feedback, the learners paid attention to the editing feedback and all of the LREs in both sessions related directly to the feedback received. Most of the F-LREs focused on the use of verbs and most of the L-LREs on the choice of prepositions. The level of engagement with the feedback was extensive, which can perhaps account for the high uptake.
The example in (14) shows evidence of the high level of engagement with the feedback provided and evidence of the learners extending their knowledge to new contexts. The feedback provided suggested that there were errors in that string of words. Although not specified, some of these errors were related to an inconsistent use of verb tense. While discussing this sentence, the learners became aware of these inconsistencies. Lina suggested that they use either the present or past tense throughout, and Bing agreed. In their revised text and their new texts, the present tense was used throughout.
(14) Excerpts from texts and pair talk dealing with the use of verbs
(a) Original version
Beijing also had the average rainfall in summer, which is 150mm
(b) Editing feedback
(Beijing also had the average rainfall in summer, which is 150mm)
(c) Relevant LRE
Lina: Beijing had…
Bing: had or… had or has
Lina: Don’t I say the other tenses we use…
Bing: had
Lina: Mmm
Bing: which has
Lina: You don’t use present, you know. But she has not correct our tense. Or did she?
Bing: Good question.
Lina: Haven’t I… this one we use present, this is all present. If one use the past then use past all.
Bing: OK. Beijing has…
(d) Revised version
Beijing has the highest average rainfall in summer of 150mm
(e) Bing’s new text
Beijing has the lowest average rainfall of 5mm, followed by…
(f) Lina’s new text
The second highest total average rainfall occurs in Beijing.
The excerpt from the pair talk during the rewriting session given in (15) suggests that the learners’ goal was to focus on amending the errors rather than rewriting the text. The pair talk also showed evidence that the learners memorized the location of the editing symbols and relied on this in their rewriting activity.
(15) Excerpts from texts and pair talk
Lina: You see… the difference. So we write the same the same… just change the one that…
Lina: This one just is not right.
Bing: Mmm? What do you mean?
Lina: Have to correct this and just write.
[…]
Lina: Mmm. and winter it has… it. Remember there’s something missing.
Bing: Yeah
Summary
There was a high level of uptake and retention of the feedback provided to these learners, who engaged with the feedback extensively. Their goal was to amend the text at the word level in response to the feedback provided.
DISCUSSION
This study sought to examine and compare how learners process direct feedback (reformulations) versus indirect feedback (editing symbols) on language errors and what impact, if any, the type of feedback and processing has on uptake (immediate revision) and retention in the long term (23 days later), as evident in individually written texts. The findings for the whole cohort showed that editing feedback elicited more LREs than reformulations and that these LREs tended to relate directly to the feedback provided. The level of engagement also seemed more extensive with editing feedback than in response to reformulations. As suggested by Ferris (Reference Ferris2002), in response to editing, learners had to identify the nature of the error and attempt to supply the correct form, using their own knowledge of grammar and word meanings to offer suggestions and countersuggestions. In contrast, engagement with reformulations tended to be limited to reading the reformulation, acknowledging or merely expressing agreement, with fewer instances of extensive engagement. However, it is important to reiterate that coding for level of engagement is a highly inferential process (Sachs & Polio, Reference Sachs and Polio2007) and that the amount of verbalization evident in the LREs may not necessarily reflect depth of cognitive processing.
In line with the findings of other studies that elicited feedback processing data (e.g., Qi & Lapkin, Reference Qi and Lapkin2001; Sachs & Polio, Reference Sachs and Polio2007), this study also found that extensive engagement with the feedback led to high levels of uptake. This was evident in the findings for the whole cohort as well as in the case-study data. The third and fourth pairs, who received editing feedback, engaged with the feedback extensively and showed high levels of uptake. In the case of the third pair, for example, extensive engagement with feedback on certain prepositions led to uptake and correct use of these prepositions. In contrast, limited or no engagement with feedback on articles resulted in no uptake and persistent inaccuracies in the use of articles. Similarly, in the case of the first pair, who received reformulation feedback, extensive engagement over the where-when distinction led to uptake; limited engagement with the word choice fluctuative resulted in no uptake. However, data from the first pair showed that uptake also depends to some extent on the nature of the errors. For more superficial errors, such as errors in mechanics, perfunctory noticing, whether verbalized or not, may be sufficient for uptake to occur.
Similarly, retention seemed to relate to the level of engagement with the feedback and the nature of the errors. Feedback on errors in mechanics was retained despite limited or no overt engagement. In the case of morphosyntactic and lexical errors, high levels of engagement led to understanding and an ability to retain the feedback in the long term (e.g., the where-when distinction in the first pair or correct use of verbs in the fourth pair). However, other affective factors also seemed to influence retention. Specifically, learners’ beliefs, attitude toward the form of feedback received, and their goals seemed to have an effect on whether the feedback was retained. Thus, in the case of the first pair, there is elaborate engagement with the reformulated phrase merely around, which led to uptake but no retention. Lack of retention seemed to be attributable to learners’ beliefs about language use. The pair did not adopt the reformulated phrase because they felt that the alternative was not necessarily a better expression. Similarly, in the case of the third pair and the use of linking phrases (e.g., as we can see that), there was extensive engagement and uptake but no retention. The learners felt that the feedback contradicted their beliefs, shaped by their previous language learning experience about what constitutes a good writing style. Studies by Swain (Reference Swain and Byrnes2006) and Swain and Lapkin (Reference Swain and Lapkin2003) also found evidence of resistance to feedback that resulted in no uptake. The case study data discussed here suggest that resistance is more likely to lead to lack of retention.
Another important affective factor that had an impact on both uptake and retention was learners’ goals. The first and fourth pairs seemed to be driven by a goal to improve the accuracy of their text. This strategy may explain high uptake. In the case of the second pair, disapproval of reformulations as a form of feedback coupled with the goal of improving their text as they saw fit (see also Hyland, Reference Hyland1998) meant that these learners ignored the feedback received; hence, there was no uptake (or retention). When the learners seemed to approve of the type of feedback received—and were driven by a goal to improve their text—they sometimes adopted the strategy of memorizing the feedback (the first pair) or the location of the editing symbols (the fourth pair).
In research on feedback, affective factors such as learners’ orientation (Lantolf & Thorne, Reference Lantolf and Thorne2006), which includes their attitudes toward the type of feedback received and beliefs about language conventions shaped by previous language instruction as well as the goals and strategies adopted, are often ignored. The case study data showed that affective factors influence not only the type of strategies learners adopt in dealing with the feedback received (e.g., memorization) but also their willingness to accept the feedback and their likelihood of retaining it.
However, these findings should be interpreted cautiously. The participants in the current study were advanced language learners, and this may have affected not only the type of errors they made in their writing but also their ability to notice and attend to the feedback received, as well as their attitude to the different types of feedback. Furthermore, the two types of written CF compared (reformulations and editing symbols) were distinct from each other and, thus, inevitably elicited different kinds of responses. Perhaps the greatest limitation is that data were collected in an experimental rather than a classroom setting and thus, important contextual factors such as the relationship between the learners and the teacher who provided the feedback could not be investigated. Recent research on feedback (e.g., Given & Schallert, Reference Given and Schallert2008) shows that this relationship may play a powerful role in determining whether learners take up the feedback provided.
Nevertheless, the findings suggest that whether and which type of feedback is effective depend on a complex and dynamic interaction of linguistic and affective factors. Future research on feedback needs to combine an examination of the product (revised and new texts) and processes in an integrated manner. To isolate and investigate the effect of linguistic factors, studies in which feedback is given on specific structures are needed (e.g., Bitchener, Reference Bitchener2008; Sheen, Reference Sheen2007) to establish whether some form of feedback (direct vs. indirect) is more effective for particular types of errors. However, an investigation of linguistic factors alone is not enough. Researchers (e.g., Cumming, Busch, & Zhou, 2002; Hyland, Reference Hyland1998, Reference Hyland2003; Sachs & Polio, Reference Sachs and Polio2007) have called for classroom-based studies that more fully investigate affective factors (e.g., goals, orientation to task, preferences); however, such research is difficult to conduct. Collecting feedback processing data in a classroom setting may be difficult; given the detailed analysis such an investigation necessitates, studies on learners’ engagement with feedback have tended to be small-scale case studies (e.g., Given & Schallert, Reference Given and Schallert2008; Hyland; Qi & Lapkin, Reference Qi and Lapkin2001; Tardy, Reference Tardy, Hyland and Hyland2006). It is perhaps through case studies, such as the study reported here, that insights into this complex issue of the impact of CF can be gained, along with an understanding of the inevitability that experimental research on the impact of CF will continue to yield mixed findings.
APPENDIX A: GRAPH REPORT
The graph below shows average rainfall (by season) for four cities. Write a report for a university lecturer describing the information shown below.
You should write at least 150 words.
APPENDIX B: EDITING CODE
APPENDIX C: GUIDELINES FOR LRE ANALYSIS
A Language Related Episode (LRE) is any segment in the data where there is an explicit focus on language.
Note:
• This focus can be in response to the feedback the participants received but can also be unsolicited.
• LREs can vary in length. They can be short (e.g., consisting of a learner simply reading out aloud a reformulated word or phrase with no response from the other member of the pair) or a long segment (e.g., where both learners discuss grammatical or lexical choices).
• LREs can be interrupted. For example, learners may deliberate over the use of articles and decide to omit it. They may then return to this decision at a later stage in their pair talk and decide to reverse their decision, and insert the article. Since both segments deal with the same ‘error’ they are counted as one episode.
CODING LRES
1. Identify in the data segments where learners seem to be focusing explicitly on language choice.
2. Distinguish LREs in terms of focus: form-focus (F-LREs), lexis-focus (L-LREs), mechanics-focus (M-LREs).
• F-LRE: focus on morphology or syntax (e.g., verb tenses, word forms, use of articles, prepositions, word order)
• L-LRE: deliberations on word meaning, searching for a word, suggesting alternative words/phrase
• M-LRE: deliberations on issues such as spelling or punctuations (or pronunciation)
3. Determine whether the LRE deals with language items that were targeted by the feedback given.
4. Determine whether the LRE is resolved correctly (√), incorrectly (X), or left unresolved (?).
• Resolved correctly (√): The resolution reached is in line with the intended feedback (or it could be an acceptable alternative in this instance).
• Resolved incorrectly (X): The resolution reached is not in line with the intended feedback (or is an unacceptable alternative in this instance).
• Unresolved (?): The learners seem unable to determine how to respond to the feedback (in the case of editing) or seem reluctant to accept the reformulation but cannot agree on an alternative.
5. LREs that deal with language items targeted by the feedback are further analyzed for the nature of engagement.
• LREs which show extensive engagement (EE): episodes where learners offer suggestions and counter suggestions, explanations, or any comments showing evidence of meta-awareness of the feedback received (e.g., We don’t have to use being). It also includes episodes where the correction is repeated by learners a number of times.
• LREs which show limited or no engagement (LE): episodes where one member of the pair just reads the feedback and the other simply acknowledged or repeats it once, without making any other additional comments.
