2.1 Wood (Reference Wood2009)

Gatbonton & Segalowitz (Reference Gatbonton and Segalowitz1988, Reference Gatbonton and Segalowitz2005) have long argued for repeated use of formulaic expressions or utterances through communicative classroom activities (see Dao, Iwashita & Gatbonton Reference Dao, Iwashita and Gatbonton2016 for a recent example). Wood's (Reference Wood2009) fluency workshop is in keeping with this idea. The workshop consists of a sequence of language learning activities with a repeated focus on formulaic sequences and fluency: input-based formulaic sequences-focused instruction, awareness-raising through marking pauses in a transcript, imitation of models through shadowing, retrieval practice through text reconstruction activities such as dictogloss (Wajnryb Reference Wajnryb1990), interactive output practice through ‘mingle jigsaw’ (Wood Reference Wood1998) and chat circle, repetition of narratives under increasing time pressure (also known as the ‘4-3-2’ procedure) (Nation Reference Nation1989), and, finally, free speech. The workshop thus progresses through the stages of input – automatization – practice – production, starting from native speaker model narratives and formulas present in those model narratives.

The participant, an adult ESL learner, in Wood (Reference Wood2009) recorded a narrative monologue before and after the six-week workshop, and these recordings were then analysed by the researcher. The speech rate and mean length of runs were calculated and compared between the pre-test recording and post-test recording. The increase in these measures was evident with the participant able to produce speech faster and speaking for longer runs between pausing. Wood (Reference Wood2009) also found an increase in length and complexity of formulaic sequences used by the learner and associated this with her gains in fluency.

Wood (Reference Wood2009) proposes five criteria for evaluating whether a word string uttered by a learner should be considered a formulaic sequence, although not all five need to be met all the time. The general premise is that formulaic sequences are produced in a holistic manner (pp. 47–48). It is worth noting that in this study the criteria used for a word string to qualify as a formulaic sequence included hesitation-free production. This is in keeping with Wray's (Reference Wray2002) definition of formulaic sequence, according to which formulaic sequences appear to be retrieved from memory as prefabricated wholes (and one would therefore not expect hesitations within a formulaic sequence). It may nonetheless be argued that using a fluency criterion for the identification of formulaic sequences in speech introduces a degree of circularity if one wishes to determine the extent to which fluent speech relies on the use of formulaic sequences.

Wood's study has already been approximately replicated by Onoda (Reference Onoda and Muller2014), who ran a similar fluency workshop with a group of EFL learners in Japan and found a significant increase in speech rate. Unfortunately, Onoda (Reference Onoda and Muller2014) did not include a control or comparison group. Neither did the study examine the learners’ use of formulaic sequences and its association with attested fluency gains.

2.2 Boers et al. (Reference Boers, Eyckmans, Kappel, Stengers and Demecheleer2006)

Boers et al. (Reference Boers, Eyckmans, Kappel, Stengers and Demecheleer2006) was a study involving two groups of high-intermediate EFL learners in Belgium, who used the same reading and listening texts in the course of a school year. The difference in instructional procedures was the way these texts were exploited in class. One group was regularly asked to work in pairs to identify (and underline or highlight) useful phrases, an intervention – ‘text chunking’ – proposed by Lewis (Reference Lewis1997). The other group – the comparison group – was not explicitly directed to the phraseological dimension of the texts. As part of the end-of-course exam, the students were asked to interact with the examiner in an audio-recorded interview. In the first part of this interview they were asked to talk about the content of a short text they were given just before the start of the interview. The second part was of a more impromptu nature where students were asked to speak about a familiar topic such as their ‘travel experience’. The examiner and a second blind judge (who listened to the recording afterwards) were asked to grade the students’ oral proficiency ‘holistically’. The mean grades showed an advantage for the group that had engaged in text chunking. The second blind judge not only gave holistic grades, but also awarded grades separately for three aspects of performance: accuracy, range of expression, and fluency. This second assessment confirmed the first: the chunking group was perceived to be more proficient than the comparison group. The difference was judged to be the most pronounced was for the performance aspects of range of expression and fluency. The students’ recorded speech was also analysed for the incidence of conventional multiword expressions (or formulaic sequences), and the counts correlated positively with the oral proficiency grades given, including the grades for fluency.

At first blush, Boers et al. (Reference Boers, Eyckmans, Kappel, Stengers and Demecheleer2006) furnishes evidence that (a) fluency and use of formulaic sequences are associated, and (b) directing learners’ attention to the phraseological dimension of input texts helps them build a repertoire of formulaic sequences that benefits their spoken fluency. Some caveats need to be mentioned, though. With regard to the fluency–formulaic sequences association, one concern is the subjective or ‘impressionistic’ nature of the way both the fluency assessment and the formulaic sequence identification were conducted. As for the effect of the instructional intervention, another concern is that – as the authors acknowledge – the greater number of multiword expressions found in the utterances produced by the chunking group did not actually reflect greater uptake of these from the texts they had explored in class. Instead, these were expressions mined from the text given as prompt for the first part of the interview. Thus it seems the intervention equipped these students with a strategic advantage (which is undeniably a positive outcome), but it is far from clear to what extent the text chunking activities actually helped the students grow a resource of formulaic sequences for active use in real-time communication.

In fact, when the study was approximately replicated with a new cohort of students but without the use of a text prompt in the end-of-course interview, no compelling evidence emerged that the chunking group was either more fluent or used a greater number of formulaic sequences picked up from the course materials (Stengers et al. Reference Stengers, Boers, Housen, Eyckmans, De Knop, Boers and De Rycker2010). The positive association between formulaic sequences counts and (impressions of) fluency was confirmed, though (Stengers et al. Reference Stengers, Boers, Housen and Eyckmans2011). There are indeed grounds for believing that it will often take productive practice for receptive knowledge to transform into productive knowledge (Griffin & Harley Reference Griffin and Harley1996; Laufer Reference Laufer1998; Zhang Reference Zhang2017): in particular, productive knowledge that is readily deployable under conditions of real-time communication. That is also part of the rationale behind Gatbonton & Segalowitz’ (Reference Gatbonton and Segalowitz2005) and Wood's (Reference Wood2009) instructional approaches, which do provide learners with ample output practice to foster procedural knowledge of formulaic sequences, and which therefore appear to hold greater promise – with a view to oral fluency development, at least – than ‘mere’ text chunking practice.

3.1 Wood (Reference Wood2009)

Since Wood (Reference Wood2009) was a single case study, it would be worthwhile replicating the study again with another single learner who is involved in a fluency workshop that follows the pattern set out in Wood's Reference Wood2009 study, to see whether similar results could be obtained a second time. In the original study the participant was a Japanese learner of English studying abroad in an intensive ESL learning environment in Canada. The participant was in an optimal situation for increasing her fluency, by not only studying English intensively in an immersion university setting but also living in a homestay situation. Both study abroad and homestay living have been associated with increase in target language proficiency (Regan, Howard & Lemee Reference Regan, Howard and Lemee2009; Di Silvio, Diao & Donovan Reference Di Silvio, Diao and Donovan2016). It may, therefore, be worthwhile gauging the effects of the formulaic sequences-focused fluency workshop in non-immersion (e.g., EFL) settings, too. Since the learner in the original study was Japanese, a fluency workshop could be conducted in an EFL class in Japan and the fluency of a learner of similar English proficiency could be measured and compared with the original study. Doing so would remove the potential influence of study abroad and homestay and help isolate the learning effects of the fluency workshop.

While increasing participant numbers and introducing control participants (i.e., participants who receive no ‘treatment’ but only take the pre-test and post-test) would together extend the original study rather than replicate it, such an extension would provide more generalisable evidence of the effects of the fluency workshop. It would also help to determine whether fluency changes for fluency workshop participants are truly attributable to the independent variable under examination – the specific nature of the instructional treatment. Results from such a study could be compared with the original study, and then results between experimental and control participants could also be compared. It is debatable, however, whether such an extension from the original design could still be classed as a replication study, since the original study had no control participant data to make comparisons with.

The dependent measures in the Wood (Reference Wood2009) study could be re-considered as well. Wood (Reference Wood2009) used a narrative monologue at the beginning and a different topic narrative monologue at the end of the six-week workshop to measure his participant's fluency gains. It was noted in the study that the participant ‘took flight’ with the second narrative and this was likely because she was more ‘engaged’ with the topic (Wood Reference Wood2009). A replication could repeat the original topic at the end as well as including a new topic in order to remove variance from familiarity with one topic over another. The use of monologues is very common in research on fluency (e.g., de Jong & Perfetti Reference de Jong and Perfetti2011; Bui & Huang Reference Bui and Huang2016; Thai & Boers Reference Thai and Boers2016), probably because uninterrupted speech lends itself more easily to measures such as speech rate and mean length of run. In conversation, these facets of speech will be influenced by interlocutors’ questions, overlaps, interruptions, and so on (see Tavakoli Reference Tavakoli2016). On the other hand, it is safe to say that more real-life communication is bi-directional than unidirectional, and so it seems pertinent to gauge the impact of a given instructional treatment also on learners’ conversational fluency. Furthermore, conversation likely elicits interactional formulas that one does not need to deliver a narrative monologue. We therefore recommend expanding the test-data collection so as to include a dialogic task, even if this throws up new challenges for fluency measurement (Tavakoli Reference Tavakoli2016). It may also be worth triangulating the outcome of objective measures (such as speech rate) with assessors’ ‘holistic’ evaluations (e.g., Préfontaine, Kormos & Johnson Reference Préfontaine, Kormos and Johnson2016).

Another dependent measure which could be fine-tuned is the identification of formulaic sequences in a learner-participant's output. As mentioned, Wood (Reference Wood2009) counted as formulaic sequences those word strings used by his participant which were produced in a hesitant-free manner. However, some of these word strings listed in the appendix to the article appear idiosyncratic and not totally target-like (e.g., for more that two hours; almost the all). These examples suggest that the analysis concerned word strings that appeared to be proceduralised knowledge on the part of the learner, regardless of whether these word strings corresponded to an external criterion, for example, grammatical accuracy, inclusion in a phrasal dictionary or corpus-frequency data such as mutual information scores. While a hesitant-free word string used by a given learner may indeed indicate that this word string is a formulaic sequence to that learner, it would nonetheless seem useful to distinguish non-target-like cases from the use of ‘standard’ expressions, especially if the aim of the investigation is to evaluate the merits of an instructional programme. For one thing, idiosyncratic renderings of multiword expressions that were meant to be ‘mined’ from the model input given as part of the fluency workshop would perhaps point to a need for better guidance at that input-mining stage. For another, although a learner's proceduralised knowledge of a word string may help that learner's fluent delivery of her message, the idiosyncratic nature of the utterance may put strain on the interlocutor's processing of the message – that, at least, has been shown in the context of reading (Millar Reference Millar2010).

In the replication we are proposing here, it would therefore seem worth distinguishing between four categories of expressions in a learner's speech: (a) standard multiword expressions produced in a hesitant-free manner; (b) standard multiword expressions not produced in a hesitant-free manner; (c) non-target-like renderings of expressions produced in a hesitant-free manner; and (d) non-target-like renderings of expressions not produced in a hesitant-free manner. Each of these categories could be informative in its own way: category (a) is indicative of the effectiveness of the treatment, category (b) shows which expressions require additional practice for them to become procedural knowledge, category (c) warns of the ‘risk’ that non-optimal knowledge can also become entrenched because of (unchecked) repeated use, and category (d) points to instances where remedial action is likely to be most timely. Across these categories, an additional distinction between a learner's (attempted) uses of multiword expressions that were included in the formulaic sequences-focused workshop and those picked up outside of that workshop would also be useful as a way of determining the impact of the workshop as such.

3.2 Boers et al. (Reference Boers, Eyckmans, Kappel, Stengers and Demecheleer2006)

Boers et al. (Reference Boers, Eyckmans, Kappel, Stengers and Demecheleer2006) assumed that participants from both the experimental and comparison groups started at a similar level of proficiency based on an interview taken three months prior to the study. This interview was not a pre-test and therefore was not compared with the post-test interview. An approximate replication would therefore benefit from the inclusion of pre-test measures that are comparable with post-test measures. This means that the same interview could be conducted pre- and post-test and analysed for the presence of formulaic sequences and indicators of fluency. The original study only used rater judgements, which are inevitably a subjective measure. An improvement to the measurement of fluency could be to add more objective measures such as analysing speech rate and mean length of runs; measures commonly used to investigate speaking fluency. Strengthening the analysis of fluency by adding objective measures would help to further clarify the effect of noticing formulaic sequences on actual speaking fluency. Further, identifying and categorising the formulaic sequences in the learners’ spoken output, as outlined in the suggestions for the replication for Wood (Reference Wood2009), would also be a worthwhile innovation in any replication of Boers et al. (Reference Boers, Eyckmans, Kappel, Stengers and Demecheleer2006). That is, any replication of the 2006 study would benefit from identifying and reporting formulaic sequences that were included in the course materials, and distinguishing these from any different sequences arising from the prompt or from outside of the course instruction.

In the original study, use of formulaic sequences was counted from the interview. This approach could be extended by introducing a direct test of formulaic sequence knowledge before and after the treatment in a cloze test format. In an interview, a speaker may not make use of formulaic sequences that they know thereby withholding evidence of learning that did actually take place as a result of the treatment. Direct testing of target formulaic sequences would in this way allow direct evaluation of formulaic sequence knowledge gains. In Boers et al. (Reference Boers, Eyckmans, Kappel, Stengers and Demecheleer2006), there was a comparison group, which is important to retain in any replication so that the effect of test repetition as well as ‘accidental’ uptake of formulaic sequences outside the study context is able to be identified against learning attributable to the intervention only. Additional learning not to be attributed to the instructional treatment under investigation is particularly likely if the study is replicated in immersion (e.g., ESL) settings.