INTRODUCTION
With the development of theories of instructed second language acquisition (SLA) and pedagogic criteria for task design and sequencing (Long, Reference Long2015), an increasing number of studies have investigated the effect of task complexity on L2 writing (Cho, Reference Cho2015; Kuiken & Vedder, Reference Kuiken and Vedder2019; Lee, Reference Lee2020; Rahimi & Zhang, Reference Rahimi and Zhang2019; Révész et al., Reference Révész, Michel and Gilabert2016; Sasayama, Reference Sasayama2016). Task complexity is “the result of the attentional, memory, reasoning and other information processing demands imposed by the structure of the task on the language learner” (Robinson, Reference Robinson2001, p. 29). Different manipulations of task complexity can generate varying cognitive demands, differently affecting L2 writing performance (Robinson, Reference Robinson2007b, Reference Robinson and Robinson2011a; Skehan, Reference Skehan1996, Reference Skehan1998; Zhang et al., Reference Zhang, Wang and Wu2017). Within the paradigm of task complexity, researchers have traditionally hypothesized, rather than validated, that the designed task complexity distinguished levels of cognitive demands (Baralt et al., Reference Baralt, Gilabert, Robinson, Baralt, Gilabert and Robinson2014; Lee, Reference Lee2019, Reference Lee2020; Révész et al., Reference Révész, Michel and Gilabert2016; Sasayama, Reference Sasayama2016; Sasayama & Norris, Reference Sasayama, Norris, Wen and Ahmadian2019). For example, task complexity differential has been typically operationalized by designing simple and complex versions (Wang & Zhang, Reference Wang, Zhang, Wen and Ahmadian2019). The complex version has usually been assumed to call for more cognitive demands than the simple one.
Independently measuring every construct, according to Norris and Ortega (Reference Norris, Ortega, Doughty and Long2003), is vital for evaluating any theoretical framework. Although task complexity is the key independent variable in the task-based research, independently validating the difference in the manipulated complexity of task demands has received little attention. In a small number of studies that have measured the manipulations of task complexity, using a post-task self-assessment questionnaire has been the most commonly used method (Rahimi, Reference Rahimi2019; Rahimi & Zhang, Reference Rahimi and Zhang2018; Robinson, Reference Robinson2001, Reference Robinson2007b). Additional cognitive-psychological measures to self-ratings have been called for to confirm presumed task complexity manipulations (Baralt et al., Reference Baralt, Gilabert, Robinson, Baralt, Gilabert and Robinson2014; Housen & Simoens, Reference Housen and Simoens2016; Révész, Reference Révész2014). Recently, only a few studies (Baralt, Reference Baralt2013; Donate, Reference Donate2018; Lee, Reference Lee2019; Révész et al., Reference Révész, Sachs and Hama2014, 2016; Sasayama, Reference Sasayama2016) have made endeavors to examine the relative effects of task complexity and cognitive demands in L1/L2 oral tasks by using techniques such as dual-task methodology, time estimation, and eye-tracking. A paucity of task-based research, however, has provided separate validations of task complexity manipulations within L2 writing tasks by using multiple measurements (Lee, Reference Lee2020), which may be one of the main reasons for the inconclusive findings of the effects of task complexity on L2 writing.
To fill the existing research gap, the lack of validation of task complexity in L2 writing, this study aimed to independently and multidimensionally validate the efficacy of the task complexity manipulations within writing tasks by using self-ratings and the dual-task methodology. Our study is expected to contribute to innovations in the independent assessment of intended task complexity of L2 writing by adopting multi-methods as a way of methodological triangulation. This is an initial effort to use the dual-task technique to measure the cognitive demands imposed by L2 writing tasks in task-based research.
LITERATURE REVIEW
TASK COMPLEXITY
Two well-known models of task complexity, Skehan’s (Reference Skehan1996, Reference Skehan1998) Limited Attentional Capacity Model and Robinson’s (Reference Robinson2001, Reference Robinson2005, Reference Robinson and Robinson2011a) Cognition Hypothesis, have identified a number of dimensions and variables by which task complexity is determined. Skehan (Reference Skehan1996, Reference Skehan1998, Reference Skehan2018) distinguished three factors contributing to the complexity of tasks: code complexity, cognitive complexity, and communicative stress. These three dimensions influence the complexity of tasks and consequently, the amount of attention and cognitive demands the task requires of the learners (Skehan & Foster, Reference Skehan, Foster and Robinson2001). Learners can attend only to one aspect of language production (i.e., complexity, accuracy, or fluency) when completing a cognitively demanding task, as “humans have a limited information processing capacity and must therefore prioritize where they allocate their attention” (Skehan, Reference Skehan, Bygate, Skehan and Swain2013, p. 189).
Contrary to Skehan’s Limited Attentional Capacity Model, Robinson’s (Reference Robinson2001, Reference Robinson2005, Reference Robinson and Robinson2011a, Reference Robinson2011b) Cognition Hypothesis holds that dimensions of task complexity belong to different attentional resource pools, namely, resource-directing dimensions that place cognitive/conceptual demands on learners, and resource-dispersing dimensions that impose performative/procedural demands on learners. Increases in task complexity along resource-directing dimensions, for example increasing the number of elements involved in the task, place cognitive demands on learners, which can direct learners’ attentional resources to language form and facilitate “the development and acquisition of new L2 form-concept mappings” (Robinson, Reference Robinson and Mayo2007a, p. 18).
Task complexity and cognitive demands are the key constructs in models of task complexity. A large part of existing empirical research has explored the effects of task complexity on L2 writing (Abrams, Reference Abrams2019; Ishikawa, Reference Ishikawa and Mayo2007; Kang & Lee, Reference Kang and Lee2019; Kuiken et al., Reference Kuiken, Mos and Vedder2005; Kuiken & Vedder, Reference Kuiken, Vedder and Mayo2007, Reference Kuiken and Vedder2008, Reference Kuiken, Vedder, Housen, Kuiken and Vedder2012; Ong & Zhang, Reference Ong and Zhang2010, Reference Ong and Zhang2013; Rahimi, Reference Rahimi2019; Rahimi & Zhang, Reference Rahimi and Zhang2019; Révész et al., Reference Révész, Kourtali and Mazgutova2017). Little attention, however, has been paid to the validity of task complexity manipulations in L2 writing research. Therefore, this study was designed to explore ways in which the validity of intended task complexity manipulations can be assessed, that is, whether writing tasks designed to be more complex indeed generate higher cognitive demands during the writing process.
SELF-RATING SCALES AS MEASURES OF COGNITIVE DEMANDS
Subjective self-rating scales have been the most commonly used method to examine the presumed task complexity in the field of SLA (Révész et al., Reference Révész, Michel and Gilabert2016; Sasayama, Reference Sasayama2016).
As one of the earliest studies, Robinson (Reference Robinson2001) independently examined the effects of task complexity on learner perceptions of task difficulty by using a 9-point self-rating scale. He found that participants rated a hypothesized-to-be simple task as significantly less difficult and less stressful than the designed complex task. To measure participants’ perceptions of task difficulty, Robinson assessed their perceived levels of task difficulty, stress, confidence, interest, and motivation. The variables in Robinson’s scale, however, do not have identical constructs with perceived cognitive demands, although they are often highly correlated (Brünken et al., Reference Brünken, Seufert, Paas, Plass, Moreno and Brünken2010). Task difficulty refers to learners’ perceptions of the cognitive demands arising from the task (Robinson, Reference Robinson2005, Reference Robinson and Mayo2007a, Reference Robinson and Robinson2011a), whereas cognitive demand reflects “the load that performing a particular task imposes on the cognitive system of a learner” (Paas & Van Merriënboer, Reference Paas and Van Merriënboer1994, p. 353). In the cognitive psychology research literature, mental effort is widely used as a direct measurement of cognitive demand, which reflects “the amount of resources actually allocated to accommodate the task demands” and “the amount of controlled processing in which the individual is engaged” (Paas et al., Reference Paas, van Merriënboer and Adam1994, p. 420). In contrast to task difficulty, which emphasizes learners’ perceptions of the tasks, mental effort focuses on the actual effort made by individuals (Sasayama, Reference Sasayama2016).
Although Robinson’s task difficulty self-rating scale has been widely and actively used in recent task-complexity studies (Rahimi, Reference Rahimi2019; Rahimi & Zhang, Reference Rahimi and Zhang2018, Reference Rahimi and Zhang2019; Robinson, Reference Robinson2007b), this scale does not exactly reflect cognitive demands placed on learners by the manipulations of task complexity. A multidimensional scale with a combination of mental effort and task difficulty is, therefore, desirable for providing a more comprehensive and scientific evaluation of perceived cognitive demands.
To date, some researchers (Donate, Reference Donate2018; Lee, Reference Lee2019, Reference Lee2020; Révész et al., Reference Révész, Michel and Gilabert2016; Sasayama, Reference Sasayama2016) have used a scale that includes mental effort and task difficulty to tap into learners’ cognitive demands in the field of task-based research. A two-item, paper-based questionnaire was first used by Sasayama (Reference Sasayama2016) to measure both task difficulty and mental effort: “(a) Was telling the story in English easy? (b) How much brain power did you use to complete this task?” (p. 238). She found that among the four oral tasks, the hypothesized simplest task was rated as the least difficult and the least mental-effort-demanding, while the task designed to be the most complex was rated as the most difficult, requiring the greatest mental effort. These may not, however, accurately represent the cognitive demands yielded by the four oral tasks per se, as participants completed each task under a dual-task condition. The results obtained for task difficulty and mental effort, therefore, may include not only participants’ perceptions on performing the oral task but also the perceptions related to the dual-task condition.
Donate (Reference Donate2018) extended Sasayama’s (Reference Sasayama2016) research on English learners of Spanish. She used the same subjective measures, self-ratings of task difficulty and mental effort and time estimation, to validate the cognitive demands posed by two oral narrative tasks, which were proven to be the simplest and the most complex tasks in Sasayama’s (Reference Sasayama2016) study. Findings showed that the hypothesized-to-be complex task required significantly greater mental effort than the simple task. No significant differences, however, were found for self-perceived task difficulty and time estimation, although the intended complexity level of these two tasks had been confirmed in Sasayama’s study. Donate (Reference Donate2018) speculated that the low level of proficiency for the novice learners of Spanish and the type of task might be factors that led to the unexpected non-significant results.
In Lee’s (Reference Lee2019) study, a questionnaire with 9-point Likert scales on task difficulty, mental effort, and stress was used to measure native English-speakers’ perceptions of task-generated cognitive demands. Three types of oral tasks (i.e., direction-giving map task, seating arrangement task, and car accident task) were employed with three intended versions of task complexity in each task-type: least-complex, mid-complex, and most-complex. The predictions that increased task complexity yielded more cognitive demands were borne out by the results of self-ratings on task difficulty, mental effort, and stress, although mental effort and stress ratings on the most and mid-complex versions of the map task did not differ significantly. Similar to Sasayama’s (Reference Sasayama2016) research, all participants in Lee’s study engaged in the dual-task condition. Consequently, the self-rating data may exaggerate the effects of task complexity manipulations.
Révész et al. (Reference Révész, Michel and Gilabert2016) used a 6-item computer-based questionnaire on a 9-point scale to measure the manipulations of the designed simple and complex versions of three pairs of oral tasks: a pair of picture narrative tasks, a pair of map tasks, and a pair of decision-making tasks. In contrast to Sasayama’s (Reference Sasayama2016) and Lee’s (Reference Lee2019) research designs, participants in Révész et al.’s (Reference Révész, Michel and Gilabert2016) study were allocated into two groups: a dual-task condition and a single condition (without an added secondary task). Only students in the single condition completed the scales after each oral task. The findings showed that for the map and decision-making tasks, but not the narrative tasks, the hypothesized complex version was rated as more difficult and consumed more mental effort than the designed simple task version.
Lee (Reference Lee2020) adapted a multidimensional questionnaire from Lee’s (Reference Lee2019) study, in addition to the measures of experts’ judgments and time-on-task, to investigate the manipulations of task complexity in L2 writing in a Korean context. Task complexity within two decision-making writing tasks, a Hotel Task and a Venue Task, was manipulated by varying the number of elements as well as a variable of [±open solution]. Participants found the hypothesized-to-be complex task more difficult, stressful, and mental-effort consuming, and they were more pressed for time when planning and writing the designed complex task than the simple one. These results were consistent with findings shown by expert judgments and time-on-task measures. No objective techniques, such as dual-task method, for cognitive demands validation were employed to triangulate the findings obtained by subjective methods in this study.
Although prior studies (Donate, Reference Donate2018; Lee, Reference Lee2019, Reference Lee2020; Révész et al., Reference Révész, Michel and Gilabert2016; Sasayama, Reference Sasayama2016) found that the multifaceted self-assessment is a viable way to tap into the task-generated cognitive demands, learners’ self-perceptions may vary according to their proficiency (Donate, Reference Donate2018; Sasayama, Reference Sasayama2016), task type (Lee, Reference Lee2019; Révész et al., Reference Révész, Michel and Gilabert2016), and task modality. More research in the field of L2 writing is, therefore, warranted to test the utility of self-rating scales in confirming the manipulations of intended task complexity, especially within an argumentative task-type that has not been sufficiently validated.
DUAL-TASK METHODOLOGY AS A MEASURE OF COGNITIVE DEMANDS
Dual-task methodology, as suggested by its name, involves performing an additional secondary task simultaneously with a primary task. During the primary task, participants react to a variable-timed probe stimulus (i.e., secondary task) as quickly as possible. Dual-task methodology, as an objective measurement, serves as a direct measure of cognitive demands without the influence of mediating variables, and is independent from the learners’ self-report (Brünken et al., Reference Brünken, Plass and Leutner2003; Schoor et al., Reference Schoor, Bannert and Brünken2012).
The principle underlying the dual-task technique is that the amount of cognitive demand induced by the primary task will be reflected in the secondary task performance, which is normally assessed in terms of reaction time and accuracy rate (Brünken et al., Reference Brünken, Plass and Leutner2003; Révész, Reference Révész2014; Révész et al., Reference Révész, Sachs and Hama2014, 2016; Schoor et al., Reference Schoor, Bannert and Brünken2012). For instance, when participants completed a primary task with more cognitive demands, less accuracy and more reaction time were expected on their secondary task performance than when they performed a simple primary task. According to the suggestions of several researchers (e.g., Brünken et al., Reference Brünken, Plass and Leutner2003; Marcus et al., Reference Marcus, Cooper and Sweller1996; Schoor et al., Reference Schoor, Bannert and Brünken2012), secondary tasks should be in the form of simple activities, such as an easy visual or auditory stimulus; provide as little disturbance as possible to the primary task; be continuously scored, independent of the primary task performance; and not suppress the quality or quantity of primary task performance.
Dual-task methodology has been employed in a great number of studies in the field of cognitive psychology (Brünken et al., Reference Brünken, Plass and Leutner2003, Reference Brünken, Seufert, Paas, Plass, Moreno and Brünken2010; Marcus et al., Reference Marcus, Cooper and Sweller1996; Schoor et al., Reference Schoor, Bannert and Brünken2012), and is considered a reliable and sensitive approach for measuring cognitive demands. Dual-task methodology, however, has been given insufficient attention in the field of SLA (cf. Révész et al., Reference Révész, Sachs and Hama2014, 2016; Sasayama, Reference Sasayama2016).
Researchers have recently adopted dual-task methodology as a measure of task-generated cognitive demands in studies of task complexity. For example, Révész et al. (Reference Révész, Sachs and Hama2014, 2016) conducted studies using it and other techniques to independently assess the validity of task complexity. In Révész et al.’s (Reference Révész, Sachs and Hama2014) study, two oral tasks with different reasoning demands were the primary tasks in which participants were asked to describe a famous person’s life by using a past counterfactual statement based on two past events they chose. The secondary task was to respond to the change of the computer background screen, which was programmed to randomly change the color to red or green during each 2,500-ms interval. Half the participants were instructed to respond to green (ignoring red), while the other half were asked to respond to red (ignoring green). Accuracy and reaction time were recorded to reflect the level of cognitive demands imposed by the primary tasks. The findings showed that participants produced higher accuracy rates on the secondary task when performing the designed simple primary task, although no significant differences were found in the secondary-task reaction time.
Révész et al. (Reference Révész, Michel and Gilabert2016) triangulated the findings obtained from the dual-task methodology and other techniques (i.e., self-rating scale and expert judgment) to respectively measure the cognitive demands imposed by three pairs of oral tasks as the primary tasks. The secondary tasks used in these two studies (Révész et al., Reference Révész, Sachs and Hama2014, 2016) were identical, although the primary tasks were quite different. Three pairs of designed simple and complex oral tasks were used as the primary tasks, and the three modes were picture narrative, map, and decision-making tasks. Participants achieved a higher mean accuracy rate but non-salient reaction time when carrying out the simpler version of the primary tasks. The results for accuracy rates were found to be consistent with the findings of self-ratings and expert judgments. Based on the findings for accuracy and reaction time in the two studies mentioned, Révész et al. (Reference Révész, Michel and Gilabert2016) postulated that accuracy rates might be more sensitive than reaction time under the color detection secondary-task condition.
Sasayama (Reference Sasayama2016) employed the dual-task method, together with time estimation and self-rating techniques, to measure intended task complexity. She selected a color-detection design as the secondary task. The color of letter A was programmed to change unpredictably from black to red and from red to black. The red color appeared during each 5,000-ms interval, while the duration of the black letter was set with a variation from 3,000 to 60,000 ms. Participants were required to react as quickly as possible only when letter A was changed from red to black. Four narrative oral tasks, which involved an increasing number of elements, were designed to vary the cognitive demands of the primary tasks. Sasayama found a trend that participants responded to the color stimulus the fastest when carrying out the hypothesized-to-be simplest task, and the slowest when engaging in the most complex one, although this difference was not significant. The same pattern, however, was not identified for the hypothesized intermediate complex versions. Sasayama (Reference Sasayama2016) reported consistent patterns of results on three measurements of task complexity: dual-task methodology, self-ratings, and time estimation.
Lee (Reference Lee2019) employed the dual-task method to validate three-levels of task complexity manipulations of three types of oral tasks completed by 42 English native speakers. For the secondary task in her study, participants reacted to screen color changes of a laptop, which changed at a 2,500 ms-interval from either white to red or white to green. Participants were instructed to press, as quickly and accurately as possible, the right shift key when the screen was changed from white to red, and the left shift key when changed from white to green. The findings showed that participants reacted slowest to the hypothesized most-complex versions than to the mid- and least-complex versions, which is consistent with the results on self-ratings, although no significant differences were found in accuracy. The task-complexity effects on secondary-task reaction time were also found to be moderated by the types of tasks. The results of this study may, however, not reveal the effects of intended task complexity on the cognitive demands of L2 learners. Compared to the native speakers, L2 learners who had language barriers may differentially react to the secondary-task color stimuli, perceive the difficulty of the tasks, and make distinct actual mental efforts during the (primary) task completion.
Révész et al. (Reference Révész, Sachs and Hama2014, 2016), Sasayama (Reference Sasayama2016), and Lee (Reference Lee2019) pioneered the dual-task paradigm to independently measure cognitive demands of oral tasks in task-based research. In contrast to the result patterns found in Révész et al.’s studies (Révész et al., Reference Révész, Sachs and Hama2014, 2016), Lee (Reference Lee2019) found a stronger effect in the dual-task experiment for reaction time than for accuracy. Lee (Reference Lee2019) conjectured that the secondary task she used may have created an interference: The counterintuitive key-color assignments “may have been complex enough to capture different reaction times according to the complexity of the primary tasks” (p. 533). The different result patterns of reaction time and accuracy found in these three studies show that the distinct design of the secondary task might lead to participants’ different responses to the stimulus. Given that employing the dual-task method to validate task complexity manipulations in the SLA field is still at a fledging period, inclusion of various versions of secondary tasks to measure the cognitive demands imposed by the same primary tasks is warranted. Additionally, the primary tasks in these three studies were oral, and the secondary tasks were visually presented. Some scholars commented that the secondary tasks should be chosen based on the modality of the primary tasks (e.g., Schoor et al., Reference Schoor, Bannert and Brünken2012). The visual secondary task with a color-detection design is obviously not appropriate when the primary tasks are written.
Reed et al.’s (Reference Reed, Burton and Kelly1985) study is one of the few that have used dual-task methodology in the field of language writing research. Their study explored the effects of writing ability and mode of discourse on cognitive capacity engagement by employing dual-task methodology. Three writing tasks in forms of descriptive, narrative, and persuasive writing served as the primary tasks. The secondary task was to react to a tone at random from a microcomputer by pressing the space key. The tone was presented 20 times in total, randomly appearing 5 times at the prewriting stage, 10 times at the writing stage, and 5 times at the rewriting stage. Cognitive-capacity engagement was analyzed by participants’ reaction time. The results showed that writers’ cognitive-capacity engagement across different modes was differentially affected by their writing ability. Writers with advanced writing ability were most engaged when carrying out persuasive writing tasks and least engaged while writing descriptive essays; whereas writers with average writing ability were the most engaged when writing descriptive essays and least engaged in narrative writing tasks. Although not related to task complexity, Reed et al.’s (Reference Reed, Burton and Kelly1985) study provides a different viewpoint on using dual-task methodology in the field of writing research. The dual-task technique has rarely been used to measure the cognitive demands imposed on learners by writing tasks in the field of task-based research. The auditory stimulus and the sound-response format, as in Reed et al.’s (Reference Reed, Burton and Kelly1985) research, can be applied to the secondary task to measure the cognitive demands of primary writing tasks on learners.
The literature review signals scant validation of manipulations of task complexity in L2 writing using multiple measures, although the effects of task complexity on EFL/L2 writing have been given increased attention. As the key independent variable in task-based research, the efficacy of task complexity manipulations in EFL/L2 writing is of vital importance and, therefore, needs to be validated.
RESEARCH QUESTIONS
This study addressed the following research questions:
-
1. Do different levels of intended task complexity result in different levels of cognitive demands during task performance, as measured by self-perceived task difficulty and mental effort self-assessment?
-
2. Do different levels of intended task complexity result in different levels of cognitive demands during task performance, as measured by dual-task methodology?
METHODS
PARTICIPANTS
Ninety-two learners of English in three intact classes from a university in China voluntarily participated in this study. Their mean ages were 20, with a range of 19–22, and the majority of them were female (90% female and 10% male). No participants had overseas learning experiences. Their years of learning English ranged from 8 to 14 years, with an average of 10.88 years.
Students in three intact classes as a whole were randomly assigned into three groups with different experimental conditions. Based on their last semester final exam results of Integrated English Course and English Writing Course, no significant differences were found in students’ English proficiency across these three intact classes. One class with 31 students was assigned as a single-task group and the other two classes as dual-task groups, with 30 students in dual-task Experimental 1 and 31 students in dual-task Experimental 2.
INSTRUMENTS
Self-Perceived Task Difficulty and Mental Effort Questionnaire
The self-perceived Task Difficulty and Mental Effort Questionnaire was adopted from Révész et al. (Reference Révész, Michel and Gilabert2016). The paper-based self-rating scale, translated into Chinese, included six items with a 9-point response scale ranging from 1 (very easy/not at all) to 9 (very difficult/intensively). Based on the precedent of Révész et al. (Reference Révész, Michel and Gilabert2016), only two items (Item 1 and Item 3) out of six relevant to measuring cognitive demands are discussed in this study. Item 1 assessed participants’ perceptions of mental effort and item 3 was about participants’ feelings of the task difficulty.
Primary Writing Tasks
The two writing tasks were adapted from Cho’s (Reference Cho2015) work with the topic “choosing the best roommates.” The prompts of the two writing tasks were translated into Chinese and modified to suit the L2 learning context in China to ensure that all the participants would fully and quickly understand the topics and instructions. These tasks were chosen because students living in the dormitory on the university campus in China were familiar with and interested in this topic.
In the two argumentative writing tasks, task complexity was manipulated by varying the number of argument elements and reasoning demands. The designed-to-be simple task required learners to choose one pair of best roommates from four candidates, each marked by four attributes: personality, hobbies, sleeping patterns, and studying style. The designed complex task asked students to choose two pairs of best roommates from six candidates, each marked by six properties: personality, hobbies, sleeping patterns, studying style, favorite subjects, and individual sanitary habits. The complex task, compared to the simple version, was characterized by more argument elements and more reasoning demands, with two more candidates and two more properties of each candidate from which to choose. The rationale for such intended task complexity manipulations was that to complete the tasks with more argument elements and reasoning demands was expected to consume more cognitive demands than those with fewer elements and lower demands on reasoning (Cho, Reference Cho2015; Kuiken & Vedder, Reference Kuiken, Vedder and Mayo2007; Rahimi, Reference Rahimi2019; Robinson, Reference Robinson2005, Reference Robinson2007b).
Secondary Tasks in Dual-Task Experiments
The secondary tasks in the dual-task experiments were simple, continuous auditory tasks. The two versions of secondary tasks were designed based on previous studies, with the format of the stimuli, such as stimulus-detection and stimulus-response, and frequency of the stimulus being taken into account (Brünken et al., Reference Brünken, Plass and Leutner2003; Marcus et al., Reference Marcus, Cooper and Sweller1996; Reed et al., Reference Reed, Burton and Kelly1985; Révész, Reference Révész2014; Révész et al., Reference Révész, Sachs and Hama2014; Sasayama, Reference Sasayama2016; Schoor et al., Reference Schoor, Bannert and Brünken2012). Two dual-task experiments with different secondary tasks, a sound-detection version and a sound-response version, were involved, as different result patterns for reaction time and accuracy were found, likely due to distinct secondary-task designs in previous studies (Lee, Reference Lee2019; Révész et al., Reference Révész, Sachs and Hama2014, Reference Révész, Michel and Gilabert2016; Sasayama, Reference Sasayama2016). Using two versions of secondary tasks to validate the intended task complexity manipulations of the same two primary tasks with similar participants is also a way to triangulate the findings obtained from dual-task methods in L2 writing.
Secondary Task 1. In dual-task Experiment 1, stimuli of the secondary task were two sounds, a high-pitch sound di (1,000 Hz) and a low-pitch sound du (300 Hz). The computer was programmed to generate di and du for 150 ms at random intervals in the range of 30–60 s. Participants were instructed to press “Z” on the keyboard as fast and accurately as possible, when they detected the sound through an earphone. Half the participants were required to react to di ignoring du, and half to du ignoring di. The reaction time and accuracy rates were obtained from E-prime 2.0 in milliseconds.
Secondary Task 2. In dual-task Experiment 2, the sound di (1,000 Hz) was programmed to occur for 150 ms in a variable interval between 20 and 40 s. Participants, upon hearing di, were asked to react as fast and accurately as possible to the stimulus. Participant’s reaction time and accuracy in responses to di by pressing “Z” on the keyboard were recorded by E-prime 2.0 in milliseconds.
PROCEDURES
Participants in the single-task group were asked to complete two writing tasks, each task followed by the Task Difficulty and Mental Effort Questionnaire. Forty minutes was provided for each writing task and 5 min for the Task Difficulty and Mental Effort Questionnaire. A 2-week interval occurred between the two writing tasks, and the two tasks were counterbalanced to avoid practice and fatigue effects.
Participants in the dual-task conditions were required to complete the primary writing tasks and secondary tasks simultaneously. Participants were tested individually in groups up to 16 students at one time in the school computer room. Procedures for the two experiments were the same, except for the secondary tasks. Each experiment was conducted twice by the researcher with the help of two research assistants. The first time took about 90 min for each participant, including a practice phase, a baseline test, and the first dual-task test, and the second time took about 60 min, involving a warm-up phase and the second dual-task test. There was a 2-week interval between the two experiments, and the order in which the participants carried out the two primary writing tasks was counterbalanced (see Table 1). Task instructions were presented in Chinese to ensure ease of comprehension by participants.
Writing-task counterbalance in the three groups
Note: The tag of the two writing tasks, such as “simple” and “complex” was anonymized as task 1 (T1) and task 2 (T2). In dual-task groups, writing tasks were completed as the primary tasks.
The 15-min practice phase was provided to familiarize participants with task instructions and experimental procedures in the dual-task experiment. Participants were asked to write a self-introduction on the paper, and, at the same time, they reacted to the secondary task. During this phase, participants were encouraged to ask the researcher and research assistants any questions regarding the procedures. The researcher also monitored the procedures to ensure that participants were appropriately following the task instructions. Some students were given additional instructions as needed. No participants reported they had hearing problems or had difficulty in recognizing the sound(s).
For the baseline test, participants were asked to complete the secondary task in 20 min without an added primary writing task. The secondary task that participants performed during the baseline test was the same as what they would complete during the actual dual-task experiments. Participants in the two experiments were each instructed to respond to the sound stimulus as fast and accurately as possible, and they were also informed that their reaction time and accuracy rate would be recorded. After this phase, participants moved on to the 40-min actual dual-task experiment.Footnote 1 They were informed that the writing tasks were a test of their writing ability and would be rated in terms of content, organization, vocabulary, language, and mechanics to ensure that they could engage in the primary tasks during the dual-task experiments.Footnote 2 They were not allowed to consult any papers, books, dictionaries, or other materials during the experiment.
After a 2-week interval, participants engaged in the second experiment. While participants completed a warm-up exercise to become acquainted with the dual-task procedure, they engaged in the actual dual-task experiment.
DATA ANALYSES
Analysis of Dual-Task Experiments
Participants’ responses to the secondary task were recorded every time they pressed the “Z” in keyboard to indicate their response to the sound stimulus. Their response data were analyzed by reaction time of the correct responses and the accuracy rate, following previous research (Révész et al., Reference Révész, Michel and Gilabert2016).
Experiment 1. Accuracy was calculated by dividing the number of times participants correctly reacted to di and ignored du (half of them correctly reacted to du and ignored di) by the total number of stimuli (both di and du) occurring. Reaction time was considered for correct responses only.
Experiment 2. Accuracy rate was calculated by dividing the number of times participants correctly reacted to di by the total number of occurrences of di. Reaction time was calculated by dividing the summed reaction time of accurate responses to di by the total number of correct responses to di.
Statistical Analysis
The analyses for self-ratings were conducted based on single-task performances only. To examine the effects of intended task complexity on self-ratings, a series of paired-samples t-tests was applied. For dual-task experiments, one-way repeated-measures ANOVAs were conducted, with task conditions (i.e., the baseline condition, the simple primary task condition, and the complex primary task condition) as the repeated condition, and reaction time and accuracy rate as the dependent variables.
The alpha level was set at p < .05. ηp2 for ANOVA, and Cohen’s d for the t-test were employed to measure effect sizes. According to Cohen (Reference Cohen1988), ηp2 values of .01, .06, and .14, and d values of .20, .50, and .80, respectively, represent small, medium, and large effect sizes.Footnote 3
RESULTS
TASK DIFFICULTY AND MENTAL EFFORT QUESTIONNAIRE
All data were first inspected for missing values and outliers with a cut-off set at three standard deviations from the mean. No missing values and no outliers were found in the data of the single-task group. Table 2 displays the descriptive results for the Task Difficulty and Mental Effort Questionnaire in the simple and the complex task versions. As Table 2 shows, the hypothesized complex task was rated, on average, to be more difficult and more mental-effort consuming than the task designed to be simple.
Descriptive statistics for self-ratings of task difficulty and mental effort in the simple and complex tasks
Note: The values represent ratings on a 9-point scale (1 = very easy/not at all, 9 = very difficult/intensively). ME = mental effort, TD = task difficulty.
As the assumption of normality was met, separate paired-samples t-tests were computed for self-ratings of task difficulty and mental effort to examine whether there were significant differences between the simple and complex tasks. As expected, participants perceived the hypothesized complex task (M = 5.61, SD = 1.56) as requiring significantly more mental effort: t (30) = −2.39, p = .024, d = −.42, compared with the task designed to be simple (M = 5.00, SD = 1.34), and the effect size was medium. The same was true for participants’ self-ratings on task difficulty for the presumed simple (M = 4.71, SD = 1.13) versus complex tasks (M = 5.35, SD = 1.33), t (30) = −2.48, p = .019, d = −.52. The effect size was medium. These findings indicate that the hypothesized-to-be complex task was more complex than the simple task in terms of self-perceived task difficulty and mental effort.
Pearson’s correlation coefficients were also calculated, results indicating a strong but not perfect correlation between perceived task difficulty and mental effort (simple task: r = .55, p = .001; complex task: r = .61, p < .001). These results suggested that the two variables were related but not isomorphic (Brünken et al., Reference Brünken, Seufert, Paas, Plass, Moreno and Brünken2010; Révész et al., Reference Révész, Michel and Gilabert2016).
DUAL-TASK EXPERIMENT 1
Table 3 summarizes the descriptive statistics for reaction time and accuracy rate on the secondary task (di and du version) under the baseline, simple, and complex task conditions. Participants responded to the sound stimuli fastest under the baseline condition and the slowest when engaging in the hypothesized-to-be complex task. The accuracy rate of the participants’ reaction decreased, as the task condition became more difficult.
Descriptive statistics for reaction time and accuracy rates across baseline, simple and complex task conditions in dual-task Experiment 1
Note: RT = reaction time, ms = milliseconds, Acc = accuracy rates.
The assumptions of normality and sphericity were examined and found to be satisfied, therefore, a series of one-way repeated-measures ANOVA was conducted to explore the effects of intended task complexity on participants’ secondary task performance in terms of reaction time and accuracy. The ANOVA tests showed that each of the two analyses yielded a significant and large effect on participants’ performance in the secondary task across three task conditions, reaction time: F (2, 58) = 37.89, p < .001, ηp 2 = .57; and accuracy rates: F (2, 58) = 21.14, p < .001, ηp 2 = .42. These results suggest that there were significant differences in reaction time and accuracy, across baseline, simple, and complex conditions.
Pairwise comparisons with Bonferroni adjustment were, therefore, performed to investigate where the statistically significant differences were located. For reaction time, participants reacted to the secondary task significantly faster under the baseline condition (M = 543.91, SD = 123.12) than either the simple condition (M = 848.13, SD = 189.61): p < .001, d = −1.90, or the complex condition (M = 934.77, SD = 311.20): p < .001, d = −1.56; the effect sizes were large. In contrast, intended increases in task complexity of the primary writing task did not have a significant impact on participants’ reaction time, p = .193, d = −.31. These results indicate that the design of the dual-task methodology was sensitive, in terms of reaction time, to distinguish the baseline and experimental conditions. Increases in task complexity of the primary tasks, however, did not statistically affect participants’ secondary-task reaction time.
Significantly higher accuracy rates were produced on the secondary task when participants were under the presumed simple primary task condition (M = 94.19, SD = 4.26), as compared with the complex primary task (M = 91.60, SD = 4.62), p = .038, d = .58. The effect size was medium. Participants under the baseline condition (M = 98.11, SD = 2.17) substantially outperformed those under either the simple primary task condition, p = .002, d = 1.19, or the complex condition, p < .001, d = 1.86; the effect sizes were large. The results reveal that accuracy as a measure of students’ cognitive demands can sensitively separate the baseline and experimental conditions and predict participants’ greater consumption of cognitive demands when the task complexity of the primary tasks was increased as intended.
In sum, significant differences were identified between the baseline and simple conditions, and between the baseline and complex conditions, which indicates the design of the sound-detection secondary task used in dual-task Experiment 1 was reasonable and sensitive. Furthermore, participants obtained significantly higher accuracy rates on the secondary task when carrying out the hypothesized simple primary writing task, although no evidence of reaction time was detected to support the intended manipulation of task complexity.
DUAL-TASK EXPERIMENT 2
Table 4 summarizes the descriptive statistics for secondary-task reaction time and accuracy rates under the baseline, simple, and complex task conditions, with one participant’s data excluded as an outlier.
Descriptive statistics for reaction time, and accuracy rates across baseline, simple and complex task conditions in dual-task Experiment 2
Note: RT = reaction time, ms = milliseconds, Acc = accuracy rates.
Prior to conducting one-way repeated-measures ANOVA, the assumptions of normality and sphericity were checked and found to be satisfied. The ANOVA results showed that the effect of task conditions (i.e., baseline, simple, and complex task conditions) on participants’ reaction time of the secondary task was significant, F (2, 58) = 67.66, p < .001, ηp2 = .70, and with a large effect size. As for accuracy, a significant effect of task conditions on participants’ performance was identified, F (2, 58) = 16.58, p < .001, ηp2 = .36, and associated with a large effect size. Pairwise comparisons of estimated marginal means with a Bonferroni adjusted α level were applied to investigate where the statistically significant differences were located. The pairwise comparisons revealed that increases in task complexity of the primary writing task had a significant impact on participants’ reaction time in the secondary task: simple task (M = 496.95, SD = 86.50) versus complex task (M = 533.37, SD = 104.71), p = .008, d = −.37, with an effect size that ranged from small to medium. Participants also responded to the sound significantly faster under the baseline condition (M = 385.14, SD = 65.33), than under either the simple primary task condition, p < .001, d = −1.42, or the complex condition, p < .001, d = −1.58; the effect sizes were large. These findings indicate that intended increases in task complexity of the primary writing task statistically increased participants’ reaction time in the secondary task. Moreover, the design of this version of the secondary task was reasonable and sensitive in terms of reaction time to distinguish the baseline and experimental conditions.
For accuracy rates, intended increases in task complexity of the primary task, in contrast to our expectation, did not contribute to a significant decline: Accuracy in the simple task (M = 95.47, SD = 2.76) was similar to that in the complex task (M = 94.04, SD = 3.10), p = .103, d = .49. Significant differences were, however, detected between the baseline (M = 97.88, SD = 3.14) and the simple conditions, p = .001, d = .81 and between the baseline and the complex conditions, p < .001, d = 1.23; the effect sizes were large. The results suggest that the designed task complexity of the primary task did not statistically affect accuracy, although the baseline and experimental conditions were successfully differentiated by accuracy.
In sum, comparisons of the baseline-simple and baseline-complex conditions indicated that the combination of the two primary tasks and the auditory secondary task in Experiment 2 constituted a relatively sensitive measure of cognitive demands. The results for reaction time of the secondary tasks offer evidence in support of the presumed task complexity manipulations of the two writing tasks; that is the hypothesized complex task was more complex. Although the results for accuracy rates in this experiment were not supportive of the task complexity, the relatively high accuracy rates support the validity of the reaction time data.
DISCUSSION
SELF-RATINGS
The results found that the hypothesized-to-be complex task was perceived as significantly more difficult and requiring significantly more mental effort than the simple task. These findings are in line with previous research (Révész et al., Reference Révész, Michel and Gilabert2016; Sasayama, Reference Sasayama2016). For example, in Révész et al.’s (Reference Révész, Michel and Gilabert2016) study, participants rated the hypothesized complex task more difficult, consuming more mental effort when completing the map and decision-making tasks than the simple task. Sasayama (Reference Sasayama2016) found that among the four tasks, the designed simplest task was rated with the lowest level of task complexity and the most complex task with the highest level of cognitive complexity, measured by both self-ratings of cognitive demands.
The findings in the present study, however, are inconsistent with aspects of Révész et al.’s (Reference Révész, Michel and Gilabert2016) study, in which no significant differences were reported between the participants’ ratings of task difficulty on the narrative tasks. These are interesting findings, as the difference in outcomes may be due to different task types used in our study and in that of Révész et al.’s (Reference Révész, Michel and Gilabert2016). Révész et al. (Reference Révész, Michel and Gilabert2016) posited that participants might be more familiar with the narrative task than the map and decision-making tasks. In contrast, the argumentative tasks used in the present study, compared to the narrative, appear to require increased cognitive demands (Andrews, Reference Andrews1995; Freedman & Pringle, Reference Freedman and Pringle1984), more critical thinking and articulation of ideas, and greater challenges in the use of linguistic elements (Hirose, Reference Hirose2003). Learners may perceive the argumentative tasks as more difficult and consuming more mental effort than the narrative tasks, which may lead to a larger gap of cognitive demands between the simple and complex tasks.
The effect size of task difficulty (d = .52) was larger than that of mental effort (d = .42), although the results of these two measures revealed similar trends across the two writing tasks. This finding suggests that the two variables, task difficulty and mental effort, are related but not identical (Brünken et al., Reference Brünken, Seufert, Paas, Plass, Moreno and Brünken2010; Révész et al., Reference Révész, Michel and Gilabert2016). Task difficulty is based on learners’ perceptions of the cognitive demands of the tasks, determined by the abilities and affective responses that learners bring to the task (Robinson, Reference Robinson2005, Reference Robinson and Mayo2007a, Reference Robinson and Robinson2011a), while mental effort is related to the perceived effort that learners used. The larger effect size of task difficulty than mental effort might suggest that, whereas participants perceived the presumed complex task as more difficult than the task designed to be simple, they, however, exerted only a little more effort in the completion of the complex task than the simple one.
The effect sizes for self-ratings in our study were smaller than those in Sasayama’s (Reference Sasayama2016) study (d = .98 for task difficulty and d = 1.00 for mental effort). A possible reason is that Sasayama’s results of self-ratings contained extra loads imposed by dual-task methodology, in addition to the task-generated cognitive demands, given participants under the dual-task condition were subjected to the self-ratings. The results in our study represented the cognitive demands yielded by the tasks only, as participants reported their perceptions under the single-task condition.
DUAL-TASK EXPERIMENTS
In dual-task Experiment 1, the results showed that participants had significantly higher accuracy rates on the secondary task when carrying out the simple primary writing task versus the complex one. This finding aligns with previous studies (Révész et al., Reference Révész, Sachs and Hama2014, 2016), and partially echoes the principle underlying the dual-task paradigm that when the level of cognitive demands imposed by the primary task is increased, less accurate and more reaction time are expected on the secondary task. The results for secondary-task accuracy, therefore, suggest that the hypothesized complex version of the primary writing task in the present study imposed greater cognitive demands than the simple task.
No significant differences were identified for secondary-task reaction time, however, when participants were under the simple primary task condition, as compared to the complex task condition. This finding is not surprising because the same trends were observed in previous research (Révész et al., Reference Révész, Sachs and Hama2014, 2016; Sasayama, Reference Sasayama2016). This is probably because the design of the secondary task in the present study is relatively similar to those used in previous studies (Révész et al., Reference Révész, Sachs and Hama2014, 2016; Sasayama, Reference Sasayama2016), although the modality of the two secondary tasks is different (i.e., one is an auditory task and the others are visual tasks). Specifically, the secondary task in the present study required participants to detect the change of two sounds. The other three studies (Révész et al., Reference Révész, Sachs and Hama2014, 2016; Sasayama, Reference Sasayama2016) directed students to notice the change of two colors and react to only one. An explanation for the non-salient reaction time, as noted by Révész et al. (Reference Révész, Michel and Gilabert2016), may be that the secondary-task reaction time is less sensitive than the accuracy rate to the cognitive demands generated by the primary task, when the dual-task condition is operationalized as a particular type of color- or sound-detection task.
Another possible explanation for the lack of simultaneous secondary-task effects for reaction time and accuracy might be the potential competition for attentional resources between reaction time and accuracy. In a dual-task experiment, the primary task is normally given priority and, in this way, the limited residual mental resources that are not being used by the primary task could be allocated to the secondary task (Marcus et al., Reference Marcus, Cooper and Sweller1996). For the present and the three aforementioned studies, students had to quickly distinguish whether the sound/color is correct and simultaneously react to the right sound or color when completing the secondary task with limited attentional resources. Completing the secondary task required participants to shift attention from determining whether the stimulus is right or wrong to producing a response to the stimulus. During this process, students may not equally allocate attention to noticing the correct stimulus (accuracy) and responding to the stimulus (reaction time). According to Wickens’ (Reference Wickens2007) Selective Attention Model, participants’ effort of retrieval in the correct sound stimulus might be traded off against the effort of responding to the sound. This suggests that the more effort on reaction time, the fewer differences in reaction time would be found between the simple and complex primary tasks, and accuracy would be curtailed. Given the increased cognitive demands imposed by the primary task, the mental resources for the secondary task would be more limited, and, consequently, the competition might be more intense between the secondary-task reaction time and accuracy. If more attention is directed to reacting to the sound, less attention is allocated to achieving accuracy, and vice versa. Thus, when students complete the complex primary task, two possible result patterns might be found. One is that participants might slightly increase secondary-task reaction time (sometimes reaction time might not be detectable) but statistically decrease accuracy rates, as found in Experiment 1 of our study. Another possible pattern might be, inversely, significantly more reaction time but less detectable changes in accuracy, which was found in Experiment 2 (discussed subsequently). This is probably why when participants completed the designed complex primary writing task in Experiment 1, the supposed-to-be longer reaction time was not found to be statistically increased, but the accuracy rate declined significantly in the secondary task.
With dual-task Experiment 2, the reaction time for the secondary task offered evidence in support of the presumed manipulation of two primary writing tasks. Increases in the task complexity of the primary writing task statistically increased participants’ reaction time in the secondary task. No significant decrease, however, was identified for secondary-task accuracy rate when students completed the hypothesized complex primary writing task versus the simple one. Although patterns for reaction time and accuracy rate, found in Experiment 2, were seemingly opposite to those in Experiment 1, the difference might be due to the designs of the secondary task, which is consistent with the speculation in Lee (Reference Lee2019). Different secondary tasks were connected to varying information processing steps, making it possible to identify on which of the processes the cognitive demand is imposed (Baddeley, Reference Baddeley1986; Brünken et al., Reference Brünken, Plass and Leutner2003). The secondary task in Experiment 1 was a sound-detection task, in which participants were asked to react fast and accurately to a specific sound and ignore another sound. Participants in Experiment 2 were instructed to respond to only one sound, although the stimulus appeared more frequently in comparison with Experiment 1. The designs of the secondary task led to different information processing steps and directed participants’ attention to different aspects, reaction time, or accuracy. As previously discussed, students appeared to focus on reaction time more than accuracy in Experiment 1, leading to significant differences in the secondary-task accuracy rate. A similar example is the color-detection task in previous studies (Révész et al., Reference Révész, Sachs and Hama2014, 2016). Competition between accuracy and reaction time in the secondary task would suggest that participants directed more attention to accuracy than reaction time when the secondary task is operationalized as a tone-response task in Experiment 2 instead of the stimulus-detection task.
From the comparison between the two dual-task experiments, it seems that the sound-detection secondary task in Experiment 1 had a stronger effect than the sound-response version in Experiment 2 on accuracy rates. The effect size for accuracy in Experiment 1 (d = .58) is larger than that found in Experiment 2 (d = .49), indicating a more detectable difference in accuracy elicited by the sound-detection secondary task. For reaction time, despite the statistically non-significant differences found in Experiment 1, the effect sizes in the two experiments were quite similar (d = .31 in Experiment 1, d = .37 in Experiment 2), suggesting a similar secondary-task effect on reaction time in the two experiments.
The pronounced secondary-task effects in Experiment 1 likely stem from its creating “the required level of interference” (Révész et al., Reference Révész, Michel and Gilabert2016, p. 731). Compared to the sound-response secondary task (responding to the sound only) in Experiment 2, the sound-detection version required participants to both notice and respond to the change of two sounds. When the “required level of interference” has been created in the secondary tasks, competition for attentional resources between the primary and secondary tasks would be greater (Révész et al., Reference Révész, Michel and Gilabert2016), leading to a pronounced secondary-task effect with detectable differences in accuracy or reaction time. In contrast, the sound-response version had a lower level of interference, and therefore, is perhaps not sensitive enough to detect the fine difference in accuracy between the two primary tasks. Reduced competition for attentional resources between the primary and secondary tasks led to participants’ rapid responses to the sound (i.e., 848.13 ms for the simple and 934.77 ms for the complex task in Experiment 1, while 496.95 ms for the simple and 533.37 ms for the complex task in Experiment 2). Of note, the effect size for reaction time elicited by the sound-response version is larger than that found for the same measure used in Sasayama (Reference Sasayama2016; d = .34) and Révész et al. (Reference Révész, Sachs and Hama2014; d = .15 for ESL learners), although the secondary-task effects yielded in Experiment 2 were not as pronounced as those elicited in Experiment 1.
To sum up, the results obtained from the dual-task experiments, that is, accuracy rates in Experiment 1 and reaction time in Experiment 2, closely align with the findings of the self-ratings on task difficulty and mental effort. These results suggest that these indicators measure the same construct, cognitive demands, but from different perspectives (Brünken et al., Reference Brünken, Seufert, Paas, Plass, Moreno and Brünken2010), which highlights the necessity of triangulation. The findings obtained from the three data sources appear to confirm that the manipulations of task complexity in this research were successful: The complex task version, as intended, yielded greater cognitive demands than the simple version.
CONCLUSION
This study was set up to validate the intended task complexity manipulated within two argumentative writing tasks by varying numbers of argument elements and reasoning demands. The results for the self-ratings on task difficulty and mental effort, and dual-task experiments indicate that the hypothesized complex task promoted more cognitive demands than the task designed to be simple.
The current study is the first that attempts to assess the manipulations of presumed task complexity in L2 writing by using a dual-task method to objectively measure cognitive demands, although some researchers (e.g., Révész et al., Reference Révész, Michel and Gilabert2016; Sasayama, Reference Sasayama2016) have pioneered such measurement in the oral tasks in L2 task-based research. Dual-task techniques, as well as self-ratings, are proven to be reliable methods for the validation of the intended task complexity manipulations in writing. Triangulation of the data from different sources is critical to ensuring that the findings for the manipulations of task complexity are accurate and reliable.
Various cognitive-demand measures have their own advantages and disadvantages. Self-ratings might be one of the easiest and most appropriate measures used in L2 classrooms. It is easy for teachers to use the Task Difficulty and Mental Effort Questionnaire to understand whether the task complexity manipulations worked as intended. Accordingly, teachers can adjust the cognitive demands of writing tasks to optimize their task-grading and sequencing decisions in L2 teaching pedagogy.
The dual-task method can directly and objectively measure cognitive demands. The dual-task Experiment 1 elicited the largest effect size (d = .58 on accuracy) between the presumed simple and complex tasks across all the measures used in this study. The sound-detection design of the secondary task in Experiment 1 may have been sufficiently robust to capture different secondary-task accuracy. This measure could be employed by curriculum designers as well as test assessors to objectively validate cognitive complexity of the L2 writing tasks in the field of curriculum design and writing assessment.
In contrast, the sound-response version might be more suitable for novice L2 learners, given its stronger effect size for reaction time than those found in previous studies (Révész et al., Reference Révész, Sachs and Hama2014; Sasayama, Reference Sasayama2016). The novice learners’ attentional limits may be easily reached during L2 writing, and the lower level of interference of the sound-response secondary task may provide as little disturbance as possible to the primary task during the dual-task experiment. This suggestion, however, should be interpreted with caution and further investigated, as participants’ proficiency level was not taken into consideration in our study.
Differences in findings of the dual-task Experiment 1 and Experiment 2 in our study highlight the importance of the choice of a secondary task in using dual-task methods, as different versions of secondary tasks may lead to participants’ differential attentional allocation and distinct secondary-task effects. Various versions of secondary tasks need to be implemented in the field of SLA to gain deeper insights into selecting appropriate secondary tasks for validating intended task complexity manipulations, especially in L2 writing, in future research.
It needs to be pointed out that our study validated the manipulation of intended task complexity based on the data from Chinese EFL learners. No baseline data were collected from native speakers of English, who have relatively few language barriers when writing compositions in English. This means that future studies can extend our study by tracking responses of native speakers to the tasks and use the two sources of the data (from L1 speakers and L2 speakers) to triangulate the manipulation of the task complexity.
Our study focused on only the argumentative genre, which limits the generalizability of findings to other task types, given task type may moderate the effects of task complexity on cognitive demands (Lee, Reference Lee2019). Future research would benefit from trialing the methods on various task types and multiple complexity levels of each task type.
Furthermore, replication of this research using self-ratings and, in particular, dual-task methods is worthwhile. For future research, the inclusion of open-ended questionnaires or interviews after the dual-task experiments may help researchers record what each participant was doing in the process of writing while carrying out the secondary task. In addition to self-ratings and dual-task methods, a combination of eye-tracking and keystroke (Révész et al., Reference Révész, Michel and Lee2019) may be a way to access the information on cognitive demands with a consideration of pauses during the writing process. Other techniques, such as time estimation (Lee, Reference Lee2020; Sasayama, Reference Sasayama2016), time-on-task (Lee, Reference Lee2020), stimulated recall (Kim et al., Reference Kim, Payant and Pearson2015), heart rate test, and neuroimaging techniques are also warranted for exploration in future studies to assess the cognitive demands in L2 writing.
SUPPLEMENTARY MATERIALS
To view supplementary material for this article, please visit http://doi.org/10.1017/S0272263121000310.
