In accordance with much applied linguistics research carried out in recent decades (e.g., DeKeyser, Reference DeKeyser, Doughty and Long2003, Reference DeKeyser2005; N. C. Ellis, Reference Ellis1994b; Rebuschat, Reference Rebuschat2013, Reference Rebuschat2015; Roehr-Brackin, Reference Roehr-Brackin2014; Roehr, Reference Roehr2010), the study reported in this article is based on the assumption that explicit and implicit knowledge of language are represented differently and that explicit and implicit learning of language involve distinct cognitive processes. This differentiation between explicit and implicit knowledge and learning is reflected in the measures typically employed to assess these constructs as well as in the conceptualization of second language (L2) instruction as either explicit or implicit (Norris & Ortega, Reference Norris and Ortega2001). In the present article, we additionally conceptualize the notion of learning difficulty in terms of this dichotomy, in that we differentiate between explicit and implicit learning difficulty—that is, the difficulty of learning something as explicit knowledge as opposed to learning something as implicit knowledge (R. Ellis, Reference Ellis2006).
It would be advantageous if a taxonomy of explicit and implicit learning difficulty (Roehr & Gánem-Gutiérrez, Reference Roehr, Gánem-Gutiérrez and Benati2009) based on criteria that can be applied to grammar points as presented in instructional materials could be used to accurately predict the learning difficulty of different L2 constructions and their associated pedagogical grammar rules. This would allow for direct implications for the adult L2 classroom; for instance, in a mixed-ability, general-purpose adult class, teachers may wish to focus on L2 structures and metalinguistic rules of moderate learning difficulty (DeKeyser, Reference DeKeyser, Doughty and Long2003), so all learners can benefit. The present study investigated this issue by (a) comparing the learning difficulty judgements of applied linguistics experts, instructed adult learners, and their teachers for selected L2 grammar points and (b) establishing the nature of the relationship between perceived learning difficulty and actual learner performance in terms of explicit and implicit knowledge of the selected L2 grammar points.
BACKGROUND
Explicit and Implicit Knowledge and Learning
Most SLA researchers would accept that literate, cognitively mature learners are likely to engage in explicit learning to at least some extent and will have varying degrees of explicit knowledge about language. Explicit knowledge is here defined as knowledge that a learner is consciously aware of, that can be called up on demand (Dörnyei, Reference Dörnyei2009) and articulated in a verbal statement (Anderson, Reference Anderson2005; R. Ellis, Reference Ellis2004; Hulstijn, Reference Hulstijn2005). Knowledge of pedagogical grammar rules—that is, metalinguistic propositions describing the form, function, and use of L2 constructions with a view to promoting and guiding instructed language acquisition (Chalker, Reference Chalker, Bygate, Tonkyn and Williams1994; Westney, Reference Westney and Odlin1994)—constitutes an example of explicit knowledge about language. It has been argued that metalinguistic propositions as instantiated in pedagogical grammar rules can be understood in terms of stable, discrete, and context-independent categories and the relations among them (Roehr, Reference Roehr2008a). To illustrate, the rule “When a countable noun is first mentioned, an indefinite article is required” includes the metalinguistic categories countable noun and indefinite article and specifies the relation between them (if X, then Y). Stability and context independence are required for rule-based, conscious processing (Cleeremans & Destrebecqz, Reference Cleeremans and Destrebecqz2005; Smith, Reference Smith2005), which is compositional and systematic in nature. The category structure that characterizes explicit knowledge about language thus relies on clear-cut yes/no distinctions. On the one hand, this category structure facilitates explicit processing, but, on the other hand, it means that metalinguistic propositions are potentially unreliable and inaccurate, because they cannot fully capture context-dependent information about the probability of occurrence of certain L2 constructions, for instance (Roehr, Reference Roehr2008a).
Explicit learning is “characterized by the learner’s conscious and deliberate attempt to master some material or solve a problem” (Dörnyei, Reference Dörnyei2009, p. 136)—for example, when the learner looks for regularities in the language input she or he is exposed to, when she or he employs given pedagogical grammar rules to aid comprehension of input or production of output, or when she or he forms conscious hypotheses about the learning target (Schmidt, Reference Schmidt and Robinson2001). Explicit learning requires effort and strategic expertise, and it both makes use of and results in explicit knowledge. Footnote 1 Explicit learning is relatively taxing because it requires attentional resources for the processing and maintenance of information in working memory. In order to achieve conscious processing, representations from different sources (e.g., visual and phonological information) must be activated, bound, and integrated into a unified, multimodal representation, which is then experienced as a coherent episode (Bayne & Chalmers, Reference Bayne, Chalmers and Cleeremans2003; Dienes & Perner, Reference Dienes, Perner and Cleeremans2003; N. C. Ellis, Reference Ellis2005). When we employ strategic expertise, analytic reasoning, and problem solving in the linguistic domain, we may also engage conscious processes. Such high-level processes are likely to depend on language learning aptitude (DeKeyser, Reference DeKeyser2012), which, according to Carroll’s (Reference Carroll and Glaser1962, Reference Carroll and Diller1981, Reference Carroll, Parry and Stansfield1990) classic model, includes phonetic coding ability, language-analytic ability, and associative memory (see Yalçin & Spada, Reference Yalçin and Spada2016).
Implicit knowledge is intuitive knowledge that cannot be brought into awareness or articulated (Dörnyei, Reference Dörnyei2009; Hulstijn, Reference Hulstijn2005). By the same token, implicit learning is an unconscious, automatic process of induction resulting in intuitive knowledge (Dörnyei, Reference Dörnyei2009; N. C. Ellis, Reference Ellis1994a). Implicit learning is a relatively slow process that typically requires input over a prolonged period of time. Implicit learning results in implicit knowledge, which, once established, can be accessed quickly and effortlessly. Implicit learning relies on similarity-based processing, which is flexible, dynamic, and susceptible to contextual variation (Diesendruck, Reference Diesendruck2005; Markman et al., Reference Markman, Blok, Kom, Larkey, Narvaez, Stilwell and Taylor2005). Accordingly, implicit representations with flexible, context-dependent category structure can fully capture prototype effects and distributional frequencies and can thus result in accurate and reliable knowledge. Although implicit learning is a powerful mechanism, its success is dependent on ample exposure to input. Explicit learning, on the other hand, is potentially fast and efficient, allowing for one-trial learning as well as successful learning from minimal exposure to input.
As time is at a premium in the L2 classroom, both teachers and learners would benefit from knowing what aspects of the L2 can be learned most successfully explicitly and/or implicitly. In connection with this, the question of whether instruction should target more difficult or less difficult L2 structures was raised more than two decades ago.
Learning Difficulty in SLA
The issue of learning difficulty in SLA has been conceptualized in (at least) two different ways (see Housen & Simoens, Reference Housen and Simoens2016): that is, in terms of linguistic or structural complexity and in terms of cognitive complexity. Linguistic or structural complexity is dependent on the characteristics of a linguistic feature or subsystem and can be operationalized by considering the level of markedness, number of transformations, and/or typological distance from the first language (L1) (Collins, Trofimovich, White, Cardoso, & Horst, Reference Collins, Trofimovich, White, Cardoso and Horst2009; Dietz, Reference Dietz2002; Housen, Pierrard, & Van Daele, Reference Housen, Pierrard, Van Daele, Housen and Pierrard2005; Spada & Tomita, Reference Spada and Tomita2010). It appears that such an approach is typically informed by theories that assume specifically linguistic processing mechanisms that are different from general learning mechanisms operating in other cognitive domains.
The notion of cognitive complexity, by contrast, is compatible with the assumption that domain-general processing mechanisms apply to all of cognition, including language. The notion of cognitive complexity, or learning difficulty, can be applied to both explicit and implicit knowledge and learning (DeKeyser, Reference DeKeyser, Doughty and Long2003; R. Ellis, Reference Ellis2006). In the context of explicit knowledge, learning difficulty depends on the properties of the metalinguistic proposition used to describe and explain the form, function, and use of linguistic constructions. In the context of implicit knowledge, learning difficulty refers to the characteristics of the linguistic constructions that are available in the input. For example, metalinguistic propositions such as pedagogical grammar rules may vary in terms of scope and reliability (DeKeyser, Reference DeKeyser1994; Hulstijn & de Graaff, Reference Hulstijn and de Graaff1994), or they may refer to more or less prototypical uses of a construction (Hu, Reference Hu2002). By the same token, the linguistic constructions described may vary in terms of perceptual salience or communicative redundancy (Hulstijn & de Graaff, Reference Hulstijn and de Graaff1994), for instance. These theoretical considerations have more recently been supported by empirical findings indicating that the difficulty of specific L2 grammar points has an influence on whether they are (first) learned implicitly or explicitly (R. Ellis, Reference Ellis2006).
Drawing on existing lists of criteria believed to influence explicit and implicit learning difficulty (DeKeyser, Reference DeKeyser2005; R. Ellis, Reference Ellis2006), Roehr and Gánem-Gutiérrez (Reference Roehr, Gánem-Gutiérrez and Benati2009) proposed a taxonomy for assessing the implicit and explicit learning difficulty of L2 grammar points that avoids composite parameters such as transparency or regularity, or parameters that are dependent on specific linguistic theories, such as processability, in order to make the given criteria applicable in practice to linguistic constructions as well as associated pedagogical grammar rules. According to this taxonomy of learning difficulty (henceforth: R&G taxonomy; see Online Appendix A), frequency in the input, perceptual salience, communicative redundancy, opacity of form-meaning mapping (one form, X meanings), and opacity of meaning-form mapping (one meaning, X forms) refer to the characteristics of linguistic constructions and influence implicit learning difficulty. Thus, for instance, high perceptual salience is expected to decrease learning difficulty as implicit knowledge, whereas high communicative redundancy is expected to increase learning difficulty. The criterion of schematicity refers to the characteristics of both linguistic constructions and metalinguistic descriptions and affects both implicit and explicit learning difficulty. Finally, conceptual complexity, technicality of metalanguage, and truth value refer to the characteristics of pedagogical grammar rules and impact on explicit learning difficulty. Thus, high conceptual complexity of a metalinguistic rule, for instance, is expected to increase learning difficulty as explicit knowledge, whereas high truth value is expected to decrease explicit learning difficulty (Roehr & Gánem-Gutiérrez, Reference Roehr, Gánem-Gutiérrez and Benati2009).
Given the large number of criteria that are likely to influence learning difficulty, implications for the L2 classroom are not immediately obvious. Even if the focus is on explicit learning difficulty only, researchers do not appear to have arrived at a consensus view as to which pedagogical grammar rules may be most helpful to the L2 learner. Earlier work advocated fairly categorically either the teaching of more complex metalinguistic descriptions (Hulstijn & de Graaff, Reference Hulstijn and de Graaff1994) or the teaching of simpler rules (DeKeyser, Reference DeKeyser1994). More recently, it has been suggested that teachers may wish to focus on metalinguistic rules of moderate learning difficulty (DeKeyser, Reference DeKeyser, Doughty and Long2003), so all learners can benefit. Although this sounds intuitively plausible, empirical research that takes into account learners’ perceptions of difficulty suggests that the issue may not be quite so straightforward.
Learners’ Perceptions
Two recent studies (Scheffler, Reference Scheffler2009; Thepseenu & Roehr, Reference Thepseenu, Roehr, Roehr and Gánem-Gutiérrez2013) examined the perceived learning difficulty and perceived usefulness of instruction of selected structures of L2 English among adult learners (for a further study on learners’ perceptions of difficulty, see Shiu, Reference Shiu2011). In an investigation with 100 L1 Polish college-level learners (Scheffler, Reference Scheffler2009), participants were asked to judge 11 grammar points on a 5-point scale ranging from very easy to very difficult as well as to assess the usefulness of instruction on these grammar points on a 5-point scale ranging from very useful to not useful at all. Findings show that if learners considered a grammar point to be more difficult, instruction was considered useful. In another study with 64 L1 Thai university-level learners (Thepseenu & Roehr, Reference Thepseenu, Roehr, Roehr and Gánem-Gutiérrez2013), participants were asked to judge 12 grammar points on the same difficulty and usefulness scales. In this case, the results show that if a grammar point was considered less difficult, instruction was considered useful. It should be noted that the grammar points investigated in the two studies were not identical. Moreover, the research design may have had an impact on the results, because the first study (Scheffler, Reference Scheffler2009) asked two different groups of learners to judge either difficulty or usefulness of instruction, whereas in the second study (Thepseenu & Roehr, Reference Thepseenu, Roehr, Roehr and Gánem-Gutiérrez2013) the same group of learners made the difficulty and usefulness-of-instruction judgements; the learners also performed a metalinguistic task prior to making their judgements. Finally, a formal L2 proficiency measure taken by the Thai learners revealed that they were at an elementary level (Thepseenu & Roehr, Reference Thepseenu, Roehr, Roehr and Gánem-Gutiérrez2013, p. 101), whereas the Polish learners appeared to have a higher level of English, though no formal assessment was reported. Despite these differences, it is nonetheless surprising that directly opposing findings were obtained.
Expert Judgements
Findings such as these raise the question of whether learners’ holistic difficulty judgements of L2 grammar points can converge and be consistent with expert judgements. This issue was addressed by Huang (Reference Huang2012), who compared applied linguistics researchers’ (n = 3) difficulty judgements based on the R&G taxonomy of learning difficulty with Taiwanese high school teachers’ (n = 20) and Taiwanese high school learners’ (n = 60) holistic difficulty judgements of 12 grammar points of L2 English. There was no significant correlation among the difficulty rankings of learners, teachers, and researchers, but the correlation between learners’ difficulty ranking and researchers’ ranking of explicit learning difficulty taken separately approached statistical significance. This seems to suggest that the learners may have focused on criteria similar to those of the researchers when making their judgements, especially the technicality of metalanguage and the conceptual complexity of pedagogical grammar rules. Qualitative findings from follow-up interviews with a subsample of the teacher participants indicated that teachers tended to differentiate between easy and difficult grammar points by drawing on their experience of students’ typical performance as well as the perceived distance of an L2 grammar point from the L1. Scrutiny of the descriptive statistics revealed that, compared with the teachers, learners showed overall less differentiation, with mean difficulty scores for all grammar points on the “easy” side of the scale (for a similar finding, see Absi, Reference Absi2014).
The potential ability of experts—whether teachers, L2 textbook authors, or L2 researchers—to offer a more differentiated and thus hopefully more accurate assessment of learning difficulty has induced researchers to rely to at least some extent on expert judgements as a predictor variable (Hu, Reference Hu2002; Robinson, Reference Robinson1996). Some empirical findings suggest that this may be a sound strategy. Thus Scheffler (Reference Scheffler2011) reports a very strong significant correlation between Polish secondary school teachers’ (n = 25) difficulty judgements of grammar points relating to the form and meaning of the L2 English verb phrase on a 5-point scale ranging from very easy to very difficult and their students’ (n = 50) performance on a metalinguistic test requiring the production of English sentences exemplifying the targeted pedagogical grammar rules. Along similar lines, Ziętek and Roehr (Reference Ziętek and Roehr2011) found that Polish college-level learners (n = 20) performed significantly worse on metalinguistic test items targeting six grammar points judged to be of higher explicit learning difficulty than on items targeting six grammar points judged to be of lower explicit learning difficulty based on the R&G taxonomy. The metalinguistic test required the correction of highlighted errors and the provision of associated pedagogical grammar rules. This finding was replicated in a later study using the same test with a different group of participants, that is, L1 Thai learners of English (Thepseenu & Roehr, Reference Thepseenu, Roehr, Roehr and Gánem-Gutiérrez2013). It is noteworthy that there were considerable differences between the samples not only in terms of L1 (Polish vs. Thai), but also in terms of language learning experience: the Thai learners had been exposed to English as their only L2, whereas the Polish learners had L2 English, L3 Italian, and in some cases a further European language as their L4.
Although the findings reported above suggest that expert judgements of learning difficulty may be reliable predictors of learners’ performance on measures of explicit knowledge, there is as yet no published research that has examined the relationship between expert judgements of learning difficulty as explicit and implicit knowledge, learners’ own perceptions of difficulty, and learners’ performance on measures of both explicit and implicit knowledge. The present study was aimed at addressing this gap.
METHODOLOGY
Research Questions
The study addressed the following research questions:
-
1. What is the relationship between applied linguists’, teachers’, and learners’ difficulty judgements of selected L2 English grammar points?
-
2. What is the relationship between perceived learning difficulty and learners’ actual performance on measures of implicit and explicit knowledge of these L2 English grammar points?
Participants
The study involved experts as well as L2 learners as participants. The L1 of all participants is Spanish (demographic information about the participants can be found in Online Appendix B). The experts included (a) applied linguists (including the first author of this article) with postgraduate-level qualifications (n = 3) and (b) university teachers of English (n = 11) at a higher-education institution in Mexico. The participating learners (n = 30) attended a general English course as part of the university’s so-called English Extension Program, which is open to students and faculty of the university as well as the general public. The only conditions for attending the course are a minimum age of 16 and completion of an L2 placement test aimed at allocating learners to the right level. The program comprises nine levels of proficiency, each with 80 hours of instruction over one semester. The learner group was recruited from Level 5 of this program.
The English program in which the participating teachers were involved and from which the participating learners were recruited is aimed at developing the four skills of speaking, listening, reading, and writing as well as grammar and vocabulary. Classes essentially follow the presentation – controlled practice – free production approach, providing practice in all skills, with an emphasis on communicative activities that reflect real-life language use: for example, ordering food in a restaurant, discussing cultural differences between Mexico and the United States, or researching and writing a short report about a chosen topic relevant to learners’ interests, such as top football teams or popular films. Both planned form-focused work in accordance with set textbook units and reactive focus-on-form activities following specific tasks are in regular use. Focus-on-form activities include the explicit presentation and discussion of pedagogical grammar rules followed by controlled exercises applying the rules. Classes are conducted primarily in L2 English, but L1 Spanish may be used in the context of form-focused activities in particular.
Instruments and Procedures
In addition to a short questionnaire aimed at eliciting biographical information, the instruments used in the present study were (a) difficulty judgement questionnaires for all participants and (b) tests of explicit and implicit L2 knowledge for the learners. These instruments are described in detail further below.
The instruments targeted 13 points of English grammar that were taken from the learners’ Level 5 textbook New American Inside Out: Intermediate Student Book (Kay & Jones, Reference Kay and Jones2009). For the purpose of the present study, each targeted grammar point was given a short label (e.g., simple past tense), following textbook conventions. In accordance with the distinction between (learning difficulty as) explicit and implicit knowledge reviewed previously, we formulated (a) a pedagogical grammar rule describing and explaining the grammar point, (b) an example sentence illustrating the grammar point, and (c) an example of a typical learner error with regard to the grammar point. In order to facilitate test design and subsequent scoring, all pedagogical grammar rules followed the same format: “When form X occurs/function X is being expressed, form Y needs to be used” (Roehr, Reference Roehr2008b; Thepseenu & Roehr, Reference Thepseenu, Roehr, Roehr and Gánem-Gutiérrez2013; Ziętek & Roehr, Reference Ziętek and Roehr2011). The metalinguistic terminology employed reflects the terminology the learners were exposed to. Grammar points that the learners had encountered in their classes and for which pedagogical grammar rules could be worded in the required format were selected. A further selection criterion was that items for our measure of implicit knowledge, an elicited imitation test (see subsequent discussion for details), could be constructed for each of the grammar points. The grammar points that fulfilled the selection criteria and were thus targeted in the present study were simple past tense, indefinite articles, simple present tense (third-person -s), comparative adjectives, second conditionals (if clauses), verb complements, many vs. much, modal verbs, yes/no questions, prepositions + verbs (-ing participle), since vs. for, dative alternation, and relative clauses. Footnote 2 The pedagogical grammar rules, example sentences, and example learner errors for each grammar point can be found in Online Appendix C.
Difficulty Judgement Questionnaires.
Difficulty judgement questionnaires were developed for the three groups of participants. The questionnaires presented the targeted grammar points as described previously and asked participants to judge their learning difficulty.
The applied linguists were asked to assess learning difficulty as implicit and explicit knowledge in accordance with the R&G taxonomy (for examples of the application of the taxonomy, see Online Appendix A). The variables frequency, perceptual salience, communicative redundancy, and opacity of form-meaning mapping apply to L2 constructions and are assumed to determine learning difficulty as implicit knowledge; schematicity applies to both L2 constructions and associated metalinguistic descriptions and thus refers to learning difficulty as both implicit and explicit knowledge; conceptual complexity, technicality of metalanguage, and truth value apply to metalinguistic descriptions and are thus assumed to determine learning difficulty as explicit knowledge. The applied linguists were asked to assign a value of high, medium, or low to each variable in the taxonomy for each of the 13 targeted grammar points.
Like the applied linguists, the participating teachers and learners were presented with the textbook label given to each grammar point (e.g., simple past tense), the associated pedagogical grammar rule, an example sentence illustrating the use of the linguistic structure, and an example of a typical learner error. They were then asked to indicate their opinion about the level of difficulty of each grammar point on a simple 5-point scale (very easy, easy, moderate, difficult, very difficult) (DeKeyser, Reference DeKeyser, Doughty and Long2003), based on their experience of teaching English (teachers) or learning English (learners). Instructions were provided in both English and Spanish.
Test of Explicit L2 Knowledge.
The test of explicit L2 knowledge for the learner group was a metalinguistic knowledge test consisting of two parts that comprised 35 items in total. Part 1 of the test was modeled on the instrument used by Ziętek and Roehr (Reference Ziętek and Roehr2011) and required learners to correct highlighted errors at sentence level and provide the underlying pedagogical grammar rule in either English or Spanish. Part 2 of the test presented the targeted pedagogical grammar rules in both English and Spanish and asked learners to write correct English sentences fully illustrating each rule (Absi, Reference Absi2014; Scheffler, Reference Scheffler2011).
Test of Implicit L2 Knowledge.
The test of implicit L2 knowledge was an elicited imitation test (R. Ellis, Reference Ellis2006; R. Ellis et al., Reference Ellis, Loewen, Elder, Erlam, Philp and Reinders2009; Erlam, Reference Erlam2006) comprising 78 sentences, with 3 grammatical and 3 ungrammatical sentences targeting each of the 13 grammar points, presented auditorily in a fixed, pseudorandom order. Two sentences for each grammar point were adapted directly from Erlam (Reference Erlam2006), whereas the remaining four were constructed to fit the cultural context and world knowledge of the learners. Participants were required to listen to a sentence, make a truth judgement on the basis of their world knowledge and beliefs by ticking true, not true, or not sure on an answer sheet, and then orally repeat in correct English the sentence they had heard.
In an elicited imitation test, the truth judgements serve as an intervening task aimed at preventing subvocal rehearsal of the stimulus sentence, so learners merely retain in memory its meaning but cannot retain its precise form. Accordingly, it is expected that the repeat sentence they produce represents their level of L2 knowledge. Given the time pressure created by the oral modality of the task and the online truth judgements that needed to be made, it was expected that learners would above all draw on any implicit knowledge they had to produce the repeat sentence (Erlam, Reference Erlam2006). A learner with full (implicit) knowledge of a targeted grammar point should reproduce grammatical sentences correctly as well as intuitively correct ungrammatical sentences. Example sentences from the test are shown as (1)–(4) below, with ungrammatical sentences marked by an asterisk and the targeted grammar point shown in parentheses.
-
(1) Zacatecas is a nicer place to visit than Leon. (Comparative adjectives)
-
(2) *The software that Bill Gates invented it changed the world. (Relative clauses)
-
(3) It is difficult to ask, “Do you really love me?” (Yes/no questions)
-
(4) *Not everyone can to learn a second language. (Modal verbs)
Instructions for the test were provided in English and Spanish. The test was presented to participants as a “beliefs questionnaire” in order to focus their attention on meaning and divert it from form. The learners began with practice items and proceeded to the test proper once they had understood what was required of them. The elicited sentences were audio recorded for subsequent analysis. Truth judgements were not scored but were monitored to ensure that participants had paid attention to and understood the meaning of the stimulus sentences.
Administration and Scoring
The learners completed the instruments under supervision in separate sessions in the following order: measure of implicit L2 knowledge (elicited imitation test), Part 1 of the test of explicit L2 knowledge (correcting errors and providing pedagogical grammar rules), Part 2 of the test of explicit knowledge (producing sentences illustrating given pedagogical grammar rules), and difficulty judgement questionnaire.
The test of implicit knowledge was scored dichotomously, with 1 point awarded for each correctly produced targeted L2 construction. Errors that did not pertain to the targeted grammar points were ignored. The maximum possible score was 89.
The test of explicit knowledge was likewise scored dichotomously. In Part 1, 1 point was awarded for each appropriate correction. In addition, 1 point was awarded for an appropriate metalinguistic explanation reflecting the “When form X occurs/function X is being expressed” clause of the targeted pedagogical grammar rule, and 1 point was awarded for an appropriate metalinguistic description reflecting the “form Y needs to be used” clause of the targeted pedagogical grammar rule. In Part 2, 1 point was awarded for each correctly produced sentence fully illustrating a given pedagogical grammar rule. Errors in the example sentences that did not pertain to the targeted grammar point were ignored. The maximum possible score for the test of explicit knowledge was 73.
The teachers completed the difficulty judgement questionnaire in their own time. Teachers’ and learners’ difficulty judgements were subsequently converted into numerical scores ranging from 1 (very easy) to 5 (very difficult).
The difficulty judgement questionnaire for the applied linguists was completed by the first author and subsequently by the other two applied linguists. When required, the first author provided clarification by offering prompts (adapted from Huang, Reference Huang2012) supplementing the information given in the R&G taxonomy: For example, in order to clarify the notion of communicative redundancy, the prompt “Can the speaker still make themselves understood even if they do not get the form right?” was offered. The three applied linguists discussed their difficulty judgements until consensus was reached for each variable on each grammar point. Subsequently, the qualitative difficulty judgements of high, medium, or low were converted into scores, with favorable values (i.e., values indicating decreased learning difficulty) scored as 1, neutral values scored as 2, and unfavorable values (i.e., values indicating increased learning difficulty) scored as 3. For example, high frequency will decrease learning difficulty, so the score would be 1; by contrast, high communicative redundancy will increase learning difficulty, so the score would be 3. Scores for learning difficulty as implicit knowledge, learning difficulty as explicit knowledge, and overall learning difficulty were calculated for each of the 13 targeted grammar points. According to the taxonomy used, the criterion of schematicity applies to both L2 constructions and associated metalinguistic descriptions and thus refers to learning difficulty as both implicit and explicit knowledge. It thus contributed to both the implicit and the explicit learning difficulty scores where these are shown separately, but it was counted only once (based on the mean of the implicit and explicit score) when assessing overall learning difficulty.
All instruments were piloted prior to use and were amended as needed. The internal consistency of the finalized tests as used in the present study was good (elicited imitation test: Cronbach’s alpha = .90; metalinguistic knowledge test: Cronbach’s alpha = .81). A one-sample Kolmogorov-Smirnov test revealed that the distribution of scores did not differ significantly from a normal distribution either on the elicited imitation test (p = .10) or on the metalinguistic knowledge test (p = .13) and its subtests (p = .48 for correction, p = .34 for description/explanation, and p = .64 for rule illustration).
RESULTS
The first research question asked about the relationship between the applied linguists’, teachers’, and learners’ difficulty judgements of selected L2 English grammar points. Table 1 displays the difficulty of the targeted grammar points as judged by the three participant groups. Recall that the lower the score, the easier the grammar point is perceived to be. Learners’ and teachers’ scores were provided holistically on a scale of 1 (very easy) to 5 (very difficult) and are directly comparable. As the applied linguists used the judgement criteria given in the R&G taxonomy, their scores are only comparable with the scores of the other two groups in terms of rank order.
Perceived learning difficulty of the targeted grammar points
Note. Recall that the criterion of schematicity contributed to both implicit and explicit learning difficulty scores but was counted only once for overall learning difficulty, so the applied linguists’ overall score is not the exact sum of their implicit and explicit scores.
The difficulty judgements displayed in Table 1 indicate that the learners tended to judge the targeted grammar points to be less difficult overall than the teachers. Only one of the mean scores approaches 3 (moderate), whereas the majority are below 2.5 and thus in the “easy” range. By contrast, the teachers have only a single score that is below 2.5 and thus in the “easy” range.
In order to ascertain whether there is any statistical relationship among the difficulty judgements made by the participant groups, Spearman rank order correlations were run. The results are displayed in Table 2.
Correlations (Spearman’s rho): Learners’, teachers’, and applied linguists’ difficulty judgements
** p < .01.
* p < .05.
The results in Table 2 confirm that learners’ and teachers’ difficulty judgements are significantly correlated. Moreover, teachers’ judgements are correlated with the applied linguists’ judgements of overall learning difficulty. This relationship seems to be driven by the correlation between teachers’ judgements and the applied linguists’ judgements of explicit learning difficulty. The applied linguists’ judgements are not significantly associated with learners’ judgements, though there is a trend toward a relationship for overall learning difficulty (p = .08).
The second research question asked about the relationship between perceived learning difficulty and learners’ actual performance on measures of implicit and explicit knowledge of the targeted grammar points. As a first step toward addressing this question, learners’ performance on the elicited imitation test (implicit L2 knowledge) and the metalinguistic knowledge test (explicit L2 knowledge) was scrutinized. Table 3 shows the descriptive statistics for the two measures.
Descriptive statistics: Tests of implicit and explicit L2 knowledge
Note. EI = elicited imitation; MLK = metalinguistic knowledge.
The results indicate that both tests were sufficiently challenging for the participants. Indeed, the elicited imitation test proved to be rather difficult, with a mean facility value of just 40%. The metalinguistic knowledge test was somewhat easier overall. This was due primarily to the error-correction task and, to a lesser extent, to the rule-illustration task; by contrast, the description/explanation task was challenging for the learners. Table 4 shows the correlations between scores on the two measures.
Correlations (Pearson’s r): Tests of implicit and explicit L2 knowledge
Note. EI = elicited imitation; MLK = metalinguistic knowledge.
** p < .01.
* p < .05.
As expected, the subtests of the measure of explicit L2 knowledge correlate with one another at a medium level of strength. Scores on the metalinguistic knowledge test as a whole and the elicited imitation test are likewise associated, indicating that learners’ explicit and implicit knowledge are related if these types of knowledge are assessed by means of measures targeting a range of L2 structures.
Table 5 displays the results from an analysis by targeted grammar point, thus showing the relationship between learners’ performance on measures of implicit and explicit knowledge of the 13 selected L2 English grammar points, on the one hand, and the perceived learning difficulty of these grammar points as judged by the participating learners themselves, their teachers, and the applied linguists using the R&G taxonomy.
Correlations (Spearman’s rho): Perceived learning difficulty and actual performance
* p < .05.
Although all relationships are negative, thus reflecting the expected association of higher perceived difficulty with lower scores on performance measures, there is only a single significant correlation in evidence: that is, between performance on the test of explicit L2 knowledge and the learners’ own difficulty judgements. This indicates that the learners themselves were the only participant group to make a reliable judgement, in this case for their performance on the metalinguistic knowledge test. It is worth noting, however, that three further correlations approach significance. The applied linguists’ overall learning difficulty judgements based on the R&G taxonomy marginally predict performance on the test of explicit L2 knowledge (p = .05). Most interestingly perhaps, the teachers’ difficulty judgements show a trend toward significance for both the test of implicit and the test of explicit L2 knowledge. Conversely, neither the learners themselves nor the applied linguists are anywhere near predicting performance on the elicited imitation test.
Finally, there is no relationship between performance on the measures of implicit and explicit knowledge in the analysis by grammar point (rho = .09, p = .76; not shown in Table 5), indicating that learners appear to have developed explicit knowledge of certain grammar points and implicit knowledge of others, and vice versa, but not necessarily both implicit and explicit knowledge of the same grammar point.
DISCUSSION
It is now possible to draw together the strands of the analysis and consider the results in terms of (a) learners’ actual performance on implicit and explicit measures of the targeted L2 English grammar points, (b) the difficulty judgements about the targeted grammar points made by the various participant groups, and (c) the relationship between perceived difficulty and actual performance.
Actual Performance
The first finding of note with regard to learners’ performance is that the elicited imitation test (mean facility value of 40%) was more challenging for the participants than the metalinguistic knowledge test (mean facility value of 57%). A possible explanation for this pattern of results is that the form-focused dimension of the instruction the learners are exposed to has had an impact on the nature of their proficiency. As explicit learning is relatively fast, (some aspects of) explicit knowledge may be acquired somewhat earlier and/or relatively more successfully in the limited-input environment of the L2 classroom, whereas the acquisition of implicit knowledge may lag behind in comparison, even if it is assumed that explicit knowledge can indirectly facilitate implicit learning, as is suggested by the weak-interface position, currently perhaps the most widely held view on this subject (Dörnyei, Reference Dörnyei2009; N. C. Ellis, Reference Ellis, Sanz and Leow2011).
Having said this, the results also show that tasks that required the application of pedagogical grammar rules (correction or rule illustration) were performed with greater ease than tasks requiring the retrieval and/or formulation of pedagogical grammar rules (description/explanation). This is in keeping with greater comfort and/or greater experience on the part of the learners with a deductive use of metalinguistic propositions, as opposed to an inductive approach requiring the identification of regularities in linguistic exemplars and the articulation of such regularities by means of rule-like statements (Norris & Ortega, Reference Norris and Ortega2001).
A second finding pertains to the relationship between learners’ implicit and explicit L2 knowledge. Overall scores on the elicited imitation test and the metalinguistic knowledge test were positively and significantly correlated. At the same time, an analysis by targeted grammar point found no significant association between implicit and explicit knowledge. Although this may seem contradictory at first glance, it is in fact consistent with existing research. Instructed learners’ implicit and explicit knowledge have been found to correlate if these types of knowledge are assessed at a global level, that is, by using measures testing a range of L2 constructions (Absi, Reference Absi2014; R. Ellis, Reference Ellis2005). This association suggests that instructed L2 learners develop both implicit and explicit knowledge to at least some extent, a circumstance that is entirely expected. However, such a correlation cannot reveal the precise nature of the interplay between implicit and explicit knowledge. The absence of a correlation of implicit and explicit knowledge in an analysis by targeted grammar point in the present study may offer a clue: It is possible that for a particular L2 construction, learners develop one type of knowledge first and subsequently construct the other type of knowledge, rather than acquiring both types of knowledge together. This finding corresponds with conclusions drawn in a larger-scale study targeting 17 grammatical features of L2 English (R. Ellis, Reference Ellis2006), which reports that although both implicit and explicit knowledge appeared to be implicated in the participants’ L2 proficiency, it seemed to be the case that implicit and explicit knowledge of different rather than the same structures was involved.
Perceived Learning Difficulty
A comparison of the difficulty judgements made about the targeted L2 grammar points by the different participant groups revealed that the learners tended to regard the targeted grammar points to be generally less difficult than did the teachers, with almost all mean difficulty scores in the “easy” range. Similar patterns of results occurred in previous research, with mean learner judgements typically exhibiting (a) less differentiation than teacher judgements and (b) a noticeable skew toward the “easy” side of the scale. This apparent display of confidence on the part of L2 learners of English has been observed in different cultural and educational contexts—that is, in high school students in Taiwan (Huang, Reference Huang2012), in university students in Syria (Absi, Reference Absi2014), in university students in Thailand (Thepseenu & Roehr, Reference Thepseenu, Roehr, Roehr and Gánem-Gutiérrez2013), and, in the present study, in university-level learners in Mexico. It is not immediately obvious why learners seem to consider grammar points as comparatively easy (and as easier than their teachers consider them to be). A certain social desirability bias may be in evidence here, with students perhaps feeling that they are expected to be able to cope with the learning task of mastering L2 structures. This feeling may then be reflected in their reported judgements. Without empirical evidence for the rationale underlying learners’ difficulty judgements, this explanation must remain speculative, however.
A second finding of interest refers to the relationship among the difficulty judgements made by the different participant groups. In the present study, the rank orders of learners’ and teachers’ difficulty judgements were found to be significantly correlated. As the learners and teachers operate in the same educational setting and may thus have used similar judgement criteria, this is perhaps not surprising. However, the result differs from Huang’s (Reference Huang2012) study, which reported no significant correlation between the difficulty rankings of learners and teachers. A possible reason for this difference in outcome is that the Taiwanese learners studied by Huang (Reference Huang2012) were adolescents and thus their perceptions may have differed from those of their teachers to a greater extent than the adult learners in the present study, given that they had comparatively less L2 learning experience than their adult counterparts.
In the present study, the teachers’ difficulty rankings correlated with the applied linguists’ judgements of overall learning difficulty based on the R&G taxonomy, a relationship that was driven by the stronger correlation between teachers’ difficulty judgements and the applied linguists’ judgements of explicit learning difficulty. Taken on its own, this result seems to suggest that teachers may have had above all the explicit dimension of L2 learning in mind when making their judgements, that is, the relative learning difficulty of pedagogical grammar rules may have been their primary judgement criterion.
Difficulty Judgements Predicting Performance
Although the preceding argument seems plausible, it is not fully supported by the findings regarding the relationship between difficulty judgements and actual performance. Although the teachers’ judgements did not correlate significantly with learners’ performance, the teachers were in fact the only participant group whose judgements showed a trend toward prediction on both L2 performance measures: that is, the metalinguistic knowledge test and also the elicited imitation test. By contrast, neither the learners themselves nor the applied linguists using the R&G taxonomy even remotely predicted performance on the test of implicit L2 knowledge. Thus it appears that the teachers may have been the best judges, in the sense that they were the only judges whose difficulty rankings showed a trend toward capturing the difficulty of the targeted grammar points in terms of both implicit and explicit knowledge. The teachers’ relative predictive success in the present study is broadly in keeping with Scheffler’s (Reference Scheffler2011) finding that the teachers he worked with were able to accurately predict their learners’ performance on a rule-illustration task.
In the present study, the learners themselves were the most reliable judges of explicit learning difficulty, with their difficulty rankings significantly correlating with their actual performance on the metalinguistic knowledge test. A possible task sequencing effect may help explain this finding. As the learners made their difficulty judgements following completion of the metalinguistic knowledge test, they may have focused primarily on explicit knowledge when considering the question of learning difficulty, even though the test and the judgements were carried out in separate sessions, rather than immediately following each other. Thus, whereas the teachers (and the applied linguists, for that matter) had to draw on their general experience, the learners may have drawn above all on a single, specific experience, namely their memory of the recent encounter with the metalinguistic knowledge test. Even if this is the case, it is worth noting that the learners in the present study arguably exhibited a high level of metacognitive awareness, given that their judgements closely reflected their actual performance on the test. Needless to say, they had not received any feedback on their test performance, so they had to rely on self-evaluation.
The applied linguists’ overall learning difficulty rankings based on the R&G taxonomy marginally predicted performance on the metalinguistic knowledge test, though the correlation did not quite reach statistical significance. Somewhat counterintuitively, the applied linguists’ rankings for learning difficulty of explicit knowledge did not show the same trend. Indeed, the applied linguists’ rankings failed to provide the reliable predictions one might have expected from the more differentiated judgement criteria that were available to them. Thus, the R&G taxonomy of learning difficulty seemingly did not prove useful in predicting learners’ implicit or explicit knowledge of the targeted L2 grammar points. This differs somewhat from previous studies in which the same taxonomy was employed successfully to differentiate between higher and lower explicit learning difficulty (Thepseenu & Roehr, Reference Thepseenu, Roehr, Roehr and Gánem-Gutiérrez2013; Ziętek & Roehr, Reference Ziętek and Roehr2011). It is not possible to conclude from the available evidence whether the proposed criteria for determining learning difficulty themselves were inadequate for the more detailed predictions of rank orders of learning difficulty and performance on measures of both implicit and explicit knowledge as required in the present study, or whether the small group of participating applied linguists (n = 3) was unable to use the criteria accurately for these purposes.
CONCLUSION
The present study investigated the relationship between the perceived learning difficulty of 13 L2 English grammar points and instructed adult learners’ actual performance on measures of implicit and explicit knowledge of these grammar points. It was found that learners’ own holistic difficulty rankings correlated significantly with their performance on the measure of explicit knowledge. Although correlations based on teachers’ holistic difficulty rankings did not reach statistical significance, the judgements of this group were the only ones that showed trends toward successful prediction of learners’ performance on both the implicit and the explicit L2 measure. Learning difficulty judgements made by a small group of applied linguists based on the theoretically informed R&G taxonomy comprising detailed judgement criteria did not lead to statistically significant predictions, although the correlation of overall learning difficulty and learner performance on the measure of explicit L2 knowledge approached significance.
Taken together, the findings seem to indicate that the learners may have been concerned above all with explicit knowledge and learning when considering the issue of learning difficulty, whereas, by contrast, the teachers exhibited a trend toward the best overall prediction. This trend supports the argument that teachers drawing on their long-term experience in a specific instructional setting may be able to identify with some precision the particular challenges their learners face when it comes to developing proficiency in certain L2 structures in terms of both implicit and explicit knowledge. Teachers’ intuitions may thus be valuable not only to learners in an educational setting but also to researchers investigating instructed SLA.
A number of limitations to the present study need to be acknowledged. As is the case so often in applied linguistics research, the findings arising from our study have limited generalizability, because they have arisen from specific samples of learners, teachers, and applied linguists living and working in a specific cultural context.
Ideally, more than one measure of implicit and explicit knowledge, respectively, would have been employed. Although every effort was made to develop valid and reliable measures, it is notoriously difficult to ensure that a test allows for the use of only one type of knowledge at the exclusion of the other, rather than a combination of implicit and explicit knowledge. Arguably, it is impossible to achieve “pure” measurement with 100% certainty, especially with regard to implicit L2 knowledge. This difficulty notwithstanding, a battery of tests would permit the assessment of divergent and convergent validity, for example, and thus offer potentially greater validity and reliability of measurement.
In addition, it would have been worthwhile to gauge individual learner variables that are known to interact with explicit L2 learning in particular, such as working memory capacity and language learning aptitude. A larger and thus more representative sample of applied linguists as judges would also have been of benefit. Moreover, it would be of interest to conduct research on L2s other than English. Although comparability of findings is desirable, different L1-L2 combinations need to be examined if truly generalizable results are to be obtained.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S0272263115000340.
