Introduction
The field of additional or second language (L2) learning is seeing renewed interest in the role of cognitive ability factors such as language learning aptitude and working memory. While numerous studies have affirmed the predictive power of aptitude in general terms (Li, Reference Li2015, Reference Li2016; Pavlekovic & Roehr-Brackin, Reference Pavlekovic and Roehr-Brackin2024), evidence concerning the specific learning stage(s) at which different aptitude components predict achievement is not unequivocal. Some studies show that aptitude is a significant predictor in the early stages of L2 learning (Artieda & Muñoz, Reference Artieda and Muñoz2016), whereas others suggest that it is important at more advanced levels (Linck et al., Reference Linck, Hughes, Campbell, Silbert, Tare, Jackson and Doughty2013).
Recent theoretical developments have led to a distinction between aptitude for explicit and aptitude for implicit learning, with a number of studies supporting this conceptual contrast (Granena, Reference Granena2012, Reference Granena, Granena and Long2013a; Pavlekovic & Roehr-Brackin, Reference Pavlekovic and Roehr-Brackin2024; Roehr-Brackin, Loaiza, & Pavlekovic, Reference Roehr-Brackin, Loaiza and Pavlekovic2023; Roehr-Brackin & Pavlekovic, Reference Roehr-Brackin and Pavlekovic2025; Suzuki & DeKeyser, Reference Suzuki and DeKeyser2017). However, it remains unclear to what extent explicit and/or implicit aptitude facilitate L2 achievement differently at different proficiency levels. This is because the majority of existing studies on aptitude only included measures of explicit aptitude (Li, Reference Li2015) or because aptitude was operationalized as a unidimensional construct (overall aptitude scores in Li, Reference Li2015), despite evidence pointing toward a multicomponent nature of both explicit and implicit aptitude. In addition, researchers have posited a predictive role for working memory, but again, evidence for its influence at different learning stages is contradictory (Kormos & Sáfár, Reference Kormos and Sáfár2008; Linck et al., Reference Linck, Hughes, Campbell, Silbert, Tare, Jackson and Doughty2013; Serafini & Sanz, Reference Serafini and Sanz2016).
Beyond conceptual and methodological differences between empirical studies, there is a tendency to rely on statistical techniques which treat the participant sample as homogeneous and proficiency as a categorical variable, even though it is measured on a continuum. As a consequence, no study to date has been able to pinpoint the threshold at which the role of proficiency in the aptitude–outcome equation moderates a shift from facilitative to neutral, or vice versa.
The present study addresses these issues by investigating the effects of explicit and implicit aptitude and working memory on L2 achievement in terms of explicit, automatized explicit, and implicit knowledge of selected morphosyntactic structures. Not only do we examine the moderating role of proficiency in this relationship, but we also treat proficiency as a continuous variable. Employing a multilevel modeling approach, we aimed to identify the tipping point at which different aptitude components lose or gain their facilitative influence on L2 outcomes.
Background
Language learning aptitude
Language learning aptitude encompasses cognitive and perceptual abilities that facilitate the rapid and effortless learning of new languages (Carroll, Reference Carroll and Diller1981). The classic model of aptitude (Carroll, Reference Carroll and Diller1981) comprises phonetic coding ability, associative memory, and language-analytic ability (Skehan, Reference Skehan1998). This model is the result of a componential approach and was developed a posteriori on the basis of empirical results. In current research, these abilities are conceptualized as components of aptitude for explicit learning (hereafter: explicit aptitude). Explicit aptitude positively predicts learning outcomes (Li, Reference Li2015, Reference Li2016), that is, higher levels of aptitude are advantageous for L2 learning.
In addition to explicit aptitude, current research has posited the complementary notion of aptitude for implicit learning (hereafter: implicit aptitude) (Li & DeKeyser, Reference Li and DeKeyser2021). It has been suggested that implicit aptitude comprises implicit learning ability and implicit learning memory (Granena, Reference Granena2020). The cognitive abilities involved are sensitivity to frequency and conditional probability, priming or the tendency to be influenced by recent events, and selective attention (Li & DeKeyser, Reference Li and DeKeyser2021). Similar to explicit aptitude, implicit aptitude is thus componential in nature. Factor analyses have yielded varied results, however. Specifically, implicit aptitude measures such as sequence learning ability and auditory pattern recognition have been found to both load positively onto the same factor (Granena, Reference Granena2012; Roehr-Brackin et al., Reference Roehr-Brackin, Loaiza and Pavlekovic2023; Roehr-Brackin & Pavlekovic, Reference Roehr-Brackin and Pavlekovic2025), or one measure has been found to load positively and the other negatively onto the same factor (Iizuka & DeKeyser, Reference Iizuka and DeKeyser2023; Pavlekovic & Roehr-Brackin, Reference Pavlekovic and Roehr-Brackin2024).
Theoretical models of aptitude informed by an information-processing paradigm have focused on explicit aptitude, although the role of implicit aptitude is acknowledged in more recent work. The staged model of aptitude (Skehan, Reference Skehan and Robinson2002, Reference Skehan, Gass and Mackey2012) contends that different aptitude components hold varying degrees of importance at different learning stages. For instance, phonetic coding or “handling sound” (Skehan, Reference Skehan, Wen, Skehan, Biedrón, Li and Sparks2019, p. 61) is particularly relevant at the initial stage. Language-analytic ability becomes crucial in the subsequent “handling pattern” stage, where learners identify and generalize linguistic patterns. Automatization, on the other hand, is most relevant in the final stage, “automatising-proceduralizing” (Skehan, Reference Skehan, Wen, Skehan, Biedrón, Li and Sparks2019, p. 61). Both Skehan (Reference Skehan, Granena, Jackson and Yilmaz2016) and Li (Reference Li, Li, Hiver and Papi2022) propose that the automatization process is heavily reliant on implicit learning abilities. Taken together, this would imply that cognitive abilities facilitating explicit learning are more relevant at the initial stages of L2 acquisition, whereas cognitive abilities underlying implicit learning become more important at advanced stages. Working memory is deemed important from the initial stages onward, with retrieval memory taking over in later stages (Skehan, Reference Skehan, Gass and Mackey2012). Along similar lines, the model of aptitude complexes proposed by Robinson (Reference Robinson2005, Reference Robinson and Pawlak2012) posits that different clusters of cognitive abilities are engaged in different learning and instructional contexts.
Working memory
The aptitude models reviewed in the preceding section acknowledge the role of working memory (WM) as an individual difference factor predicting L2 outcomes (Robinson, Reference Robinson2005, Reference Robinson and Pawlak2012; Skehan, Reference Skehan and Robinson2002, Reference Skehan, Granena, Jackson and Yilmaz2016). WM refers to the ability to concurrently store and process information during cognitive tasks (Baddeley, Reference Baddeley1986; Cowan, Reference Cowan2005). Baddeley’s multicomponent model of WM has been influential in L2 learning research (Baddeley, Reference Baddeley, Wen, Mota and McNeill2015, Reference Baddeley2017; Wen, Reference Wen, Wen, Skehan, Biedrón, Li and Sparks2019), which has particularly focused on the components of phonological working memory (PWM) and executive working memory (EWM). PWM is responsible for the short-term storage of phonological information and articulatory rehearsal, while processes such as inhibition, updating, and switching are part of EWM (Wen, Reference Wen, Wen, Skehan, Biedrón, Li and Sparks2019). The importance of PWM and EWM in the processing, learning, and use of an L2 is widely accepted (Juffs & Harrington, Reference Juffs and Harrington2011; Linck, Osthus, Koeth, & Bunting, Reference Linck, Osthus, Koeth and Bunting2014). Research tends to support a “the-more-the-better” hypothesis, suggesting that learners with higher WM will outperform those with lower WM (Miyake & Friedman, Reference Miyake, Friedman, Healy and Bourne1998). However, conflicting findings exist, with some studies showing no significant role of WM (Juffs, Reference Juffs2004, Reference Juffs2005). Discrepancies in findings can arise from different WM components being investigated, differences in measurement techniques, and factors such as characteristics of the outcome measures used (Serafini & Sanz, Reference Serafini and Sanz2016). Therefore, the precise role of (components of) WM at different learning stages remains unclear.
Evidence for the role of aptitude and working memory at different levels of L2 proficiency
Studies investigating the effects of aptitude and WM have found that facilitative effects depend on learners’ overall L2 proficiency, with recent research suggesting that explicit aptitude is more crucial at initial stages of L2 learning, whereas implicit aptitude plays a greater role at advanced stages (Li & DeKeyser, Reference Li and DeKeyser2021).
Linck et al. (Reference Linck, Hughes, Campbell, Silbert, Tare, Jackson and Doughty2013) used the Hi-LAB battery to investigate the role of cognitive and perceptual abilities at advanced proficiency levels. They categorized participants into mixed-attainment and high-attainment groups based on their scores on the Defense Language Proficiency Test or by the demands of their occupations. The findings revealed that implicit aptitude as measured by a serial reaction time task, associative memory as assessed by a paired associates test, and PWM as measured by a letter span task were all positively correlated with high language attainment. Surprisingly, the researchers also noted a negative correlation between switching ability, a component of EWM measured by a task-switching test, and high attainment. They attributed this to more pronounced switching between L2 and L1 and consequently greater reliance on L1, which in turn would hinder high attainment in the L2 (Linck et al., Reference Linck, Hughes, Campbell, Silbert, Tare, Jackson and Doughty2013). Together, this suggests that both explicit and implicit aptitude as well as WM play predictive roles in language success at advanced proficiency levels, although some components—such as an EWM measure in this case—may emerge as negative predictors.
Artieda and Muñoz (Reference Artieda and Muñoz2016) included beginner- and intermediate-level learners of L2 English in their study to examine the relevance of aptitude as assessed by the LLAMA battery (Meara, Reference Meara2005; Meara & Rogers, Reference Meara and Rogers2019). English proficiency was determined using end-of-year school L2 proficiency tests assessing language use in terms of reading, listening, writing, and speaking. Results revealed that when treated as a unidimensional construct, aptitude exhibited a consistent medium effect size at both beginner and intermediate levels of proficiency. When subtest scores were analyzed separately, auditory pattern recognition had a significant impact at the beginner level only. Conversely, language-analytic ability was relevant at both the beginner and intermediate levels, albeit more so at the intermediate level. Associative memory did not play a role at either proficiency level.
A meta-analysis of 66 aptitude studies (Li, Reference Li2016) demonstrated that different aptitude components predicted various L2 skills, including listening, reading, speaking, grammar, and overall L2 knowledge. Phonetic coding ability emerged as a strong predictor of overall proficiency, particularly in beginner learners, pointing toward its greater relevance at initial stages of L2 learning. Language-analytic ability strongly predicted grammar learning and reading comprehension, while associative memory showed a weak relationship with L2 proficiency. Moreover, the meta-analysis revealed that explicit aptitude was more predictive of L2 achievement in high-school learners than in university students. Considering that high-school participants typically exhibited lower proficiency compared with university students, this implies that explicit aptitude is more relevant for beginners, consistent with Carroll’s comment that classic aptitude pertains to learning a language “from scratch” (Carroll, Reference Carroll, Parry and Stansfield1990, p. 24).
Working with learners at an advanced proficiency level—defined as having achieved N1 on the standardized Japanese Language Proficiency Test (where N5 represents the lowest and N1 the highest level on the scale)—Suzuki and DeKeyser (Reference Suzuki and DeKeyser2017) examined the predictive relationships of auditory pattern recognition, language-analytic ability, and PWM with specific types of L2 knowledge, that is, implicit and automatized explicit knowledge, each operationalized via three different tests targeting structures of Japanese. The findings provide evidence for a significant relationship between language-analytic ability and automatized explicit knowledge.
Some researchers specifically examined the role of WM at different learning stages. In a study investigating the impact of WM at three proficiency levels (Serafini & Sanz, Reference Serafini and Sanz2016), beginner, intermediate, and advanced learners were assessed in terms of their L2 Spanish morphosyntactic development. Results revealed that PWM as measured by a digit span task was a significant predictor of performance on an oral elicited imitation task in the beginner and intermediate groups. However, no such relationship was observed in advanced learners. Similarly, on an untimed grammaticality judgment task, PWM showed a significant relationship for intermediate learners only. EWM as assessed by an operation span task did not emerge as a significant predictor in a regression analysis, but it showed a weak to moderate positive correlation with the elicited imitation task in a simple bivariate analysis for beginner and intermediate learners.
Kormos and Sáfár (Reference Kormos and Sáfár2008) explored the relationship between WM and various L2 English skills, including reading, listening, speaking, writing, use of English, and overall L2 proficiency, among beginner- and intermediate-level learners. Results showed that PWM as measured by a nonword span test was significantly related to overall proficiency, use of English proficiency scores, and a composition task in the intermediate group. Conversely, EWM as measured by a backward digit span task predicted overall proficiency scores and achievement in L2 reading, listening, and speaking in the beginner group.
Taken together, the empirical studies reviewed in the preceding paragraphs point toward a moderating role for L2 proficiency in the effects of aptitude and working memory on L2 outcomes. At the same time, findings are not necessarily convergent. Contrasting results can be due to the use of different measures of aptitude and WM. However, they may also be attributable to the conceptualization of proficiency in terms of two or three broad categories. Furthermore, L2 achievement has been operationalized in various ways. As current research posits explicit and implicit aptitude, it would arguably be theoretically most coherent to conceptualize L2 outcomes with reference to the complementary notions of explicit and implicit knowledge. While aptitude is not assumed to directly determine knowledge type, it is seen as influencing the nature and efficiency of the learning processes that ultimately give rise to different types of L2 knowledge.
Explicit and implicit knowledge and learning
Implicit learning pertains to the acquisition of implicit or unconscious knowledge without the intention to learn or awareness of the knowledge acquired. By contrast, explicit learning involves the acquisition of conscious or declarative knowledge under intentional learning conditions (Rebuschat, Reference Rebuschat2013). Implicit knowledge is characterized as tacit, intuitive, and beyond conscious introspection; it typically involves rapid access (Hulstijn, Reference Hulstijn and Rebuschat2015; Rebuschat, Reference Rebuschat2013). Explicit knowledge is conscious, available for introspection, and potentially verbalizable (N. Ellis, Reference Ellis and Rebuschat2015; Rebuschat, Reference Rebuschat2013). In the present context, learning is conceptualized as the mechanism through which input is processed and mental representations are formed; the resulting knowledge is thus the outcome of the learning process.
Tasks that limit conscious reflection and require language processing in real time are hypothesized to access implicit knowledge (Suzuki & DeKeyser, Reference Suzuki and DeKeyser2015; Vafaee, Suzuki, & Kachinske, Reference Vafaee, Suzuki and Kachinske2017). Self-paced reading and word monitoring tasks are examples of such measures. Sensitivity to grammatical errors as measured by reaction times to correct vs. incorrect stimuli is considered an indicator of implicit knowledge of the targeted morphosyntax.Footnote 1
Some studies (R. Ellis, Reference Ellis2005; Godfroid & Kim, Reference Godfroid and Kim2021) have supported elicited imitation as a measure of implicit knowledge. By contrast, Suzuki and DeKeyser (Reference Suzuki and DeKeyser2015) argue that learners dynamically utilize both implicit knowledge and explicit knowledge that has been automatized during elicited imitation. Therefore, they propose the notion of automatized explicit knowledge—a type of knowledge that is similar to implicit knowledge in that it involves fast access to linguistic representations, which in turn correlate with awareness. Essentially, elicited imitation performance is indicative of automatization because learners are asked to repeat sentences correctly under time pressure (Suzuki & DeKeyser, Reference Suzuki and DeKeyser2015). This view is supported by recent neurolinguistic evidence, especially with regard to the production (imitation) phase of the task (Suzuki et al., Reference Suzuki, Jeong, Cui, Okamoto, Kawashima and Sugiura2023).
Explicit knowledge is typically gauged via accuracy-focused tasks which allow for conscious reflection. Examples are gap-fill tests in which participants select a correct answer from a number of given options and tests of metalinguistic knowledge (R. Ellis, Reference Ellis2005) in which participants are required to supply both a correct answer and the underlying metalinguistic rule.
The current study
In summary, existing research has shown that explicit aptitude is important for L2 learning at lower levels of proficiency, whereas its significance at later stages is less clear. Conversely, it has been proposed that implicit aptitude plays a facilitative role at advanced levels, though empirical evidence is still in short supply. The role of WM at different proficiency levels is unclear. PWM may be particularly important at initial stages, while results pertaining to EWM are contradictory, with some studies indicating its significance for beginners and others for advanced learners.
To date, studies investigating aptitude and WM at different learning stages have treated proficiency as a categorical variable, that is, participants are classified as “beginners,” “intermediate,” or “advanced,” with a range of proficiency scores included in each category (Artieda & Muñoz, Reference Artieda and Muñoz2016; Serafini & Sanz, Reference Serafini and Sanz2016). While this is certainly a possible approach, it does not allow us to determine at what point on the proficiency continuum specific cognitive abilities become, or cease to be, relevant—an issue which is of critical importance both for a theoretically driven research agenda and for practical purposes. Accordingly, the present study addressed the following research questions:
RQ1: To what extent are the effects of explicit and implicit aptitude and working memory on learners’ knowledge of selected L2 morphosyntactic structures moderated by L2 proficiency?
RQ2: At which point on the L2 proficiency continuum do the facilitative effects of explicit and implicit aptitude and working memory become or cease to be significant?
Methodology
The current study had a correlational design with online administration of all measures. Explicit and implicit aptitude were assessed by means of the LLAMA test suite and a probabilistic serial reaction time task. Working memory was measured via forward digit and operation span tasks. The L2 outcome measures focused on three morphosyntactic structures of English assessed by means of a self-paced reading task to capture implicit knowledge, an elicited imitation test to capture automatized explicit knowledge, and a gap-fill test to capture explicit knowledge. L2 proficiency was evaluated by means of a monologic oral production task which was analyzed in terms of complexity, accuracy, and fluency. In the subsequent sections, the study participants, instruments, and procedure are described in full.Footnote 2
Participants
The study involved 83 L1 Croatian learners of English who had been exposed to L2 instruction for between 6 and 13 years (M = 10, SD = 1.69) as part of their compulsory secondary-school education. The sample consisted of 59 female and 22 male participants as well as 2 participants who opted not to disclose their gender. The learners’ ages ranged from 15 to 18 years (M = 16.16, SD = 1.28). All participants had been studying English as a compulsory subject, receiving 3 hr of instruction per week since the beginning of secondary school. Classes focus on communicative language skills but also include form-focused instruction to prepare students for the nationwide exam at the end of their final year. A total of 68 participants (81.9%) had never visited an English-speaking country, while 2 participants (2.4%) had stayed abroad for a year. The remaining 13 participants (15.7%) reported short stays averaging 25 days.
Target structures
The L2 outcome measures targeted three morphosyntactic structures: past simple tense, passive voice, and articles. These structures feature prominently in the participants’ grammar syllabus and are the source of frequent errors, as identified by the learners’ English teacher (D. Linić Učur, personal communication, 6 July 2020). They were therefore deemed suitable for achieving maximum discrimination between participants. For a detailed description of the structures and L1–L2 comparisons, see Appendix A in the online supplementary materials. Table 1 provides examples of each structure and the corresponding regions of interest (ROIs) in the self-paced reading task (for further details, see below).
Ungrammatical sentences for the three target structures with ROIs in the self-paced reading task
Table 1. Long description
The table consists of seven columns and four rows including the header. The header row identifies the Structure, an introductory phrase, R O I 1, R O I 2, R O I 3, and two trailing columns for sentence completion.
* Row 1 (Past): Structure is Past. Introductory phrase is ‘When I was a child, I’. R O I 1 is ‘visit’. R O I 2 is ‘my’. R O I 3 is ‘grandma’. Completion is ‘every weekend.’
* Row 2 (Passive): Structure is Passive. Introductory phrase is ‘The garbage is’. R O I 1 is ‘collect’. R O I 2 is ‘every’. R O I 3 is ‘day.’
* Row 3 (Articles): Structure is Articles. Introductory phrase is ‘People used to think an’. R O I 1 is ‘Earth’. R O I 2 is ‘was’. R O I 3 is ‘flat.’
Instruments
L2 outcome measures: implicit, automatized explicit, and explicit L2 knowledge
The L2 outcome measures were three tasks aimed at capturing, respectively, implicit, automatized explicit, and explicit knowledge of the three target structures.
Implicit knowledge was assessed by means of a self-paced reading task (Jiang, Reference Jiang2007) using a moving-window presentation. Participants were required to read sentences word by word and as quickly as possible while focusing on meaning to accurately answer comprehension questions. Each sentence began with the first word displayed on the left-hand side of the computer screen. Upon pressing a designated key, the word disappeared, and the next word appeared to the right. Half of the target sentences were grammatically correct, and half contained a grammatical violation.
Three regions of interest (ROIs) were embedded in the task (see Table 1): (1) the critical word where the grammatical violation could be detected, (2) the word following the critical word, and (3) the word following the word at ROI 2. A delay in reaction time (RT) was expected for ungrammatical sentences at ROI 1, while ROIs 2 and 3 captured any spillover effects.
The stimuli consisted of 72 target sentences and 24 fillers; the latter were all grammatically correct. Each sentence was followed by a yes/no comprehension question, with an equal ratio of positive and negative responses. Sentences testing the same structure did not appear consecutively more than twice. A grammatical sensitivity index (GSI) was calculated by subtracting RTs for grammatically correct sentences from RTs for ungrammatical sentences at ROIs 1, 2, and 3 combined (Suzuki, Reference Suzuki2017).
Automatized explicit knowledge (Suzuki & DeKeyser, Reference Suzuki and DeKeyser2015) was assessed via an oral elicited imitation test (R. Ellis, Reference Ellis2005; Erlam, Reference Erlam2006; Godfroid & Kim, Reference Godfroid and Kim2021). Participants listened to a series of sentences, each followed by five random numbers. They were instructed to (1) listen to the sentence, (2) read aloud each number as it appeared, and (3) repeat the sentence in correct English. As participants had to attend to the numerical stimuli, rehearsal and rote repetition of the target sentences were prevented (Mackey & Gass, Reference Mackey and Gass2022). Participants had a 9-s time limit for sentence repetition, which had been determined through piloting.
There were 72 sentences covering the three target structures, with each set comprising an equal number of grammatically correct and grammatically incorrect sentences. Sentences testing the same structure did not appear more than twice in succession. The task did not include fillers to reduce participant fatigue. Piloting with 22 participants followed by post-task interviews had shown that test takers had not identified the target structures during the pilot study.
Sentence lengths ranged from 8 to 24 syllables, with a mean length of 15.68 syllables. Mean sentence duration was 5.53 s, with the shortest at 3.9 s. This duration exceeds the typical time span of 1.5–2.0 s for information to decay in phonological short-term memory (Baddeley, Thomson, & Buchanan, Reference Baddeley, Thomson and Buchanan1975). The test was scored for accurate production of the targeted structures. Avoidance was scored as zero; errors that did not pertain to the targets were ignored. The maximum score was 72.
Explicit knowledge was measured through a gap-fill test comprising 75 sentences targeting the three morphosyntactic structures and employing a three-way multiple-choice answer format from which participants were required to choose one correct option in each case. The maximum score was 75.
Examples of items from the L2 outcome measures can be found in Appendix A of the online supplementary materials.
Measures of explicit and implicit aptitude
Language learning aptitude was assessed by means of the LLAMA suite (Meara, Reference Meara2005; Meara & Rogers, Reference Meara and Rogers2019) and a probabilistic serial reaction time (SRT) task (Kaufman et al., Reference Kaufman, DeYoung, Gray, Jiménez, Brown and Mackintosh2010). The LLAMA comprises four subtests, as follows.
LLAMA B (version 2) evaluates associative memory by requiring participants to learn 20 new vocabulary items for novel picture stimuli within a 2-min learning phase. Subsequently, in an untimed test phase, participants are tasked with matching presented words to their corresponding pictures from a set of 20 options. The maximum score was 20, with 1 point awarded for each correct answer and no penalty for guessing.
LLAMA D (modified version 2) gauges auditory pattern recognition ability. During an exposure phase, participants hear 10 words in an unknown language. In the subsequent test phase, participants hear words from the same language, including both previously encountered and new items, and indicate familiarity through a yes/no response. The maximum score was 40 (20 new and 2 × 10 previously encountered items). Incorrect responses were penalized to compensate for guessing.
LLAMA E (modified version 3) measures sound-symbol correspondence. Participants are presented with 24 phonetic symbols, each representing a distinct syllable. Upon clicking on a symbol, the corresponding syllable is played. Participants can click on any symbol any number of times during a 2-min learning phase. In the subsequent untimed test phase, participants hear two-syllable combinations and must select the correct answer from a set of 20 combinations of previously encountered symbols. As this subtest employed the newest available version, which no longer relies on a two-way multiple-choice format, a partial-credit scoring scheme was used. Specifically, the test phase required participants to select from multiple possible answers, some of which could be only partially correct. The maximum score was 40. Up to 2 points were awarded for each correct answer—1 point each for a correctly identified syllable and its correct position.
LLAMA F (modified version 3) is a grammatical inferencing task in which participants must work out the rules of an unknown language. During a 4-min learning phase, participants interact with buttons revealing simple pictures accompanied by written descriptions. With 20 items in total, participants can click on any button as many times as they wish. In the subsequent untimed 20-item test phase, participants are presented with similar pictures and must select combinations of words that accurately describe the given pictures. Participants were able to construct their responses word by word, making partial scoring essential to capture the nuances of their answers. Each correct word yielded up to 2 points: 1 point for identifying the correct word and 1 point for placing it in the correct position. The maximum possible score was 132.
At the time of data collection, the official scoring system for LLAMA E and LLAMA F had not yet been published; however, it has since been described in detail by Rogers, Meara, and Rogers (Reference Rogers, Meara, Rogers, Wen, Skehan and Sparks2023).
The probabilistic SRT task requires participants to react to visual stimuli presented as black squares appearing in one of four possible locations on the computer screen. Stimulus sequences are generated according to a probabilistic rule, with 85% of the stimuli following a training sequence and the remaining 15% following a control sequence. The task comprises 8 blocks, each containing 120 trials, resulting in a total of 960 trials. Following the protocol outlined by Kaufman et al. (Reference Kaufman, DeYoung, Gray, Jiménez, Brown and Mackintosh2010), trials were initially randomized within each block and subsequently presented in the same fixed order for each participant. Task performance was assessed by calculating the difference in mean RT between training and control trials.
Measures of working memory
Phonological working memory (PWM) was measured by a forward digit span (FDS) task based on the format developed by Linck et al. (Reference Linck, Hughes, Campbell, Silbert, Tare, Jackson and Doughty2013), which adapted a component of the operation span task by Unsworth, Heitz, Schrock, and Engle (Reference Unsworth, Heitz, Schrock and Engle2005). Participants listened to number sequences in their L1, ranging from three to nine digits in length, and were required to repeat back the numbers in order. Each set consisted of four sequences of a particular length, resulting in a total of seven sets and 28 sequences. The maximum score was 168.
Executive working memory (EWM) was assessed using an automated operation span (OSPAN) task developed by Unsworth et al. (Reference Unsworth, Heitz, Schrock and Engle2005). In this task, participants first solved simple mathematical problems and indicated whether the displayed solution was correct or incorrect. They were then presented with a letter to memorize. Following a sequence of mathematical problems and letters, participants were required to recall the letters in order. The task comprised 18 sets of sequences ranging from three to eight letters, resulting in a total of 99 letters. The maximum score was 99.
Measure of L2 proficiency
Speaking proficiency was gauged via a monologic oral production task. Participants were given 3 min to talk without interruption about their typical day at school. Their speech was audio-recorded, transcribed, and analyzed using the CLAN software (MacWhinney, Reference MacWhinney2000). The analysis yielded measures of complexity, accuracy, and fluency (CAF). Guiraud’s index was used to calculate lexical complexity (Guiraud, Reference Guiraud1954). Compared with the standard type–token ratio, which tends to yield lower values for longer texts due to word repetition, Guiraud’s index incorporates the square root of the token count, resulting in a more stable measure. The number of clauses per C-unit was used to calculate morphosyntactic complexity (Foster, Tonkyn, & Wigglesworth, Reference Foster, Tonkyn and Wigglesworth2000). Lexical and morphosyntactic accuracy were operationalized as lexical and morphosyntactic errors per 100 words, respectively (adapted from Michel, Kuiken, & Vedder, Reference Michel, Kuiken and Vedder2007). Fluency was calculated using pruned speech rate, that is, the number of spoken words per minute (adapted from Freed, Reference Freed and Riggenbach2000).Footnote 3
Procedure
The oral production task was conducted via Zoom, while all other measures were programmed into PsychoPy and administered via the Pavlovia platform (Peirce et al., Reference Peirce, Gray, Simpson, MacAskill, Höchenberger, Sogo and Lindeløv2019). Test instructions were provided in L1. Participants were instructed to use headphones in a quiet setting while working at their own computers. The first author monitored the participants through Zoom to ensure protocol adherence and answer any clarification questions. Data were collected in three separate sessions, as shown in Figure 1.
Flow chart of the procedure.
Figure 1. Long description
The flowchart is divided into three horizontal boxes labeled Day 1, Day 2, and Day 3. A legend at the bottom indicates that light blue boxes represent cognitive measures and medium blue boxes represent language measures.
* Day 1 (65 min total): The sequence begins with a Self-paced reading task (language measure, 20 min), followed by a Serial reaction time task (cognitive measure, 25 min), and ends with an Operation span task (cognitive measure, 20 min).
* Day 2 (60 min total): The sequence begins with an Oral-production task (language measure, 3 min), followed by a Forward digit span task (cognitive measure, 20 min), and ends with an Elicited imitation test (language measure, 35 min).
* Day 3 (50 min total): The sequence begins with a Gap-fill test (language measure, 25 min) and ends with a LLAMA test (cognitive measure, 25 min).
Arrows indicate the progression from one task to the next within each day.
Data analysis
To begin with, we computed reliability indices and scrutinized the data distributions. We calculated descriptive statistics for all variables. Bivariate correlations and exploratory factor analysis were used to explore the interrelationships and factor structure of the variables, respectively.
To address the research questions, we employed a multilevel modeling approach. Multilevel modeling allows for the inclusion of random effects, i.e., variables that are expected to generate random and thus unwanted variation. In the context of the present study, participants were included as random effects, enabling intercepts and slopes of predictor variables to vary across individuals (see Linck, Reference Linck, Granena, Jackson and Yilmaz2016, for the benefits of using multilevel models for analyzing individual differences; see Cunnings & Finlayson, Reference Cunnings, Finlayson and Plonsky2015, for a detailed overview of multilevel modeling).
We examined main effects of aptitude and working memory, thus establishing their predictive power. To identify the moderating role of proficiency, we examined interactions between proficiency on the one hand and the relationship between cognitive ability (aptitude, WM) and L2 outcome measures on the other hand. Interaction terms and their significance were calculated and the slopes of the predictor–outcome relationship at different levels of proficiency were scrutinized.
Finally, we employed simple slopes analysis (Aiken, West, & Reno, Reference Aiken, West and Reno1991; Rogosa, Reference Rogosa1981) and Johnson–Neyman intervals (Johnson & Neyman, Reference Johnson and Neyman1936) to determine the tipping point at which any effects of aptitude and WM changed from significant to nonsignificant or vice versa. Simple slopes analysis involves calculating the significance of a relationship between two variables at various (arbitrary) levels of the moderating variable. Johnson–Neyman intervals allow us to calculate the range of significance for a relationship and to visualize the results by showing the values and significance of the slope of a relationship across a continuous moderating variable, i.e., proficiency.
The significance level was set at .05. Reliability and normality checks along with factor analyses were performed in SPSS, version 29.0 (IBM Corp., 2023). Correlational analyses, multilevel modeling, and interaction analyses were carried out in RStudio, version 2024.04.1 (Posit Team, 2024). A list of the R packages used in the study is provided in Appendix B in the online supplementary materials.
Results
This section presents the descriptive results for the variables under study, followed by factor analyses. We then provide answers to our research questions on the basis of a multilevel modeling approach.
Preliminary analyses
Descriptive statistics were calculated for the L2 outcome measures, that is, the self-paced reading (SPR) task as a measure of implicit knowledge, the elicited imitation (EI) test as a measure of automatized explicit knowledge, and the gap-fill (GAP) test as a measure of explicit knowledge. In the SPR task, participants’ mean RT for ungrammatical sentences (449 ms) was longer than for grammatical sentences (439 ms), as expected. A series of t tests was conducted to investigate differences between grammatical and ungrammatical sentences. The GSI of 10 ms for all three structures combined was significant with a small effect size, t(82) = −3.596, p = .001, d = 0.28 (Cohen, Reference Cohen1988). This suggests that participants were sensitive to grammatical violations, which, in turn, is indicative of implicit knowledge. When the target structures were scrutinized separately, passive and articles yielded statistically significant GSIs, whereas the GSI for the past tense failed to reach significance (further details on the results of the t tests can be found in Appendix C of the online supplementary materials).
Participants scored relatively highly on both EI and GAP. Full descriptive statistics for all outcome measures are presented in Table 2.
Descriptive statistics for the outcome measures
Table 2. Long description
The table consists of 11 columns and 3 data rows.
Header Structure:
- Measure
- n (number of participants)
- k (number of items)
- M (Mean) subdivided into Raw and Cor. (Corrected)
- S D (Standard Deviation) subdivided into Raw and Cor. (Corrected)
- Skew
- 95% C I (Confidence Interval)
- S-W (p) (Shapiro-Wilk test)
- Alpha (reliability index)
Data Rows:
1. S P R: n=83, k=72, Raw M=10.80 milliseconds, Raw S D=27.35 milliseconds, Skew=-0.01, 95% C I=4.82 to 16.77, S-W (p)=.863, Alpha=.97 for list 1 and .99 for list 2.
2. E I: n=82, k=72, Raw M=53.40, Cor. M=74.17, Raw S D=12.14, Cor. S D=16.86, Skew=-1.41, 95% C I=70.47 to 77.87, S-W (p)=.001, Alpha=.81.
3. G A P: n=82, k=75, Raw M=57.51, Cor. M=76.68, Raw S D=7.70, Cor. S D=10.26, Skew=-0.90, 95% C I=76.68 to 78.93, S-W (p)=.001, Alpha=.99.
Note: n = number of participants; k = number of items; 1 milliseconds; 2 reliability index for list 1; 3 reliability index for list 2. One participant was unable to continue beyond the first session, so n is 83 for SPR and 82 for EI and GAP.
Scores on the cognitive ability measures were adjusted to a scale out of 100. On the LLAMA tests, participants scored highest on LLAMA B, followed by F, E, and finally D. On the FDS and OSPAN, participants also scored highly. Boxplots are available in Figure 2; full descriptive statistics are available in Appendix C in the online supplementary materials.
Boxplots of the cognitive variables.
Figure 2. Long description
A vertical boxplot chart on a light gray grid. The Y axis represents scores from 0 to 100 in increments of 25. The X axis lists six variables from left to right. The boxes are shaded in a gradient from light blue to dark blue.
* L L A M A B. Median is approximately 60. The box ranges from 45 to 70. A single outlier dot is near 5.
* L L A M A D. Median is approximately 40. The box ranges from 25 to 55. The lower whisker reaches 0.
* L L A M A E. Median is approximately 38. The box is tall, ranging from 18 to 72. The upper whisker reaches 100.
* L L A M A F. Median is approximately 55. The box ranges from 32 to 68.
* F D S. Median is approximately 80. The box ranges from 68 to 90.
* O S P A N. Median is approximately 88. The box ranges from 80 to 94. A single outlier dot is at 40.
The first four L L A M A variables show lower medians and higher variance compared to the F D S and O S P A N variables, which show higher scores and tighter distributions.
Participants’ scores on the SRT task were computed and differences in mean RT across the eight blocks were examined to establish learning effects. Error responses (9% of the data) were excluded. Significant outliers, defined as values exceeding three standard deviations from the mean RT for each participant in each block (1% of the data), were also removed, resulting in N = 83. Each participant’s SRT score was determined by subtracting the mean RT in the training condition (447 ms, SD = 67) from that in the control condition (457 ms, SD = 71). Mean RTs, broken down by block, are available in Appendix C of the online supplementary materials. A series of t tests was conducted to assess learning effects (see Appendix C). Differences between training and control conditions across blocks 4–8 were significant, t(80) = 9.545, p < .001, with a medium effect size, d = 0.75 (Cohen, Reference Cohen1988), so data from these blocks were included in the final SRT score calculation. Split-half reliability with Spearman–Brown correction yielded a coefficient of 0.44 for blocks 4–8. This value is comparable to the reliability of similar tasks in previous studies (Granena, Reference Granena2013b; Kaufman et al., Reference Kaufman, DeYoung, Gray, Jiménez, Brown and Mackintosh2010; Suzuki & DeKeyser, Reference Suzuki and DeKeyser2015, Reference Suzuki and DeKeyser2017).
The descriptive statistics for complexity, accuracy, and fluency as components of speaking proficiency are shown in Table 3.
Descriptive statistics for proficiency variables
Table 3. Long description
The table contains six columns: Measure, n, M, S D, Skew, and S–W (p). It lists data for five measures:
* Lexical complexity: n is 82, M is 7.74, S D is 0.86, Skew is 0.118, S–W (p) is .870.
* Morphosyntactic complexity: n is 82, M is 1.59, S D is 0.28, Skew is 0.681, S–W (p) is .035.
* Lexical accuracy: n is 82, M is 1.02, S D is 0.78, Skew is 1.166, S–W (p) is .001.
* Morphosyntactic accuracy: n is 82, M is 1.16, S D is 1.27, Skew is 2.907, S–W (p) is .001.
* Fluency: n is 82, M is 95.96, S D is 25.37, Skew is 0.263, S–W (p) is .828.
An exploratory factor analysis was conducted on complexity, accuracy, and fluency measures using principal component analysis (PCA) with direct oblimin (oblique) rotation, as the underlying factors were expected to be related (Tabachnick & Fidell, Reference Tabachnick and Fidell2014, p. 651). The analysis yielded a single component, labeled “speaking proficiency.”
A second exploratory factor analysis (PCA) was then performed to determine how the cognitive ability measures would factor together. This analysis identified three components: (1) explicit aptitude (LLAMA B, E, and F), (2) working memory (OSPAN and FDS), and (3) implicit aptitude (SRT and LLAMA D). The factor scores were used in subsequent analyses.
A final exploratory factor analysis (PCA) was conducted to examine how the language measures clustered. Two components were identified: (1) an explicit knowledge component, which included both gap-fill and elicited imitation scores, and (2) an implicit knowledge component comprising the self-paced reading scores. These factor loadings, together with the significant relationship between elicited imitation and self-paced reading shown in Figure 3, corroborate the interpretation of elicited imitation as a measure of automatized explicit knowledge as defined by Suzuki and DeKeyser (Reference Suzuki and DeKeyser2015). That is, elicited imitation appears to gauge a type of knowledge that is neither fully implicit nor fully explicit, but rather shares characteristics of both.
Correlations (Spearman’s rho) between measures of aptitude and working memory, L2 knowledge, and L2 proficiency.
Figure 3. Long description
A multi-panel correlation matrix with seven variables: Explicit aptitude, Implicit aptitude, Working memory, S P R, E I, G A P, and Speaking proficiency.
* Diagonal Panels: Display density plots for each variable, showing the distribution of data.
* Lower-Left Panels: Display scatterplots with blue data points and black linear regression lines shaded with gray confidence intervals. Notable trends include a strong positive linear relationship between E I and G A P, and moderate positive trends between Speaking proficiency and both E I and G A P.
* Upper-Right Panels: Display Spearman rho correlation coefficients. Significant values include:
- Explicit aptitude with E I (0.342**) and G A P (0.339**).
- Working memory with Explicit aptitude (0.220*).
- S P R with E I (0.269*) and Speaking proficiency (0.280*).
- E I with G A P (0.860***) and Speaking proficiency (0.662***).
- G A P with Speaking proficiency (0.617***).
* Axes: The x-axes and y-axes represent the scales for each respective variable, ranging from standardized scores (negative 4 to 2) to raw percentage or time-based scales (0 to 100).
Further details on the procedure and results of the factor analyses can be found in Appendix D of the online supplementary materials.
To what extent are the effects of explicit and implicit aptitude and working memory on learners’ knowledge of selected L2 morphosyntactic structures moderated by L2 proficiency?
To address the first research question, interrelationships between the cognitive ability factors, the outcome measures, and speaking proficiency were calculated. Figure 3 shows Spearman correlation coefficients (upper triangle), scatterplots for variable pairs (lower triangle), and the distribution of data for each variable (on the diagonal).
As can be seen in Figure 3, explicit aptitude is positively and moderately related to EI (r ρ = .342) and GAP (r ρ = .339) scores. Conversely, WM and implicit aptitude are not related to any of the outcome measures. Among the cognitive ability factors, WM is positively related to explicit aptitude (r ρ = .220).
To investigate the predictive power of explicit and implicit aptitude and WM, three multilevel models were built, that is, one model for each of the three outcome measures: SPR assessing implicit knowledge, EI assessing automatized explicit knowledge, and GAP assessing explicit knowledge. The linearity assumption was met for each predictor in each model. The models enabled us to identify main effects of explicit aptitude, implicit aptitude, and WM (predictors), as well as the interaction terms with proficiency (moderator). All predictors were mean-centered. Figure 4 provides a visual representation of the models; the source code and full details of the results are available in Appendix E in the online supplementary materials.
Multilevel models.
Figure 4. Long description
The diagram displays three models, labeled Model 1, Model 2, and Model 3, each illustrating the influence of SPEAK. PROFICIENCY on three mediators: EXPLICIT APTITUDE, IMPLICIT APTITUDE, and WORKING MEMORY, which then point to a final outcome variable.
Legend at the bottom:
- Solid line: Significant relationship.
- Dash-dot line: Significant interaction.
- Dotted line: Non-significant relationship or interaction.
Model 1:
- SPEAK. PROFICIENCY has non-significant dotted paths to all three mediators.
- Mediators have paths to IMPLICIT KNOWLEDGE (S P R). EXPLICIT APTITUDE (+) p = .152 (dotted), IMPLICIT APTITUDE (+) p = .697 (dotted), and WORKING MEMORY (-) p = .828 (dotted).
Model 2:
- SPEAK. PROFICIENCY has a significant interaction (dash-dot line with a minus sign) with EXPLICIT APTITUDE, and non-significant dotted paths to the other two mediators.
- Mediators have paths to AUTOMATIZED EXPLICIT KNOWLEDGE (E I). EXPLICIT APTITUDE (+) p = .000 (solid), IMPLICIT APTITUDE (+) p = .044 (solid), and WORKING MEMORY (-) p = .076 (dotted).
Model 3:
- SPEAK. PROFICIENCY has non-significant dotted paths to all three mediators.
- Mediators have paths to EXPLICIT KNOWLEDGE (G A P). EXPLICIT APTITUDE (+) p = .000 (solid), IMPLICIT APTITUDE (+) p = .747 (dotted), and WORKING MEMORY (-) p = .026 (solid).
In the case of automatized explicit knowledge (EI), a significant positive main effect of explicit aptitude (Model 2: estimate = 4.82, SE = 1.05, t = 4.597, p < .001) and a significant positive main effect of implicit aptitude (Model 2: estimate = 2.12, SE = 1.04, t = 2.041, p = .044) were in evidence. In the case of explicit knowledge (GAP), a significant positive main effect of explicit aptitude was found (Model 3: estimate = 3.36, SE = 0.85, t = 3.977, p < .001). Moreover, the analysis yielded a significant negative main effect of WM (Model 3: estimate = −2.19, SE = 0.86, t = −2.560, p = .026). There were no significant predictors for implicit knowledge (SPR, Model 1).
To investigate the moderating effects of proficiency, the three multilevel models were examined for significant interaction terms between proficiency on the one hand and the relationship between explicit and implicit aptitude and WM and the outcome measures on the other hand. The presence of such interactions would indicate that the relationship between predictor and outcome varies significantly across different proficiency levels. The interaction terms are illustrated in Figure 4, which shows a single significant negative interaction between explicit aptitude and proficiency in Model 2, predicting automatized explicit knowledge (EI) (estimate = −1.99, SE = 0.99, t = −1.997, p = .048). The negative interaction term indicates that the positive relationship between explicit aptitude and automatized explicit knowledge differs in strength across different levels of the moderator variable: at lower proficiency levels, explicit aptitude is a positive predictor of automatized explicit knowledge, but the relationship weakens as proficiency increases, making explicit aptitude a less robust predictor at higher proficiency levels. No other interactions were significant.
Our findings are visualized in more detail in Figure 5, which plots the relationships of explicit and implicit aptitude with the three outcome measures (y-axis) at five proficiency levels: mean proficiency − 2 SD, mean proficiency − 1 SD, mean proficiency, mean proficiency + 1 SD, and mean proficiency + 2 SD (x-axis). Following the lines from left to right in each graph, we can see that they do not run in parallel, but gradually diverge or converge instead; this indicates that the slope of the relationship is different at different proficiency levels. The middle plot on the left shows that the slope is much steeper at lower proficiency levels (e.g., M − 2 SD) compared with higher proficiency levels (e.g., M + 2 SD), that is, the relationship loses power as proficiency increases. This is the significant interaction reported above: proficiency negatively moderates the effect of explicit aptitude on automatized explicit knowledge.
Relationships between L2 knowledge and cognitive aptitudes.
Figure 5. Long description
The grid consists of two columns and three rows. The left column is titled EXPLICIT APTITUDE and the right column is titled IMPLICIT APTITUDE.
Row 1: The Y-axis is IMPLICIT KNOWLEDGE. In the Explicit Aptitude panel, lines are mostly flat or slightly positive with wide, overlapping confidence intervals. In the Implicit Aptitude panel, there is a slight positive linear increase for higher proficiency levels.
Row 2: The Y-axis is AUTOMATIZED EXPLICIT KNOWLEDGE. Both panels show a clear positive linear trend where higher aptitude scores correlate with higher knowledge scores. The slopes are steeper for higher proficiency levels.
Row 3: The Y-axis is EXPLICIT KNOWLEDGE. Similar to row 2, there is a positive linear increase across all proficiency levels, with the highest proficiency group (+ 2 S D) maintaining the highest position on the Y-axis.
Common Elements: All X-axes represent aptitude scores ranging from -2 to 2. A legend at the bottom defines Proficiency levels using five line styles: a solid dark line for + 2 S D, a dashed line for + 1 S D, a long-dashed line for Mean, a dash-dot line for - 1 S D, and a dotted line for - 2 S D. Each line is accompanied by a shaded ribbon representing the confidence interval.
Interestingly, a similar visual pattern can be observed in the top and bottom plots on the left-hand side, despite the absence of significant effects of explicit aptitude in Model 1 or significant interaction terms in Models 1 and 3. Overall, the pattern suggests that in the case of implicit and explicit knowledge too, the facilitative effect of explicit aptitude diminishes as proficiency increases.
The plots on the right-hand side of Figure 5 focus on implicit aptitude and show the opposite pattern. Despite a lack of significance, the trend is particularly clear in the middle plot which displays the relationship between implicit aptitude and automatized explicit knowledge. At lower proficiency levels, the slope is near zero, as indicated by an almost horizontal line; in other words, there is no relationship between implicit aptitude and automatized explicit knowledge. However, as proficiency increases, the slope becomes steeper, signaling a stronger relationship. The same pattern holds true for implicit and explicit knowledge, though it is less pronounced. Overall, the observable pattern points toward greater effects of implicit aptitude at more advanced proficiency levels.
Despite the absence of significant interaction terms in Model 3, which showed a significant negative main effect of WM on explicit knowledge, we plotted the relationship at five proficiency levels, as above. Figure 6 shows that, again, the slope of the relationship is different at different proficiency levels, pointing toward a (nonsignificant) interaction. Specifically, the observable pattern indicates that the negative effects of WM on explicit knowledge decrease as proficiency increases.
Relationship between explicit knowledge and working memory.
Figure 6. Long description
A line graph with the X axis labeled WORKING MEMORY ranging from minus 4 to 2 and the Y axis labeled EXPLICIT KNOWLEDGE ranging from 50 to 110. Five regression lines with shaded confidence intervals represent different proficiency levels.
* Plus 2 S D (solid dark teal line): Remains nearly horizontal at approximately 95 on the Y axis across all working memory values.
* Plus 1 S D (dashed teal line): Shows a slight negative linear decrease from 93 to 86.
* Mean (short-dashed teal line): Shows a moderate negative linear decrease from 91 to 78.
* Minus 1 S D (dash-dot light teal line): Shows a steeper negative linear decrease from 88 to 69.
* Minus 2 S D (dotted light teal line): Shows the steepest negative linear decrease, starting at 86 and ending at 60.
The confidence intervals widen as working memory increases, particularly for the lower proficiency groups.
At which point on the L2 proficiency continuum do the facilitative effects of explicit and implicit aptitude and working memory become or cease to be significant?
To address the second research question, we further scrutinized the interactions described in the preceding section by means of simple slopes analysis and Johnson–Neyman (J–N) intervals. Simple slopes analysis involves the methodical examination of the slope of a relationship between two variables at different points of a moderating variable. If and where the slope is significant, the relationship is significant. In our case, this means examining the slope of the relationship between aptitude and WM on the one hand and the L2 outcome measures on the other hand at different proficiency levels. We used the same points on the continuum as above, that is, M − 2 SD, M − 1 SD, M, M + 1 SD, and M + 2 SD.
First, we focused on the significant interaction of proficiency with the relationship between explicit aptitude and automatized explicit knowledge as identified in Model 2. The results of the simple slopes analysis are shown in Table 4.
Simple slopes analysis of proficiency as a moderator of the relationship between explicit aptitude and automatized explicit knowledge
Table 4. Long description
The table consists of five columns: Proficiency level, Est. (Estimate), Std. error, t, and p.
* At proficiency level M minus 2 S D: Est. is 8.79, Std. error is 2.45, t is 3.59, and p is less than .001.
* At proficiency level M minus 1 S D: Est. is 6.80, Std. error is 1.60, t is 4.26, and p is less than .001.
* At proficiency level M (Mean): Est. is 4.82, Std. error is 1.05, t is 4.60, and p is less than .001.
* At proficiency level M plus 1 S D: Est. is 2.83, Std. error is 1.27, t is 2.22, and p is .030.
* At proficiency level M plus 2 S D: Est. is 0.85, Std. error is 2.03, t is 0.42, and p is .680.
The data shows a decreasing trend in the estimate and t-values as proficiency levels increase.
As Table 4 indicates, the critical shift happens between M + 1 SD (p = .030) and M + 2 SD (p = .680) proficiency. For values of M + 1 SD and lower, the relationship is significant, while for values of M + 2 SD and higher, it is no longer significant.
J–N intervals capture the entire range of significance and nonsignificance of a moderating variable, so they can help determine the precise point of change. A significant J–N interval represents a range of values of the moderating variable for which the slope of the relationship between the other two variables is significantly different from zero. A significant slope means a significant relationship, so the J–N intervals in fact show the range of proficiency where there is a significant relationship between the predictor and the outcome. Figure 7 shows the relevant J–N interval plot.
Johnson–Neyman plot of the interaction between speaking proficiency and the relationship between explicit aptitude and EI scores.
Figure 7. Long description
The line graph features an X-axis labeled Speaking proficiency ranging from negative 4 to 2 and a Y-axis labeled Slope of explicit aptitude ranging from 0 to 20. A regression line descends from left to right.
* A light blue shaded region represents a significant interaction where p is less than .05. This region covers the left side of the graph from a speaking proficiency of negative 4 to approximately 1.1.
* A vertical dashed teal line marks the boundary at approximately 1.1 on the X-axis.
* To the right of this line, a light red shaded region represents a non-significant interaction labeled n.s.
* A thick black horizontal line on the X-axis at Y equals 0 indicates the Range of observed data, extending from approximately negative 3 to 2.2.
* The confidence interval bands are widest at the far left and far right, narrowing toward the center of the plot.
As can be seen in Figure 7, the slope coefficient as represented by the diagonal line is positive (above zero) and significant (turquoise) at lower levels of proficiency. As proficiency increases, the value of the slope decreases until it reaches nonsignificant levels (red). The simple slopes analysis further yields the interval of all observed (standardized) values [−2.89, 2.22] along with the interval of significance [−2.89, 1.10]. Taken together, this indicates that the relationship is significant for values of proficiency between M − 2.89 SD and M + 1.10 SD, while it is nonsignificant for values of proficiency between M + 1.10 SD and M + 2.22 SD. We can therefore conclude that the tipping point is M + 1.10 SD. Below this value, there is a significant positive relationship between explicit aptitude and automatized explicit knowledge. As proficiency increases, the magnitude of this relationship decreases, and from M + 1.10 SD proficiency onward, the relationship is no longer significant.
Second, we examined the (nonsignificant) interaction of proficiency with the relationship between explicit aptitude and explicit knowledge (Model 3). The interval of all observed (standardized) values [−2.89, 2.22] remains the same. A simple slopes analysis (see Appendix F in the online supplementary materials for full details) and calculation of the J–N intervals placed the critical point at M + 1.13 SD proficiency, as illustrated in Figure 8.
Johnson–Neyman plot of the interaction between speaking proficiency and the relationship between explicit aptitude and GAP scores.
Figure 8. Long description
A line graph with the X-axis labeled Speaking proficiency ranging from -4 to 2 and the Y-axis labeled Slope of explicit aptitude ranging from -5 to 10. A horizontal line is fixed at 0 on the Y-axis. A thick black line on the X-axis indicates the Range of observed data from approximately -3 to 2.2.
A regression line slopes downward from left to right. To the left of a vertical dashed line at approximately 1.1 on the X-axis, the area around the regression line is shaded light blue, indicating a significant relationship where p is less than .05. In this region, the slope of explicit aptitude is positive but decreasing as speaking proficiency increases.
To the right of the vertical dashed line, the regression line and its surrounding confidence interval are shaded light red, labeled n.s. for non-significant. In this region, the confidence interval crosses the zero line on the Y-axis.
A legend at the bottom identifies the black bar as Range of observed data, the red shaded area as n.s., and the blue shaded area as p is less than .05.
The critical point at M + 1.13 SD for explicit knowledge is almost identical to the one detected for automatized explicit knowledge at M + 1.10 SD, indicating a consistent trend in the relationship between explicit aptitude and (automatized) explicit knowledge: as proficiency increases, the facilitative effect of explicit aptitude decreases.
Third, we focused on the (nonsignificant) interaction of proficiency with the relationship between implicit aptitude and automatized explicit knowledge (Model 2), again performing a simple slopes analysis (see Appendix F in the online supplementary materials for full details) and examining the J–N intervals shown in Figure 9.
Johnson–Neyman plot of the interaction between L2 speaking proficiency and the relationship between implicit aptitude and EI scores.
Figure 9. Long description
The X-axis represents Speaking proficiency ranging from -4 to 2. The Y-axis represents the Slope of implicit aptitude ranging from -5 to 10. A horizontal line is fixed at Y equals 0. A central regression line trends upward from left to right. This line is surrounded by a shaded confidence interval.
Two vertical dashed teal lines mark a specific interval on the X-axis between approximately -0.3 and 0.4. Within this narrow vertical band, the shaded area is teal, indicating a significant interaction where p is less than .05. Outside of this band, to the left and right, the shaded area is light red, indicating non-significant regions labeled n.s.
A thick black horizontal segment on the Y equals 0 line indicates the Range of observed data, spanning from approximately -3 to 2.2. The legend at the bottom identifies the black line as Range of observed data, the red shading as n.s., and the teal shading as p is less than .05.
Figure 9 suggests an overall positive relationship between implicit aptitude and automatized explicit knowledge, even though it is statistically nonsignificant except within a narrow range around the proficiency mean. Importantly, the strength of this relationship increases as proficiency increases, as indicated by the increasing values of the slope. This points toward a tipping point around mean proficiency after which implicit aptitude has an increasingly facilitative effect.
Fourth, we focused on the (nonsignificant) interaction of proficiency with the relationship between working memory and explicit knowledge (Model 3). We performed a final simple slopes analysis (see Appendix F in the online supplementary materials for full details) and examined the J–N intervals shown in Figure 10.
Johnson–Neyman plot of the interaction between L2 speaking proficiency and the relationship between working memory and GAP scores.
Figure 10. Long description
The line graph features an X axis labeled Speaking proficiency ranging from negative 4 to 2 and a Y axis labeled Slope of working memory ranging from negative 10 to 5.
* A central regression line shows a linear increase from the bottom left to the top right.
* A vertical dashed teal line intersects the X axis at approximately 0.3.
* To the left of this dashed line, the area around the regression line is shaded light teal, indicating a significance level of p < .05.
* To the right of this dashed line, the area around the regression line is shaded light pink, indicating non-significance n.s.
* A horizontal black line at Y equals 0 represents the baseline.
* A thick black segment on the Y equals 0 line indicates the range of observed data, spanning from approximately negative 3 to positive 2.2 on the X axis.
* The legend at the bottom identifies the thick black line as Range of observed data, the pink shaded area as n.s., and the teal shaded area as p < .05.
As can be seen in Figure 10, the critical point is very close to the mean at M − 0.32 SD proficiency. This suggests that the negative effect of WM on explicit knowledge stops being significant around mean proficiency.
Finally, we examined all other relationships and potential significant interactions presented in Figure 4. However, as illustrated in Figure 4 and detailed in Appendix E in the online supplementary materials, no additional relationships or interactions reached significance. Although some visual patterns in Figure 5 suggest a significant interaction—specifically between proficiency and the relationship between explicit aptitude and implicit knowledge—no such interaction was found. Consequently, no additional J–N plots were produced.
Discussion
The current study aimed to determine the moderating influence of L2 proficiency in the relationship between explicit and implicit aptitude and WM on the one hand and L2 outcomes in terms of explicit, automatized explicit, and implicit knowledge of selected morphosyntactic structures on the other hand. To this end, we first established how the cognitive ability variables factored together. We arrived at three factors which are in line with recent findings (Granena, Reference Granena2012; Roehr-Brackin et al., Reference Roehr-Brackin, Loaiza and Pavlekovic2023; Roehr-Brackin & Pavlekovic, Reference Roehr-Brackin and Pavlekovic2025): explicit aptitude (LLAMA B, E, F), implicit aptitude (LLAMA D and SRT), and working memory (FDS and OSPAN). Using multilevel modeling, we examined main effects and interactions, with the former revealing the predictive power of the cognitive ability factors and the latter the moderating influence of L2 proficiency. On the basis of simple slopes analysis and J–N intervals, we identified the tipping points at which the moderating influence of proficiency underwent a critical change, either losing or gaining facilitative impact.
The role of working memory at different proficiency levels
Contrary to expectation, we found a significant negative main effect of WM in the model predicting explicit knowledge, that is, higher levels of WM were associated with lower scores on the gap-fill task used in the present study. A (nonsignificant) interaction with proficiency indicated that the negative relationship was restricted to lower proficiency levels. While seemingly counterintuitive, our finding is reminiscent of the role of switching ability reported by Linck et al. (Reference Linck, Hughes, Campbell, Silbert, Tare, Jackson and Doughty2013), who identified a negative effect of this executive function. While an explanation is not immediately obvious, it is possible that heavy reliance on WM and executive function in particular may counteract the use of explicit knowledge in contexts where there is no time pressure and where the use of metalinguistic knowledge is encouraged. In such a scenario, reliance on explicit aptitude rather than WM would appear to be advantageous, at least at lower proficiency levels. This is indeed what our study showed.
The role of explicit aptitude at different proficiency levels
Explicit aptitude emerged as a significant positive predictor of automatized explicit and explicit knowledge—a finding which further substantiates previously reported associations between explicit aptitude and outcome measures targeting or favoring the use of explicit knowledge (Li, Reference Li2015, Reference Li2016). Importantly, we found that proficiency was a moderator in this relationship: the role of explicit aptitude decreased with increasing proficiency, significantly so for automatized explicit knowledge measured via an elicited imitation task, and observable as a trend for explicit knowledge measured via a gap-fill task. Put differently, explicit aptitude is of benefit at lower levels of proficiency but does not convey any additional advantages at higher levels of proficiency.
Our finding aligns with previous research on the role of explicit aptitude in beginner- and intermediate-level learners (Artieda & Muñoz, Reference Artieda and Muñoz2016; Li, Reference Li2015, Reference Li2016) while also providing additional empirical evidence that explicit aptitude has a limited impact at higher levels of L2 proficiency (Morgan-Short, Faretta-Stutenberg, Brill-Schuetz, Carpenter, & Wong, Reference Morgan-Short, Faretta-Stutenberg, Brill-Schuetz, Carpenter and Wong2014). This supports the argument that explicit aptitude is crucial at initial stages (Carroll, Reference Carroll, Parry and Stansfield1990) when learners are likely to face challenges, in particular with regard to morphosyntax, as targeted in our study as well as a number of previous studies. For instance, Li (Reference Li2013) found that language-analytic ability as a component of explicit aptitude played a crucial role when learners were confronted with a hard and opaque structure; Yalçın and Spada (Reference Yalçın and Spada2016) reported that language-analytic ability was relevant for judging the grammaticality of difficult structures.
Beyond corroborating previous research, our findings represent a critical step forward for current research, since we were able to identify the exact point at which facilitative effects of certain aptitude components ceased or came into play, as appropriate. In the case of explicit aptitude predicting automatized explicit knowledge, we can specify the scores that would need to be achieved on the components of our speaking proficiency measure for the effects of explicit aptitude to become nonsignificant. To illustrate, we identified M + 1.10 SD as the tipping point (see “Results” section above). As speaking proficiency was operationalized in terms of standardized factor scores, we can interpret the threshold by looking for participants who scored at or near the relevant value and then trace back their performance on the component measures. The nearest scores to M + 1.10 SD in our data set are standardized proficiency scores of 1.09 and 1.11. These correspond to the following combination of speaking proficiency values: Guiraud’s index (lexical complexity) between 7.65 and 8.61, ratio of clauses to utterances (morphosyntactic complexity) between 1.65 and 2.10, lexical and morphosyntactic errors per 100 words of 0.00–0.47 and 0.00–0.23, respectively, and fluency of 105–127 words per minute. In other words, in terms of speaking proficiency components, these are the points at which explicit aptitude ceases to have beneficial effects for performance on an elicited imitation task aimed at assessing automatized explicit knowledge.
It goes without saying that the above interpretation is only meaningful in the context of the present study, since it is dependent on our specific operationalizations. We have, however, presented a proof of concept which is clearly generalizable beyond the current study: it is possible to identify an exact threshold on the proficiency continuum at which specific cognitive abilities cease to be beneficial or come into play as facilitators.Footnote 4
The role of implicit aptitude at different proficiency levels
Implicit aptitude was a significant positive predictor of automatized explicit knowledge. This effect was moderated (nonsignificantly) by proficiency. The overall trend showed that a facilitative effect of implicit aptitude was absent at lower levels and became more pronounced as proficiency increased. In other words, higher implicit aptitude benefited learners at higher proficiency levels. This finding aligns with previous research that has emphasized the importance of implicit aptitude for high achievers (Linck et al., Reference Linck, Hughes, Campbell, Silbert, Tare, Jackson and Doughty2013) or in later stages of learning (Morgan-Short et al., Reference Morgan-Short, Faretta-Stutenberg, Brill-Schuetz, Carpenter and Wong2014). The implications are twofold: at lower proficiency levels, learners with low implicit aptitude are not disadvantaged, nor do learners with high implicit aptitude derive benefit. However, at higher proficiency levels, learners with high implicit aptitude can leverage it to their advantage, while those with low implicit aptitude may be hindered.
Interestingly, implicit aptitude was not a significant predictor of implicit knowledge. While a detailed exploration of the reasons behind this lack of a predictive relationship is beyond the scope of this paper, one possible explanation is learning context. Suzuki and DeKeyser (Reference Suzuki and DeKeyser2015) found that the relationship between implicit aptitude and implicit knowledge was present only in learners with extended lengths of residence in a target-language-speaking country. In contrast, the vast majority of participants in the current study had not spent much or any time in an immersion environment. Of course, this interpretation remains speculative at this stage.
Limitations and suggestions for future research
Inevitably, the present study had a number of limitations, four of which we will mention here. First, our outcome measures focused on selected morphosyntactic structures of English, offering insight into a limited domain of achievement in a specific L2. Second, our elicited imitation task did not include any fillers. We wanted to reduce participant fatigue during the overall onerous data collection sessions, and post-task interviews in our pilot study suggested that participants were not aware of which features were targeted. At the same time, fillers would have been desirable for both increased task validity and comparability with other studies. Third, our proficiency measure, though multicomponent in nature, focused on speaking skill, whereas a fully comprehensive proficiency assessment would need to include listening, reading, and writing as well. Fourth, the interpretation of the significant tipping point which we identified for the influence of explicit aptitude on automatized explicit knowledge is only meaningful in the context of the current study.
Future research should aim to address the above limitations by including different L2s and different linguistic focus points in their outcome measures and by considering a more comprehensive proficiency measure. To achieve a level of generalizability that can be of practical use, proficiency scores benchmarked against the Common European Framework of Reference or other internationally recognized standards would be most desirable. At the same time, we believe that the theoretical and empirical contribution of our findings to the field of L2 learning research is readily apparent and that the analytic approach employed in our study offers a new and fruitful avenue for future research.
Conclusion
The current study has confirmed the predictive power of explicit and implicit aptitude and WM in (instructed) L2 learning. Moreover, the study offers further evidence for the moderating role of L2 proficiency in the relationship between cognitive ability factors and L2 outcomes. In sum, our findings show that as proficiency increases, the beneficial influence of explicit aptitude diminishes while a facilitative influence of implicit aptitude kicks in. Finally, and arguably most importantly, we have demonstrated that it is possible to pinpoint the exact threshold on the proficiency continuum at which a critical switch away from or toward significant facilitation occurs.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/S0272263126101880.
Data availability statement
Materials and measures are available at https://doi.org/10.17605/OSF.IO/Y4RMN.
Open access
