Introduction
Foreign language aptitude—hereafter “aptitude”—has been conceptualized as a key cognitive individual difference that predicts second language (L2) learning outcomes (Carroll, Reference Carroll and Diller1981, Reference Carroll, Parry and Stansfield1990; Granena & Long, Reference Granena, Long, Granena and Long2013; Robinson, Reference Robinson2005; Skehan, Reference Skehan, Wen, Skehan, Biedroń, Li and Sparks2019). Empirical evidence in this regard, however, remains mixed. While meta-analytic syntheses have reported moderate average correlations between aptitude and L2 learning outcomes (Bokander, Reference Bokander, Wen, Skehan and Sparks2023; Li, Reference Li2016), individual studies often showed inconsistent relationships between specific aptitude components and L2 learning.
These discrepancies may stem from measurement-related issues, particularly limited convergent validity across heterogeneous aptitude batteries. Thus, the operationalization of aptitude remains a central concern. As a latent construct, aptitude is defined through how it is operationalized in assessment tasks. Although existing batteries—MLAT (Carroll & Sapon, Reference Carroll and Sapon1959), PLAB (Pimsleur, Reference Pimsleur1966), CANAL-FT (Grigorenko, Sternberg, & Ehrman, Reference Grigorenko, Sternberg and Ehrman2000), LLAMA (Meara, Reference Meara2005; Meara & Rogers, Reference Meara and Rogers2019), and Hi-LAB (Linck et al., Reference Linck, Hughes, Campbell, Silbert, Tare, Jackson and Doughty2013)—have contributed valuable empirical evidence, systematic validation has received relatively limited attention (cf. Bokander, Reference Bokander, Wen, Skehan and Sparks2023; Bokander & Bylund, Reference Bokander and Bylund2020). As a result, theoretical and methodological concerns persist regarding how the construct is operationalized and validated.
Two issues are particularly salient in this regard. First, aptitude research has often been “instrument-led rather than construct-led” (Skehan, Reference Skehan, Wen, Skehan and Sparks2023, p. 232), with available tests (e.g., LLAMA and Hi-LAB) shaping research agendas despite differences in their theoretical underpinnings and measurement foci. Second, limited accessibility of aptitude batteries constrains the accumulation of validation evidence: some tests such as the CANAL-FT and Hi-LAB (both government-funded) have primarily been used by their developers and associates, often within specific learner populations (e.g., advanced learners), whereas more accessible batteries such as the LLAMA tests have attracted broader use but still require further validation.
To address these issues, Pan and Marsden (Reference Pan and Marsden2024a) developed the theory-informed Tests of Aptitude for Language Learning (TALL). The battery operationalizes aptitude components drawn from the earlier stages of Skehan’s (Reference Skehan, Granena, Jackson and Yilmaz2016) Stages Approach, with its working memory (WM) component further specified following Wen’s (Reference Wen2016) Phonological-Executive Model. Building on initial evidence for its internal validity, the present study extends the validation process by examining the predictive validity of TALL in relation to L2 proficiency among university learners of English. In doing so, it provides empirical support for the extrapolation inference within an argument-based validation framework (Kane, Reference Kane and Brennan2006), by evaluating the extent to which TALL scores predict outcomes in the target domain of L2 learning. A transparent and replicable analytical procedure is also reported to facilitate ongoing validation of TALL as an open research tool.
Literature review
The role of aptitude in L2 learning
Over six decades, aptitude has been investigated as a cognitive individual difference that predicts L2 learning outcomes. Carroll (Reference Carroll and Diller1981) argued that the primary function of aptitude tests is predictive: they are intended to correlate positively with subsequent L2 learning success. Empirical studies generally support this view: learners who attain very high L2 proficiency tend to possess exceptional aptitude (e.g., Abrahamsson & Hyltenstam, Reference Abrahamsson and Hyltenstam2008; DeKeyser, Reference DeKeyser2000), and meta-analytic evidence suggests moderate overall correlational effects (e.g., r = .49 in Li [Reference Li2016] and r = .39 in Bokander [Reference Bokander, Wen, Skehan and Sparks2023]) of complete aptitude batteries on general L2 proficiency.
However, findings at the level of specific aptitude components have been inconsistent. For example, Li’s (Reference Li2016) meta-analysis found that rote memory measured by MLAT had the weakest correlations (r = .22) with L2 proficiency. Likewise, Bokander’s (Reference Bokander, Wen, Skehan and Sparks2023) synthesis indicated that evidence for LLAMA subtests as predictors of general L2 proficiency was inconsistent: many reported correlations were non-significant or weak, although some studies reported moderately significant associations for specific subtests and learner groups (e.g., Artieda & Muñoz, Reference Artieda and Muñoz2016; Bokander, Reference Bokander2020). Evidence from other batteries (e.g., CANAL-FT and Hi-LAB) similarly points to moderate but variable effects across learning contexts and outcome measures. These findings raise two (related) issues: (a) the extent to which different aptitude components contribute to L2 learning outcomes, and (b) how the aptitude construct is operationalized in measurement.
Recent theoretical proposals address the first issue by linking aptitude components to stages of L2 learning rather than treating it as a unitary predictor across the entire learning process. According to Skehan’s (Reference Skehan, Granena, Jackson and Yilmaz2016) Stages Approach, L2 development includes three stages—input-oriented, interlanguage development, and performance-oriented—each involving various processes (such as input processing, noticing, pattern identification, and so on) that impose different cognitive demands. Correspondingly, aptitude components are hypothesized to play stage-specific roles (Table 1). Wen’s (Reference Wen2016) Phonological/Executive (P/E) model further specifies distinctions between phonological storage and executive function within WM, positing that these WM capacities contribute differentially in processes across stages. According to Wen and Skehan (Reference Wen and Skehan2021), phonological storage primarily facilitates the input-oriented stage by acting as a temporary buffer that holds incoming speech for segmentation and noticing. Executive function is particularly critical during feedback, where it enables learners to retain their original utterance, attend to corrective input, and update the developing interlanguage system. These frameworks are highly useful for operationalizing—through measurement batteries—the aptitude construct as they aim to address construct comprehensiveness. As such, well-validated measures that have been informed by these theoretical constructs can, in turn, yield fine-grained data for nuanced empirical investigations into aptitude (such as its role in different developmental stages or learning processes).
The stages approaches (Skehan, Reference Skehan, Granena, Jackson and Yilmaz2016; Wen, Biedroń, & Skehan, Reference Wen, Biedroń and Skehan2017)
Table 1. Long description
The table is organized into three columns: S L A stages, L 2 cognitive processes, and Aptitude constructs.
1. Input-oriented stage:
• Input processing (segmentation) involves Attentional control and Working memory.
• Noticing involves Phonetic coding ability and Working memory.
2. Development of interlanguage stage:
• Pattern identification involves Phonetic coding ability, Working memory, and Language analytic ability.
• Complexification (generalizing, extending, restructuring, integrating) involves Language analytic ability and Working memory.
• Handling feedback involves Language analytic ability and Working memory.
3. Performance-oriented stage:
• Error avoidance involves Working memory and Retrieval memory.
• Automatization involves Retrieval memory.
• Creating a repertoire and achieving salience involves Retrieval memory and Chunking.
• Lexicalization involves Chunking.
The second issue lies in the operationalization of aptitude as a predictor, involving questions about the validity and comparability of different aptitude batteries, discussed next.
Aptitude batteries
A range of aptitude batteries have been developed, including MLAT, PLAB, LLAMA, CANAL-FT, and Hi-LAB. These batteries differ in their theoretical grounding, targeted components, and degree of accessibility, which are summarized in Table 2. Discrepancies among these batteries have yielded somewhat inconsistent evidence about the magnitude and nature of the role of aptitude across different facets of L2 learning (Li & Zhao, Reference Li and Zhao2021), and also pose operationalization challenges that are examined below.
Major language aptitude batteries.
Table 2. Long description
The table is organized into six columns.
1. M L A T: Based on the Carrollian model. Components include Associative memory (A M), Phonetic coding ability (P C A), Grammatical sensitivity (G S), and Inductive language learning ability (I L L A). Subtests cover Number Learning, Phonetic Script, Spelling Clues, Word in Sentences, and Paired Associates using English and Maya. It is not accessible to individual researchers.
2. P L A B: Based on the Carrollian model. Components include G P A, Motivation, Vocabulary knowledge (V K), Language analytic ability (L A A), and P C A. Subtests include G P A, Motivation, Vocabulary, Language Analysis, Sound Discrimination, and Sound-Symbol Association using English and artificial languages. It is commercially accessible.
3. L L A M A: Based on the Carrollian model. Components include A M, P C A, and I L or L A A. Subtests L L A M A underscore B, D, E, and F use artificial languages, Northern Canadian dialects, and the Roman alphabet. Accessible with developer assistance.
4. C A N A L dash F T: Based on the C A N A L dash F model. Components include Selective encoding (S E), Accidental encoding (A E), Selective comparison (S C P), Selective transfer (S T), and Selective combination (S C B). Sections 1 through 5 use the Ursulu language and English. It is not accessible to other researchers.
5. Hi dash L A B: Based on the Hi dash L A B model. Components include Executive functioning (E F), Phonological short-term memory (P S T M), A M, Long-term memory (L T M), Implicit learning (I L), Processing speed (P S), and Auditory perceptual acuity (A P A). It features 11 tests including Running Memory Span, Antisaccade, Stroop, Task Switching, and Phonemic Discrimination using English, Hindi, and Russian. It is not accessible to other researchers.
Challenge 1: Theory–data relationship in test development
Operationalizing aptitude requires specifying the construct to be examined. Historically, however, aptitude test development has sometimes preceded or evolved alongside theoretical frameworks. While this has been interpreted as a limitation (as noted by Kormos, Reference Kormos, Granena and Long2013), it may also reflect a reciprocal relationship between theory and empirical data, where each informs the other iteratively. For example, three theoretical frameworks—Carrollian, CANAL-F, and Hi-LAB—have been initially data-driven, whereas other frameworks—such as the Aptitude Complexes/Ability Differential (ACAD) framework (Robinson, Reference Robinson and Robinson2002) and the Stages Approach (Skehan, Reference Skehan, Granena, Jackson and Yilmaz2016)—remain underutilized in test development. A theory-informed approach to test design can help operationalize aptitude in such a way that it can then serve further investigations for theory testing.
Challenge 2: Language(s) used in aptitude tests
The language of test instructions and stimuli can influence test performance, albeit in different ways. Although instructions are typically brief and translatable across contexts, the language they are in and their format may introduce differences in processing demands. Thus, providing instructions in test-takers’ first language (L1) can help reduce construct-irrelevant variance and thereby support test validity (Purpura, Brown, & Schoonen, Reference Purpura, Brown and Schoonen2015).
A greater challenge lies in the encoding language—the language of test items and stimuli—that cannot be altered without potentially affecting the construct being measured. As shown in Table 2, many aptitude batteries (e.g., MLAT, PLAB, CANAL-FT, and Hi-LAB) rely on English-based stimuli, potentially introducing confounds when used with L2-English learners. Addressing this issue has involved using artificial or miniature languages (e.g., LLAMA; Rogers, Meara, Barnett-Leigh, Curry, & Davie, Reference Rogers, Meara, Barnett-Leigh, Curry and Davie2017; Rogers, Meara, & Rogers, Reference Rogers, Meara, Rogers, Wen, Skehan and Sparks2023), which aim to create language neutrality (and even, in the case of LLAMA’s associative test, neutrality of meaning too by using novel shapes and cartoon-like characters). However, complete neutrality is difficult to achieve, as such stimuli may still draw, even unintentionally, on phonological, orthographic, structural properties, or form–meaning relations more familiar to some learner populations than others.
An alternative is to adopt an L1-sensitive approach—as we took here—in which the language of the test is tailored to a specific learner population in order to reduce construct-irrelevant influences. An obvious challenge for this approach is that further validation is required to examine its applicability across different L1 populations, using parallel versions in other languages.
Challenge 3: Comprehensive test validation
Construct validity—whether test performance reflects the intended underlying construct (Messick, Reference Messick and Linn1989)—remains underexplored in aptitude research (cf. Bokander, Reference Bokander, Wen, Skehan and Sparks2023; Bokander & Bylund, Reference Bokander and Bylund2020). Existing studies have focused primarily on predictive validity, typically through examining correlations between aptitude scores and learning outcomes, often operationalized as global L2 proficiency scores from standardized tests (e.g., Doughty, Reference Doughty2019; Kiss & Nikolov, Reference Kiss and Nikolov2005; Linck et al., Reference Linck, Hughes, Campbell, Silbert, Tare, Jackson and Doughty2013). While such evidence is essential, it alone is insufficient for establishing construct validity. Reliability—the consistency of test-takers’ responses to the measures—has also been examined, with generally acceptable estimates reported for MLAT (Carroll, Reference Carroll and Diller1981) and Hi-LAB (Hughes, Golonka, Tseng, & Campbell, Reference Hughes, Golonka, Tseng, Campbell, Wen, Skehan and Sparks2023), but more variable results for LLAMA (Bokander & Bylund, Reference Bokander and Bylund2020). However, reliability does not in itself guarantee validity. A more comprehensive validation approach, drawing on multiple sources of evidence, is therefore required to substantiate the construct validity of aptitude measures (Li & Zhao, Reference Li and Zhao2021).
Challenge 4: Accessibility of aptitude batteries
Accessibility is a precondition for independent and objective validation by a research community beyond the test developers themselves (Isbell & Kim, Reference Isbell and Kim2023). However, most aptitude batteries remain unavailable to individual researchers. A notable exception is the freely accessible LLAMA tests, and although item-level data are not automatically released to researchers, they can be requested. Because LLAMA is openly accessible, independent researchers have been able to scrutinize its construct validity (e.g., Bokander, Reference Bokander, Wen, Skehan and Sparks2023; Bokander & Bylund, Reference Bokander and Bylund2020; Iizuka & DeKeyser, Reference Iizuka and DeKeyser2023; Suzuki, Reference Suzuki2021). This underscores the role of open accessibility of instruments in enabling cumulative validation efforts and enhancing methodological rigor in L2 research (Marsden, Reference Marsden, McKinley and Rose2020; Plonsky, Reference Plonsky2024).
TALL as an aptitude battery
To address the challenges outlined above, TALL (Pan & Marsden, Reference Pan and Marsden2024a) was designed for L1 Chinese speakers with four key features.
First, TALL is theoretically informed, drawing on the “Input-oriented” and “Interlanguage development” stages of Skehan’s (Reference Skehan, Granena, Jackson and Yilmaz2016) Stages Approach, supplemented by Wen’s (Reference Wen2016) Phonological/Executive (P/E) model for WM. Building on these frameworks, TALL operationalizes aptitude as a set of components underlying early phases of L2 learning, namely phonetic coding ability, language analytic ability, and WM. It also includes associative memory, given its consistent presence across aptitude batteries and its central role in aptitude conceptualization (Skehan, Reference Skehan, Granena, Jackson and Yilmaz2016). In line with this focus, TALL excludes the ‘Performance-oriented’ stage, which primarily involves the automatization and fluent use of acquired knowledge and is less amenable to reliable operationalization within aptitude measures.
Second, TALL adopts an L1-sensitive design tailored to L1-Chinese learners, in line with the considerations outlined above. As such, TALL is not intended as a universally applicable aptitude battery, and its use beyond this population requires empirical validation. Instructions are provided in Mandarin to minimize unnecessary processing demands. For language-based subtests, a miniature language adapted from Lithuanian is used to reduce the likelihood of prior language knowledge influencing performance of this learner population. WM subtests use L1-based stimuli to minimize confounding effects of L2 proficiency while maintaining relevant psycholinguistic constraints, such as phonotactic legality and minimal semantic activation in the subtest measuring PSTM.
Third, TALL has undergone internal validation at the subtest, item, and battery levels (Pan & Marsden, Reference Pan and Marsden2024a), following procedures adapted from the validation of LLAMA by Bokander and Bylund (Reference Bokander and Bylund2020). Results suggest acceptable reliability and alignment with the intended components, providing an initial foundation for systematic validation and motivating further investigation of external validity (e.g., convergent, divergent, and predictive; Li & Prior, Reference Li and Prior2022) to ascertain its overall construct validity.
Finally, TALL is openly available via an online platform (www.tall-webtest.com), with test manuals, materials, datasets, and analysis code available in the IRIS repository (Marsden & Mackey, Reference Marsden and Mackey2014), thereby facilitating independent validation and cumulative research efforts.
The present study
This study contributes to the ongoing validation of TALL. Building on the internal validation reported by Pan and Marsden (Reference Pan and Marsden2024a), it examines the predictive validity of the battery in relation to L2 proficiency. Using openly available data (Pan & Marsden, Reference Pan and Marsden2024b; see Appendix A), the study addresses the question of how well TALL and its subtests predict learners’ L2 proficiency. The findings are intended to provide initial evidence for the extrapolation inference—linking test performance to the target domain it is intended to predict—within an argument-based validation framework (Kane, Reference Kane and Brennan2006).
Methods
Participants
First-year undergraduates aged 18 to 20 years were recruited from 11 universities in China. All participants were enrolled in non-linguistics programs (e.g., civil engineering, computer science, financial management). They had been studying English in school for approximately 10–12 years and were generally at an intermediate proficiency. None reported having received formal instruction in additional foreign languages, thus their familiarity with Lithuanian or related languages is highly unlikely.
Data affected by technical issues and cases identified as outliers—based on behavioral indicators of inattentive or invalid responding—were excluded. The final sample comprised 165 participants.Footnote 1 Each participant received a computer-generated TALL score report and 50 yuan (≈ $7.15) as compensation.
Instrument
TALL subtests and methodological features
To operationalize the aptitude components posited by the Input-oriented and Interlanguage development stages of Skehan’s (Reference Skehan, Granena, Jackson and Yilmaz2016) Stages Approach and Wen’s (Reference Wen2016) P/E model of WM, TALL comprises five subtests (Pan & Marsden, Reference Pan and Marsden2024c, Reference Pan and Marsden2024d; see Appendix B for subtest manuals and stimuli; and see Pan & Marsden [Reference Pan and Marsden2024a] for test environment details):
-
1. Vocabulary Learning (TALL_VL) measures associative memory in both aural and written modalities. The subtest requires participants to form and retain associations between verbal stimuli and their meanings. Participants study 20 novel nouns paired with images of familiar objects in 2 min and are then tested on their accuracy in matching the presented nouns to the corresponding pictures. Although informed by LLAMA_B, TALL_VL differs in its use of images of familiar concrete nouns (rather than nonexistent objects/unfamiliar shapes), with the intention of approximating typical language learning situations in which new words are learned for known entities.
-
2. Sound Discrimination (TALL_SD) measures phonetic coding ability in the aural modality. Informed by Part 5 of the PLAB, this subtest measures how well learners perceive, encode, and identify unfamiliar sounds embedded in short phrases. First, participants are exposed to three target sounds, each paired with a different image of an object, followed by four sets of three short phrases containing the target sounds. Then, participants are required to select the picture that corresponds to the sound embedded in 30 short phrases.
-
3. Language Analysis (TALL_LA) measures language analytic ability in both aural and written modalities. The subtest consists of a learning phase and a test phase. During a five-minute learning phase, participants are presented with 20 pictures paired with phrases and sentences. The subsequent test phase assesses their ability to infer the meanings of linguistic components of those phrases and sentences (two nouns, three verbs, and two modifiers [adjectives or adverbs]) and to induce three grammatical rules (nominal endings, verbal inflections, and word order). In the 30-item test phase, participants are required to select the sentence (from four on-screen options) that accurately describes the meaning of each presented picture. This subtest differs from language analytic subtests in other batteries (e.g., PLAB-4 and LLAMA_F) in its use of concrete pictorial stimuli and the inclusion of novel items in the test phase to assess rule generalization.
-
4. Serial Nonword Recall (TALL_SNWR) measures participants’ PSTM of WM by requiring participants to repeat a series of nonwords in the order of presentation. The subtest consists of three practice trials followed by 17 test trials, each comprising a sequence of two to seven nonwords. Participants initiate the audio recording by clicking an on-screen button, and their responses are recorded for subsequent scoring. This subtest uniquely collects production data (i.e., articulations of the nonword stimuli). Unlike the PSTM subtest in Hi-LAB, which presents stimuli visually, TALL_SNWR adopts the typical auditory-based recall paradigm (Gathercole, Reference Gathercole2006).
-
5. Complex Span Task (TALL_CST), administered in both aural and written modalities, measures executive function of WM. It requires participants to alternate between sentence processing and letter recall within each trial. In each trial, participants are first presented with a sentence and must judge whether the sentence makes sense; they are then presented with a letter to remember, followed by another sentence for judgment and another letter, and so on. At the end of each trial, participants are required to recall all letters in the presented order. The subtest comprises 15 trials, varying in length from three to seven sentence-letter pairs. Sentence judgments function as attention-demanding distractors, while letter recall is scored to serve as the primary measure of WM capacity. This subtest follows the commonly used complex span paradigm (Conway et al., Reference Conway, Kane, Bunting, Hambrick, Wilhelm and Engle2005), adapted here to include domain-specific verbal stimuli.
Modalities and versions. TALL has two parallel test suites, each containing the same five subtests. Three subtests (TALL_VL, TALL_LA, and TALL_CST) are administered in both aural and written modalities. In the aural suite, all subtests are presented auditorily. In the written suite, these three subtests are presented in the written modality, while TALL_SD and TALL_SNWR necessarily remain in the auditory modality.
Each subtest includes two parallel stimulus versions (A and B). For the three modality-specfic subtests, both versions are available in aural and written modalities. These versions are designed to be functionally equivalent in terms of structure, content coverage, and difficulty. Initial validation evidence has shown similar reliability and item functioning across versions in general (Pan & Marsden, Reference Pan and Marsden2024a), supporting their interchangeability for research purposes.
Languages of encoding and instructions. TALL is suitable for measuring L1 Chinese speakers’ aptitude. The target language in TALL_VL, TALL_SD, and TALL_LA is a miniature language derived from Lithuanian, a Baltic language that was, according to our biographical data about other language experiences, unfamiliar to the participants. The WM subtests are encoded in the participants’ L1. Specifically, the nonwords in TALL_SNWR conform to Mandarin phonotactics and are designed to minimize semantic association. TALL_CST presents Mandarin sentences for processing judgment and English letters for recall. This subtest is adapted from the Chinese version of reading span tasks (Gass, Winke, Isbell, & Ahn, Reference Gass, Winke, Isbell and Ahn2019), with modifications such as controlled sentence length and placement of lexical plausibility cues at sentence ends to ensure full-sentence processing.
All instructions are provided in Mandarin Chinese to avoid potential confounds related to English proficiency caused by instructions being in English.
L2 proficiency
Participants’ L2 English proficiency was measured using their self-reported total scores on the National Matriculation English Test (NMET), which they had taken 6 months before data collection. The NMET, regulated by the Ministry of Education, serves as the English test in China’s nationwide, high-stakes university-entrance examination (also known as Gaokao). Test takers are students who have completed their high school education, typically having received approximately 1,500 to 2,000 hours of formal English classroom instruction over their 10 to 12 years of study, starting from the third grade of primary school. This test has been taken by the vast majority of the approximately 11 million students who have registered for the Gaokao each year over the past five years. Based on the China Standards of English Language Ability (CSE), the proficiency required for the NMET aligns with CSE Levels 4 to 5. This corresponds broadly to B1 on the Common European Framework of Reference for Languages (Peng, Liu, & Cai, Reference Peng, Liu and Cai2022). Test takers have 120 minutes to complete the NMET, which is administered in person under invigilated examination conditions. The answer sheets are scanned and distributed to examiners (scorers), who are primarily high school English teachers and university instructors, recruited and trained by provincial testing authorities. The scoring process is completed approximately 15 to 20 days after the test. Test takers subsequently receive a total score and are not provided with a subsection breakdown.
Because NMET test forms differ across provinces, only students who had taken the National Set 1 (administered in the municipality where the study was conducted) were recruited. National Set 1 consists of four sections with a total score of 150 points: Section 1 (30 points) assesses listening comprehension, comprising 20 multiple-choice questions based on 10 recorded dialogues or monologues. Section 2 (40 points) assesses reading comprehension through five passages. It includes 15 multiple-choice questions related to four passages (each 250–350 words) and a five-item gap-filling task based on the fifth passage (around 250 words). Section 3 (45 points) measures grammar and vocabulary knowledge with a 20-item multiple-choice cloze task and a 10-item word-filling cloze. Section 4 (35 points) assesses writing skills, featuring two writing tasks of a 10-point short passage correction and a 25-point guided composition, typically a letter of around 100 words.
Procedure
Participants completed two sessions of TALL, separated by an interval ranging from 31 to 83 days (mean = 41.5, SD = 9.9). In each session, they completed the five subtests in a fixed order—TALL_VL, TALL_SD, TALL_LA, TALL_SNWR, and TALL_CST—using either the aural or the written suite. A within-subject, counterbalanced design was implemented to control for potential practice and familiarity effects while reducing between-participant variability. Three factors were systematically counterbalanced across participants: modality (aural vs. written), material version (A vs. B), and session order (Session 1 vs. Session 2). Participants were therefore assigned to one of four sequences: (a) Version A–aural in Session 1 followed by Version B–written in Session 2; (b) Version B–aural in Session 1 followed by Version A–written in Session 2; (c) Version A–written in Session 1 followed by Version B–aural in Session 2; or (d) Version B–written in Session 1 followed by Version A–aural in Session 2. Within each subtest, item order was randomized for every participant to further reduce order effects. Language background information and NMET scores were collected before the first session began.
Data analyses
All TALL subtest scores and NMET scores were standardized to z-scores to place all variables on a common scale and facilitate comparability across subtests. Composite TALL scores were computed by averaging the standardized subtest scores within each suite.
Given the non-normal distribution of the data (see Appendix C), Kendall’s τ was used for correlation analyses (Field, Miles, & Field, Reference Field, Miles and Field2012). To enable comparison with previous studies reporting Pearson’s r (e.g., Bokander, Reference Bokander, Wen, Skehan and Sparks2023; Li, Reference Li2016), Pearson correlations were also calculated.
Regression and dominance analyses
To examine the predictive validity of each TALL suite, simple regression analyses were conducted separately for the aural and written composite scores, with the composite scores as the predictor and NMET as the dependent variable. To examine the relative contributions of individual subtests, multiple regression analyses were used to examine the relationship between multiple predictor variables (PVs; the TALL subtest scores) and a single dependent variable (DV; the NMET scores). Assumptions (linearity, homoscedasticity, normality of residuals, and absence of multicollinearity) were checked (see Appendix C for the results; Jeon, Reference Jeon and Plonsky2015), with results indicating deviations from normality. To address this issue, robust regression models were fitted using the lmrob function in the robustbase package (Maechler et al., Reference Maechler, Rousseeuw, Croux, Todorov, Ruckstuhl, Salibian-Barrera and Di Palma2021), which provides more reliable estimates when ordinary least squares assumptions are violated (Wilcox, Reference Wilcox2012).
To examine the relative contribution of each subtest to the overall predictive validity of TALL, dominance analysis (DA), one method of relative importance analysis, was employed to provide a more appropriate assessment of predictor importance than standardized regression coefficients in the presence of correlated predictors. This is because standardized coefficients may not accurately reflect each PV’s contribution when PVs are highly correlated with each other, as is expected for subtests representing components of a common aptitude construct. Such correlations among PVs can lead to suppression effects, making it difficult to interpret standardized coefficients as indicators of each PV’s unique contribution (Mizumoto, Reference Mizumoto2023). Consequently, predictor importance may be underestimated or overestimated.
DA addresses this limitation by providing an estimate of predictor importance that explicitly accounts for shared variance among predictors. Although DA requires examining all possible combinations of predictors, modern computing power makes this feasible for regression analyses, and thus it is the preferred method (Mizumoto, Reference Mizumoto2023). DA was conducted using the calc.relimp function in the relaimpo package (Grömping, Reference Grömping2006). Specifically, it estimates predictor importance by decomposing the total explained variance (R 2) across all possible subset models. Each predictor’s contribution is averaged to obtain a general dominance weight. These weights represent each predictor’s proportion of the total explained variance. As they sum to the total R 2, they can be directly compared and rescaled to percentages of the model’s explained variance, providing an interpretable assessment of relative importance when predictors are interrelated.
All analyses (Pan, Reference Pan2026; see Appendix C) were run in R 4.2 (R Core Team, 2022).
Results
Descriptive statistics
Table 3 reports the descriptive statistics for each TALL subtest and NMET. Normality tests indicated that, with the exception of Version B of TALL_VL in the written suite, all other variables deviated significantly from normality.
Descriptive statistics of NMET and TALL subtests and reliability information (as reported by Pan & Marsden, Reference Pan and Marsden2024a)
Table 3. Long description
The table is organized by subtest categories: N M E T, T A L L underscore V L, T A L L underscore S D, T A L L underscore L A, T A L L underscore S N W R, and T A L L underscore C S T.
* N M E T: n equals 165, Mean 126.11, S D 15.81, Skew minus 1.9, S dash W p less than .001.
* T A L L underscore V L: Four datasets (Version A and B in aural and written suites) with k equals 20. Means range from 5.46 to 9.95. Reliability alpha ranges from .70 to .84.
* T A L L underscore S D: Versions A and B in both suites with k equals 30. Means are 21.80 and 23.61. Reliability alpha is .89 and .78 respectively.
* T A L L underscore L A: Four datasets with k equals 30. Means range from 19.73 to 24.73. Reliability alpha ranges from .91 to .96.
* T A L L underscore S N W R: Versions A and B in both suites with k equals 17. Means are 5.70 and 5.71. Reliability alpha is .86 and .88.
* T A L L underscore C S T: Four datasets with k equals 15. Means range from 12.25 to 13.32. Reliability alpha ranges from .64 to .84.
Reliability columns include omega sub h, omega sub t, alpha, and 95 percent C I lower and upper bounds. Most S dash W p-values are less than .001, indicating non-normal distributions.
Note: Key to column headings: n = number of participants (each did two rounds of tests, e.g., Version A in the aural suite and Version B in the written suite); k = number of test items or trials; SD = standard deviation; skew = skewness; S–W (p) = Shapiro–Wilk test of normality; ωh = omega hierarchical; ωt = omega total; α = Cronbach’s alpha; 95% CI = 95% confidence interval with lower and upper bounds.
Predictive power of TALL and its subtests
Separate regression models were fitted for the aural and written suites, as participants completed the two suites on separate occasions. Robust regression analyses were applied to both composite and subtest scores.
The aural suite
The correlation between TALL aural suite’s composite scores and NMET was moderate (τ = .25 and r = .30). Robust regression analysis showed that composite scores significantly predicted NMET performance (b* = .39, p < .001), explaining 12.5% of the variance in NMET scores (R² = .125, 95% CI [.03, .23]).
Regarding subtest-level predictors, Table 4 and Figure 1 report the correlations, regression coefficients, and dominance weights in the aural suite. Only TALL_LA predicted NMET significantly. The rank-order discrepancy between the correlation τ and the standardized b* signaled suppression effects; dominance analysis provided a more appropriate assessment of predictor importance under these conditions. TALL_LA still emerged as the most important predictor, contributing 51.4% of the variance explained by the model (R 2 = .170, 95% CI [.10, .29]), whereas TALL_VL contributed the least. Pairwise comparisons showed that the dominance weight of TALL_LA differed significantly from that of TALL_VL (d = .46, 95% CI [.07, .70]) and TALL_SNWR (d = .44, 95% CI [.00, .70]).
Correlation, robust regression, and dominance analysis coefficients (aural suite)
Table 4. Long description
The table presents data for 165 subjects across six variables: 1 N M E T, 2 T A L L underscore V L, 3 T A L L underscore S D, 4 T A L L underscore L A, 5 T A L L underscore S N W R, and 6 T A L L underscore C S T.
Correlation Matrix (tau):
* Variable 2 correlates with 1 at .17.
* Variable 3 correlates with 1 at .18 and 2 at .21.
* Variable 4 correlates with 1 at .22, 2 at .22, and 3 at .25.
* Variable 5 correlates with 1 at .13, 2 at .20, 3 at .15, and 4 at .03.
* Variable 6 correlates with 1 at .14, 2 at .15, 3 at .09, 4 at .07, and 5 at .22.
Regression and Dominance Analysis:
* T A L L underscore V L: b-star -.00, p .93, Dominance weight .009 (5.31 percent), 95 percent C I [.014, .220], Rank 5.
* T A L L underscore S D: b-star .08, p .14, Dominance weight .031 (18.42 percent), 95 percent C I [.030, .427], Rank 2.
* T A L L underscore L A: b-star .18, p .001, Dominance weight .087 (51.37 percent), 95 percent C I [.205, .737], Rank 1.
* T A L L underscore S N W R: b-star .05, p .37, Dominance weight .012 (7.21 percent), 95 percent C I [.007, .288], Rank 4.
* T A L L underscore C S T: b-star .10, p .08, Dominance weight .030 (17.69 percent), 95 percent C I [.016, .452], Rank 3.
Total R-squared is .170 (100 percent) with a 95 percent C I of [.099, .294]. Boldface values indicate p less than .05.
Note: N = 165. R² = .170, 95% CI [.10, .29]. Boldface indicates the coefficient with p < .05.
Dominance weights (proportion of explained variance) and corresponding 95% CIs of TALL subtests (aural suite). Horizontal error bars show 95% CIs computed from 5,000 bootstrapped replications. Asterisks indicate that the CIs do not contain 0 (p < .05).
Figure 1. Long description
The horizontal bar chart plots Predictor on the Y-axis against Dominance Weights on the X-axis, ranging from 0.00 to 0.75.
From top to bottom, the predictors and their values are:
* T A L L underscore V L: 0.053 with a narrow confidence interval.
* T A L L underscore S N W R: 0.072 with a narrow confidence interval.
* T A L L underscore C S T: 0.177 with a wide confidence interval.
* T A L L underscore S D: 0.184 with a moderate confidence interval.
* T A L L underscore L A: 0.514 with the widest confidence interval.
On the right side of the chart, two vertical brackets indicate statistical comparisons. The inner bracket connects T A L L underscore S N W R to T A L L underscore L A and is marked with an asterisk. The outer bracket connects T A L L underscore V L to T A L L underscore L A and is also marked with an asterisk.
Bottom right text indicates Weighted R-squared equals 0.170 and N equals 165.
The written suite
The correlation between TALL written suite’s composite scores and NMET was moderate (τ = .28 and r = .36). Robust regression analysis showed that the composite scores significantly predicted NMET performance (b* = .41, p < .001), explaining 13.1% of the variance in NMET scores (R² = .131, 95% CI [.04, .24]).
Table 5 and Figure 2 show that TALL_SD and TALL_LA both predicted NMET significantly in the written suite. The rank order of τ agreed with the rank order of the standardized b* coefficients, suggesting the absence of suppression effects. TALL_SD was the strongest predictor, contributing 48.7% to the variance explained by the model (R 2 = .197, 95% CI [.09, .29]), followed by TALL_LA (25.3%), whereas TALL_CST contributed the least. Pairwise dominance analysis indicated that TALL_SD differed significantly from TALL_VL (d = .41, 95% CI [.02, .69]) and TALL_CST (d = .47, 95% CI [.15, .70]).
Correlation, robust regression, and dominance analysis coefficients (written suite)
Table 5. Long description
The table contains 12 columns: Variables, six columns under the tau symbol (1 through 6), b star, p, Dominance weight (percentage), 95 percent C I, and Rank.
Variables and their data:
* 1 N M E T: Serves as the primary reference.
* 2 T A L L underscore V L: Tau 1 is .15. b star is .05, p is .30, Dominance weight is .015 (7.51 percent), 95 percent C I is [.006, .260], Rank 4.
* 3 T A L L underscore S D: Tau 1 is .25, Tau 2 is .12. b star is .19, p is .00, Dominance weight is .096 (48.70 percent), 95 percent C I is [.194, .724], Rank 1.
* 4 T A L L underscore L A: Tau 1 is .20, Tau 2 is .10, Tau 3 is .20. b star is .12, p is .02, Dominance weight is .050 (25.34 percent), 95 percent C I is [.043, .527], Rank 2.
* 5 T A L L underscore S N W R: Tau 1 is .19, Tau 2 is .20, Tau 3 is .16, Tau 4 is .12. b star is .09, p is .09, Dominance weight is .033 (16.76 percent), 95 percent C I is [.018, .402], Rank 3.
* 6 T A L L underscore C S T: Tau 1 is .12, Tau 2 is .13, Tau 3 is .16, Tau 4 is .19, Tau 5 is .20. b star is minus .03, p is .54, Dominance weight is .003 (1.68 percent), 95 percent C I is [.007, .120], Rank 5.
The Total row shows a Dominance weight of .197 (100 percent) and a 95 percent C I of [.089, .287]. Note: N equals 165. R super 2 equals .197. Boldface indicates p less than .05.
Note: N = 165. R² = .197, 95% CI [.09, .29]. Boldface indicates the coefficient with p < .05.
Dominance weights (proportion of explained variance) and corresponding 95% CIs of TALL subtests (written suite). Horizontal error bars show 95% CIs computed from 5,000 bootstrapped replications. Asterisks indicate that the CIs do not contain 0 (p < .05).
Figure 2. Long description
A horizontal bar chart with the Y-axis labeled Predictor and the X-axis labeled Dominance Weights ranging from 0.00 to 0.75.
From top to bottom, the predictors and their values are:
* T A L L _ C S T: 0.017 with a narrow error bar.
* T A L L _ V L: 0.075 with an error bar extending to approximately 0.25.
* T A L L _ S N W R: 0.168 with an error bar extending to approximately 0.40.
* T A L L _ L A: 0.253 with an error bar extending to approximately 0.55.
* T A L L _ S D: 0.487 with the widest error bar extending to approximately 0.73.
On the right side, two vertical brackets with asterisks indicate statistical significance. One bracket compares T A L L _ V L to T A L L _ S D, and a larger outer bracket compares T A L L _ C S T to T A L L _ S D.
Bottom right text indicates Weighted R-squared = 0.197 / N = 165.
Discussion
Predictive power of TALL for L2 proficiency scores
The results of regression analyses indicated that composite scores of TALL accounted for 12.5% (aural suite) and 13.1% (written suite) of the variance in participants’ L2 proficiency scores. When individual subtest scores were entered as predictors, the corresponding multiple regression models accounted for more variance, explaining approximately 17.0% (aural suite) and 19.7% (written suite) of the total variance. These variance estimates fall within the 10%–20% range typically reported for aptitude–L2 ultimate attainment relationships (Granena & Long, Reference Granena, Long, Granena and Long2013). Building on the internal validation evidence reported in Pan and Marsden (Reference Pan and Marsden2024a), the present findings extend the validation argument from internal test properties to relations with an external criterion. Within Kane’s (Reference Kane and Brennan2006) argument-based validity framework, this provides support for the extrapolation inference, suggesting that performance on TALL accounts for variance in the target domain of L2 proficiency measured by the NMET. As a theory-informed instrument, TALL thus provides empirical evidence that aptitude components proposed in the earlier stages of the Stages Approach contribute to L2 proficiency in this learner population.
The observed correlations between composite TALL scores and NMET (r = .30 for the aural suite; r = .36 for the written suite) can be interpreted as moderate in magnitude. Relative to established benchmarks, these effects can be interpreted as follows: they fall around the 50th percentile in Botes, Resnik, Greiff, and Stempfer’s (Reference Botes, Resnik, Greiff and Stempfer2025) second-order synthesized distribution (25th: .20, 50th: .33, 75th: .47) of 633 correlational effects of cognitive individual differences (including aptitude, language awareness, working memory) on L2 proficiency; they fall between small and medium according to Plonsky and Oswald’s (Reference Plonsky and Oswald2014) correlational effect size benchmarks (small: .25; medium: .40; large: .60) for L2 research; and they lie below the meta-analytic mean (r = .49) for aptitude—measured by MLAT and PLAB—and general L2 proficiency correlations reported in Li (Reference Li2016). They are broadly comparable to the moderate correlation between composite LLAMA scores and general L2 proficiency (r = .39) synthesized in Bokander (Reference Bokander, Wen, Skehan and Sparks2023).
From a theoretical perspective, TALL draws on the earlier stages in the Stages Approach, which is intended to account for the componential abilities assumed to be most influential during early stages of L2 learning. Given that the participants in the present study were intermediate learners with several years of prior instruction, a relatively modest aptitude-proficiency relationship might be expected, as other factors (e.g., motivation, input quantity, instructional quality) are likely to play an increasingly prominent role over time. The observed moderate effect sizes therefore suggest that the abilities targeted by TALL remain relevant predictors of L2 attainment beyond the very initial stages of learning, at least in this instructional context. At the same time, these estimates may represent a conservative lower bound of TALL’s predictive potential: stronger associations may be expected among beginner learners, as suggested in previous research (Li, Reference Li, Wen, Skehan, Biedroń, Li and Sparks2019; Skehan, Reference Skehan, Wen, Skehan, Biedroń, Li and Sparks2019). Further research across a wider range of proficiency levels is therefore needed to determine more precisely how aptitude, as measured by TALL, operates across different stages of L2 development, and to more directly evaluate the proposals of the Stages Approach.
Contribution of subtests in predicting L2 proficiency scores
Before interpreting the relative contributions of individual subtests in predicting L2 proficiency scores, it is useful to situate these findings in relation to the internal validation evidence reported in Pan and Marsden (Reference Pan and Marsden2024a). That study established generally satisfactory reliability and internal structure for TALL subtests. In comparison with earlier aptitude batteries, most TALL subtests demonstrated comparable or, in some cases, stronger reliability (e.g., language analytic and phonetic coding measures relative to those in LLAMA and Hi-LAB), although TALL_CST in the written suite showed lower coefficients than the WM span subtests in Hi-LAB.
Building on this internal validation evidence, the present dominance analyses further specify how TALL subtests contribute to predicting L2 proficiency outcomes. The results indicated that, in the aural suite, TALL_LA (for language analytic ability) had the largest dominance weight, with significant differences from TALL_VL and TALL_SNWR. It was followed, in decreasing order of contribution, by TALL_SD, TALL_CST, TALL_SNWR, and TALL_VL. In the written suite, the order differed: TALL_SD (for phonetic coding ability) was the strongest predictor, with TALL_LA second; both predicted NMET scores significantly. TALL_SNWR, TALL_VL, and TALL_CST were then ranked sequentially as non-significant predictors.
Overall, this predictive pattern suggests that, for intermediate-level instructed learners tested on comprehensive explicit-knowledge-oriented measures such as the NMET, the relevant aptitude components are weighted more heavily toward analyzing linguistic rules and perceiving sound contrasts. The memory-related components measured here, perhaps more critical in contexts involving initial lexical acquisition or real-time processing, show reduced predictive power under the present operationalizations.
Language analytic and phonetic coding abilities as key predictors
The significant effect of TALL_LA in both suites aligns with Bokander’s (Reference Bokander, Wen, Skehan and Sparks2023) synthesis, in which LLAMA_F (measuring language analytical ability) emerged as the most consistent predictor of general L2 learning outcomes. In line with previous empirical evidence (e.g., Roehr-Brackin & Tellier, Reference Roehr-Brackin and Tellier2019), our findings lend further support, as proposed in the Stages Approach (Skehan, Reference Skehan, Granena, Jackson and Yilmaz2016), to the central role of language analytic ability in L2 learning, at least in instructed settings with proficiency measures that are likely to draw on explicit knowledge. In other words, the prominent role of language analytic ability in this study can be understood precisely because the NMET and the instructional context both emphasize explicit grammatical analysis and metalinguistic awareness—capacities that lie at the core of language analytic ability.
TALL_SD predicted NMET scores significantly in the written suite, and made the highest contribution among all subtests, echoing Li’s (Reference Li2016) synthesis in which phonetic coding ability outperformed every other aptitude component measured by MLAT/PLAB. This result underscores a fundamental role of a basic capacity to handle unfamiliar sounds in learning a new language, as proposed in the Stages Approach (Skehan, Reference Skehan, Granena, Jackson and Yilmaz2016).
The different rank order of TALL_SD’s dominance weight in the aural versus written suite is noteworthy: TALL_SD accounted for more variance in NMET when embedded in the mixed-modality written suite than in the purely aural suite. One possible explanation is that variance in phonetic coding ability may have become more salient in predicting (mixed-modality operationalizations of) proficiency when TALL_SD was situated within a battery where three other subtests were written than when sitting in an “all-aural” aptitude battery. In the latter, the other aural subtests may also have, to some extent, tapped into ability to handle auditory input, thus potentially reducing or masking the unique explanatory power of TALL_SD.
The reduced distinctiveness of TALL_SD in the aural suite may reflect shared reliance on auditory encoding processes across aptitude tasks, whereas its stronger contribution in the written suite may indicate a more distinctive role for this aptitude component—that is, its specific function in auditory processing is more clearly discernible from other cognitive demands when the battery includes subtests delivered in the written modality. This pattern does not undermine the independent value of an aural aptitude suite. Rather, the shared variance among aural subtests may reflect a broader auditory processing capacity foundational to L2 learning (Saito, Reference Saito2019, Reference Saito2023) and central to the input processing, noticing, and pattern identification in the early stages of L2 learning (Skehan, Reference Skehan, Granena, Jackson and Yilmaz2016; Wen & Skehan, Reference Wen and Skehan2021). The question of how pure-aural versus mixed-modality batteries differentially represent aptitude constructs, and which configuration offers superior predictive utility for specific learning contexts, warrants systematic investigation. Such work would need to manipulate modality as a systematically varied design feature and to examine whether modality effects interact with instructional contexts (e.g., auditory-dominant immersion versus visually dominant classroom instruction).
Limited predictive power of memory-related components
The results showed that no significant association emerged between TALL_VL (measuring associative memory) and NMET. This may be related to an intermediate plateau of participants’ L2 proficiency level, where vocabulary learning may have progressed beyond basic associative stages and may instead draw on more advanced aspects of lexical development, such as depth of knowledge and automaticity, reducing the role of rote associative memory. This finding is consistent with Li’s (Reference Li2016) meta-analytical results, which suggest that rote memory tended to be the weakest predictor of general L2 proficiency among the aptitude components examined.
The finding that TALL_SNWR (measuring PSTM) did not significantly predict general L2 proficiency contrasts with Linck et al.’s (Reference Linck, Hughes, Campbell, Silbert, Tare, Jackson and Doughty2013) reported significant effects of PSTM—measured in the Hi-LAB—in predicting high levels of L2 attainment. Two complementary explanations may account for this discrepancy. First, PSTM may not serve as a kind of bottleneck for the intermediate level of proficiency in the present study. Second, the null effect may be task-related for L2 proficiency: the NMET places substantial weight (approximately 80%) on grammar knowledge, reading comprehension, and writing. These test components rely more heavily on established linguistic knowledge than on temporary phonological storage, thereby reducing the extent to which PSTM is engaged and, consequently, the extent to which its predictive power can be observed.
TALL_CST (measuring executive function of WM) in both suites did not significantly predict L2 proficiency, paralleling Linck et al.’s (Reference Linck, Hughes, Campbell, Silbert, Tare, Jackson and Doughty2013) findings that executive function measures in the Hi-LAB showed largely non-significant associations, some of which were negative, with high-attainment outcomes among advanced foreign language learners. Several factors may have attenuated the predictive power of TALL_CST, particularly in the written modality. First, the subtest showed relatively low item difficulty (Pan & Marsden, Reference Pan and Marsden2024a), thereby offering limited discrimination among participants with higher levels of executive WM capacity, echoing concerns regarding the suitability of complex span tasks for high-ability populations (Draheim, Harrison, Embretson, & Engle, Reference Draheim, Harrison, Embretson and Engle2018). Second, the use of L1 sentences as processing stimuli—intended to minimize confounds with L2 proficiency—may have reduced their effectiveness as distractors, especially for participants with high levels of L1 literacy. As a result, the task may not have sufficiently engaged processing demands relevant to L2 use. This consideration connects to a broader theoretical issue: the executive function of WM is often considered domain-general (e.g., Wen & Skehan, Reference Wen and Skehan2021), and its relevance for L2 learning may depend on the extent to which WM tasks are coupled with language-specific processing demands. From this perspective, the limited involvement of L2 processing in both the present WM measure and the NMET outcome may have reduced the chances of observing any contribution of executive WM to L2 proficiency. Third, the reliability coefficients of the subtest in the written suite were lower than those of other subtests, suggesting that further refinement may be necessary as additional validation evidence becomes available.
Taken together, the weak and non-significant effects of two WM-related subtests suggest that the contribution of PSTM and executive function to general L2 proficiency may be more limited or, more precisely, conditional than assumed in the theoretical frameworks that informed TALL’s design. Specifically, their relevance may depend on the extent to which WM resources are engaged in language-specific processing. In the present study, this coupling may have been attenuated, given that the global NMET score did not isolate components of L2 performance that are likely to draw on WM resources. The findings do not necessarily challenge the role of WM as posited in Skehan’s (Reference Skehan, Granena, Jackson and Yilmaz2016) Stages Approach, but rather highlight the need for research designs that can more directly capture the conditions under which different WM components pertain to L2 learning. This calls for more fine-grained research examining the relationships between distinct components of WM and specific aspects of L2 learning (as proposed by Juffs & Harrington, Reference Juffs and Harrington2011).
Limitations and future directions
Micro-analytic approaches and implicit aptitude measures
The current study relied on global NMET scores as the proficiency criterion because component-level breakdowns are not provided by the testing body. Therefore, we could not link specific aptitude components to distinct L2 skills. Our approach was in line with Artieda and Muñoz (Reference Artieda and Muñoz2016) and Bokander (Reference Bokander2020), who also used global proficiency outcome measures: a school proficiency test or C-test, respectively. Nevertheless, future research on TALL’s validation should move toward componential micro-analytic approaches that pair subtests with targeted learning outcome measures aligned with cognitive processes theoretically defined in the Stages Approach.
Equally important is the development of (more) implicit aptitude measures applying a componential view, an aspect not covered by the current TALL. Although arguably some of TALL’s subtests (e.g., sound discrimination and language analysis in the aural suite) may partially tap into (some) implicit processes, the battery is more likely to capture explicit, controlled processes. It may therefore be particularly well suited to measure aptitude in instructed contexts, such as that examined in the current study. However, aspects of L2 learning (including pronunciation, vocabulary, and grammar) can also be acquired implicitly, without awareness, in both instructed and naturalistic settings. Recent research has begun to explore the measurement of implicit aptitude (e.g., studies included in the special issue by Li & DeKeyser, Reference Li and DeKeyser2021), and conceptualization of implicit aptitude may include possibilities such as statistical learning, error-based prediction, or priming or alignment mechanisms. As noted by Li and Zhao (Reference Li and Zhao2021), clear theoretical definitions are needed to capture the unconscious learning of L2 skills before such constructs can be validly operationalized and incorporated into aptitude testing.
Sampling bias and necessary extensions
The current study was not immune to sampling biases commonly observed in SLA research (Andringa & Godfroid, Reference Andringa and Godfroid2020). Despite being non-Western, our participants shared certain characteristics with WEIRD (Western, Educated, Industrialized, Rich, and Democratic; Henrich, Heine, & Norenzayan, Reference Henrich, Heine and Norenzayan2010) populations, as all were university students with relatively high cognitive ability and L1 literacy. Considering that aptitude (measured by the MLAT) can predict both L1 and L2 attainment (Sparks, Dale, & Patton, Reference Sparks, Dale and Patton2023) and that WM capacity may vary across the lifespan (Gajewski, Hanisch, Falkenstein, Thönes, & Wascher, Reference Gajewski, Hanisch, Falkenstein, Thönes and Wascher2018), the observed patterns need to be replicated among populations of learners with lower L1 literacy and different ages (children, adolescents, and older adults).
Relatedly, the ecological validity of TALL needs to be considered. Although internet-based delivery facilitates access to diverse populations, reaching truly diverse populations (e.g., individuals with lower literacy) may require special testing venues with stable internet and on-site assistance.
Cross-linguistic configurations
Although our Lithuanian-based miniature language was used to mitigate confounds with existing language knowledge, its shared alphabet and phonological similarities with English may still have inflated performance. TALL would need to be adapted to investigate whether learners whose language backgrounds differ to a lesser or greater extent from Lithuanian yield different results.
Conclusion
This study examined the extent to which TALL, a theory-informed battery drawing on Skehan’s (Reference Skehan, Granena, Jackson and Yilmaz2016) Stages Approach and Wen’s (Reference Wen2016) P/E model, accounts for L1 Chinese speakers’ differences in their L2 English proficiency. The findings provide evidence for the predictive validity of TALL, with composite scores on its aural and written suites explaining a meaningful proportion of variance in learners’ L2 proficiency scores (approximately 12.5% and 13.1%, respectively), and with the individual subtests together explaining approximately 17.0% and 19.7%, respectively. These estimates are comparable to effect sizes reported in recent meta-analytic syntheses, thereby supporting the extrapolation inference of TALL as a battery in relation to L2 proficiency.
Importantly, the study also provides initial, theory-relevant evidence regarding how aptitude components relate to L2 proficiency in instructed learning contexts. In particular, language analytic and phonetic coding abilities emerged as the strongest predictors where the outcome measure foregrounds written grammatically accurate expression and understanding, whereas associative memory and WM components appeared to have limited or context-dependent contributions under the present operationalizations of aptitude and L2 outcomes. These patterns are generally consistent with the emphasis placed on analytic and phonetic coding abilities being particularly important in earlier stages of L2 learning and perhaps during online processing and initial lexical acquisition. They also pose interesting questions about the conditions under which memory-related components (associative memory, PSTM, and executive WM) are expected to contribute (more) to L2 development. Although the present findings cannot be interpreted as a direct empirical test of the theoretical frameworks, they nevertheless provide exploratory evidence that helps to identify which theoretically motivated aptitude components are most strongly associated with L2 proficiency for these intermediate instructed learners when tested on relatively explicit measures of L2 knowledge. In other words, this study illustrates the use of a theory-informed battery for profiling aptitude component-proficiency relationships, thereby informing future hypothesis-driven investigations, rather than constituting an empirical test of stage-specific theoretical predictions.
In this sense, the contribution of the study lies not only in advancing the validation of TALL as a psychometrically sound and practically useful open research tool, but also in showing how such an instrument can be used to support more targeted, theory-driven investigations of language aptitude. By making available the instrument, data, analysis scripts, and effect-size estimates, the study offers a transparent and replicable procedure for future research seeking to test, refine, and extend theoretical accounts of language aptitude and its role in L2 learning.
Funding
This project was supported by two grants awarded to the first author from the School of Foreign Languages and Cultures, Chongqing University: the Fundamental Research Funds for the Central Universities (CDJSKJC03) and the Central Universities Research and Innovation Team and Platform Development Project (2025CDJSKPT08).
Appendices
Appendix A. Data files
Pan, J., & Marsden, E. (2024b). Scores from TALL subtests. Data from “Developing and validating an internet-based battery of Tests of Aptitude for Language Learning (TALL)” [Dataset]. IRIS Database, University of York, UK. https://doi.org/10.48316/dd14t-dbNml
These data are also available on the OSF at https://osf.io/grv5m/?view_only=6b424a9b385e4ff3ba889b69912a8a23
Appendix B. Subtest manuals and materials
Pan, J., & Marsden, E. (2024c). TALL subtest manuals. Instructional / Intervention / Teaching / Training materials from “Developing and validating an internet-based battery of Tests of Aptitude for Language Learning (TALL)” [Text/Materials]. IRIS Database, University of York, UK. https://doi.org/10.48316/AZZWN-mJXhL
The subtest manuals are also available on the OSF at https://osf.io/thpju/?view_only=90810a0b582e4c4e8a9de5891122abfa
Pan, J., & Marsden, E. (2024d). TALL subtest materials. Instructional / Intervention / Teaching / Training materials from “Developing and validating an internet-based battery of Tests of Aptitude for Language Learning (TALL)” [Text/Materials]. IRIS Database, University of York, UK. https://doi.org/10.48316/kYHn1-ndaUe
The auditory and written stimulus materials are also available on the OSF at https://osf.io/bhca3/?view_only=6b424a9b385e4ff3ba889b69912a8a23
Appendix C. Analysis code
Pan, J. (2026). Analysis code in R. Code from “Validating the theory-informed Tests of Aptitude for Language Learning (TALL) as predictors of L2 proficiency: Initial evidence from multiple regression analyses” [Software]. IRIS Database, University of York, UK. https://doi.org/10.48316/GeNyl-ksSch
R code documented in an R Markdown file is also available on the OSF at https://osf.io/uv8pq/overview?view_only=d34a1f446dcf4727a6f44873a027c2e0
Open access
