Highlights
-
1. Typological distance constrains the cognitive validity of LLM-derived predictors in L2 reading.
-
2. Surprisal robustly predicted processing difficulty across all L1 groups; contextual entropy effects were qualified by cognitive ability differences among groups.
-
3. Verbal short-term storage and reading comprehension accuracy significantly interacted with typological distance (TD) to modulate early processing effort.
-
4. Higher verbal short-term storage and reading comprehension accuracy buffered the processing burden imposed by greater L1–L2 divergence.
1. Introduction
Prediction is a pervasive cognitive capacity underlying biological perception and decision-making, manifesting in a wide range of forms. It may operate as conscious verbal inference, but can also be realized through nonverbal processes such as intuition and anticipatory action. From cross-modal information integration to complex social interaction, predictive processing constitutes a foundational framework for intelligent behavior. Within human higher cognition, language comprehension represents one of the most refined instantiations of this general predictive capacity. Accordingly, predictive processing constitutes a fundamental cognitive mechanism in language comprehension, enabling individuals to utilize contextual information to anticipate upcoming input, thereby facilitating interpretation and reducing cognitive load (Huettig, Reference Huettig2015; Ryskin & Nieuwland, Reference Ryskin and Nieuwland2023).In L2 reading, however, the efficiency of this predictive mechanism is further constrained by a complex interplay of factors, including linguistic input properties, the TD between the learner’s L1 and L2 and individual differences in cognitive resources (Hopp, Reference Hopp2022; Lee & Fraundorf, Reference Lee and Fraundorf2022). Understanding how linguistic background, TD and cognitive resources jointly shape predictive processing is important for explaining how predictive mechanisms are established and adapted across bilingual experience. This question is particularly relevant to bilingual development, as bilingual children’s predictive use of morphosyntactic cues varies with crosslinguistic relationships and individual differences in language experience (Bosch & Foppolo, Reference Bosch and Foppolo2022; Lemmerth & Hopp, Reference Lemmerth and Hopp2019).
Recent advances in large language models (LLMs) provide powerful computational tools for quantifying predictive processes. Information-theoretic metrics derived from LLMs such as surprisal (reflecting the unexpectedness of a word; Hale, Reference Hale2001; Levy, Reference Levy2008) and contextual entropy (capturing uncertainty about subsequent words; Shannon, Reference Shannon1948) have shown correlations with human neural processing dynamics and effectively index real-time cognitive effort in L2 learners (Cong, Reference Cong2025; Sun, Reference Sun2023; Zhang et al., Reference Zhang, Li, Yang, Xu, Cauwenberghs and Jung2025). Concurrently, eye-tracking offers a sensitive method for examining these processes through measures like first-pass reading time (FPRT), providing insight into the cognitive mechanisms underlying prediction (Haller et al., Reference Haller, Bolliger and Jäger2024; Wilcox et al., Reference Wilcox, Pimentel, Meister, Cotterell and Levy2023; Zhang & Gong, Reference Zhang and Gong2024). Nevertheless, several important questions remain unresolved. First, evidence for the validity of LLM-derived predictive metrics as indices of L2 reading effort has come mainly from Indo-European language pairs (Wilcox et al., Reference Wilcox, Pimentel, Meister, Cotterell and Levy2023; Xu et al., Reference Xu, Chon, Liu, Futrell, Bouamor, Pino and Bali2023). It remains unclear whether the predictive validity of surprisal and contextual entropy generalizes to L2 Chinese learners from diverse Asian L1 backgrounds. Second, although cognitive abilities such as working memory (WM) support predictive processes (Huettig & Janse, Reference Huettig and Janse2016), their role in L2 reading cannot be reduced to general effects on reading time. More critical is whether these abilities modulate learners’ sensitivity to LLM-derived probabilistic cues, thereby helping to separate the cognitive validity of computational metrics from learner-level compensatory processing. Third, TD is known to increase processing demands and constrain syntactic transfer (Hopp, Reference Hopp2022; Sasson et al., Reference Sasson, Schiff and Zluf2025), yet its role as a boundary condition shaping L2 predictive processing efficiency remains insufficiently understood. Clarifying this role is critical for revealing how predictive processing is shaped by the interaction between linguistic experience and learner-level resources, thereby providing a more comprehensive account of variability in L2 reading across crosslinguistic contexts. Therefore, the present study examines how TD modulates the relationship between learner-level measures, including cognitive and performance-related indices, and LLM-derived predictive metrics.
This study contributes to understanding L2 predictive processing in three ways. First, it extends research on LLM-derived predictive metrics by testing the crosslinguistic generalizability of their predictive validity across Asian language pairs (Chinese versus Lao, Burmese, Thai and Vietnamese). Such evidence helps determine whether surprisal and contextual entropy capture general predictive processing mechanisms or are shaped by language-specific patterns. Second, it highlights the role of learner-level cognitive resources in shaping the relationship between LLM-derived predictions and real-time L2 processing. Incorporating cognitive differences into computational models provides a more complete account of how similar predictive demands may be processed differently depending on learners’ cognitive resources and linguistic backgrounds. Third, it tests whether TD conditions the association between learner-level resources and online reading behavior. This perspective positions TD as a boundary condition for cognitive compensation, rather than merely a source of processing difficulty. Ultimately, this work contributes to theoretical understanding of the role of LLMs in crosslinguistic research, which may inform future work on adaptive language-learning models tailored to learners’ linguistic and cognitive profiles.
1.1. LLM predictive metrics in L2 processing: Cognitive mechanisms and computational optimization
Language comprehension is a dynamic and incremental process that entails the continuous integration of lexical input, with associated cognitive load being closely tied to contextual predictability (Kuperberg & Jaeger, Reference Kuperberg and Jaeger2016). During L2 reading, learners must integrate incoming linguistic information with contextual predictions while coordinating multiple sources of linguistic knowledge and experience. Recent advances in LLMs have furnished powerful computational tools for probing these mechanisms through information-theoretic metrics such as surprisal and contextual entropy (Hale, Reference Hale2001; Levy, Reference Levy2008; Wilcox et al., Reference Wilcox, Gauthier, Hu, Qian and Levy2020). Surprisal quantifies the unexpectedness of a word within its context. High surprisal indicates a violation of contextual expectations, often triggering semantic reanalysis and increased cognitive effort (Cevoli et al., Reference Cevoli, Watkins and Rastle2022; Pimentel et al., Reference Pimentel, Meister, Wilcox, Levy and Cotterell2023). Contextual entropy, conversely, captures the uncertainty in upcoming lexical selection. High entropy necessitates enhanced cognitive control to resolve competition among potential interpretations (Pan et al., Reference Pan, Guo, Jiang, Liu and Wu2024). Together, these metrics provide a computational window into the real-time allocation of cognitive load during predictive processing, enabling researchers to examine how predictive processing varies as a function of linguistic experience and individual cognitive resources (Lowder et al., Reference Lowder, Cardoso, Pittman and Zhou2023; Sun, Reference Sun2023).
From a computational standpoint, the predictive validity of these metrics has been substantially augmented by LLMs (Haller et al., Reference Haller, Bolliger and Jäger2024; Wilcox et al., Reference Wilcox, Gauthier, Hu, Qian and Levy2020). Transformer-based architectures, trained on extensive text corpora, develop rich probabilistic representations of language. Their incremental prediction mechanism shows considerable alignment with human neural processing dynamics (Haller et al., Reference Haller, Bolliger and Jäger2024; Sommerschield et al., Reference Sommerschield, Assael, Pavlopoulos, Stefanak, Senior, Dyer, Bodel, Prag, Androutsopoulos and de Freitas2023), thereby offering a robust basis for quantifying cognitive load in L2 reading. For example, Sun (Reference Sun2023) developed an attention-aware methodology that dynamically weights surprisal and semantic relevance to accurately estimate cognitive effort among L2 learners. These findings support the use of LLM-derived metrics as computational indicators of predictive processing in L2 reading, while also highlighting the need to examine how learner characteristics and linguistic backgrounds may influence the interpretation of these metrics.
Converging psycholinguistic evidence further attests to the cognitive plausibility of LLM-derived metrics. Gu et al. (Reference Gu, Nastase, Zada and Li2025) demonstrated that surprisal estimates from LLMs simulate event-related potential (ERP) differences (e.g., N400 amplitudes) between native and L2 readers, suggesting shared computational mechanisms across languages. Similarly, Zhang et al. (Reference Zhang, Li, Yang, Xu, Cauwenberghs and Jung2025) found that among L2 learners, lower contextual entropy correlates with improved comprehension, whereas higher entropy leads to prolonged gaze duration and increased regression probability – reflecting measurable cognitive responses to expectation violation and integration difficulty (Izmalkova & Blinnikova, Reference Izmalkova and Blinnikova2024). Together, these findings indicate that LLM-derived predictive metrics can capture multiple aspects of human predictive processing during L2 reading. However, whether the relationship between these computational measures and learner behavior varies according to crosslinguistic background and individual cognitive characteristics remains unclear.
Notwithstanding these advancements, the application of such metrics in L2 research – particularly across typologically diverse language pairs – faces several critical limitations. First, although these metrics have demonstrated robust predictive validity in well-studied language contexts, their generalizability to low-resource Asian language pairs (e.g., Chinese–Vietnamese, Chinese–Burmese) remains underexplored. Second, contextual entropy estimates depend on the amount of preceding context available, and differences in contextual structure across languages may influence entropy values, thereby complicating crosslinguistic comparisons of predictive measures (Gu et al., Reference Gu, Nastase, Zada and Li2025). Third, substantial variability exists among L2 learners in linguistic experience and cognitive resources (Kim et al., Reference Kim, Kim and Kim2024; Škrjanec et al., Reference Škrjanec, Broy and Demberg2023). However, the extent to which such learner-level differences modulate LLM-derived predictive effects in L2 reading remains insufficiently understood. Finally, biases in LLM training corpora may lead to inaccurate probability estimates, particularly for underrepresented languages, thereby weakening the correspondence between computational metrics and human language processing (Huang et al., Reference Huang, Liang, Wu, Zhu and Yu2023).
1.2. The moderating role of linguistic typology distance and cognitive resources in L2 predictive processing
The efficiency of predictive processing in L2 reading is not determined in isolation by linguistic or cognitive factors, but rather by their dynamic interplay. Specifically, L1–L2 TD and individual cognitive resources jointly shape predictive efficiency, with the former establishing the baseline processing load and the latter modulating and compensating for it.
TD – defined as the degree of structural divergence between the L1 and L2 across grammatical, morphological and phonological systems – directly elevates the cognitive demands of prediction. Structural divergences prolong lexical decoding and syntactic integration (Jeong et al., Reference Jeong, Sugiura, Sassa, Haji, Usui, Taira, Horie, Sato and Kawashima2007; Vainio et al., Reference Vainio, Pajunen and Hyönä2015) and increase prediction errors resulting from cue-weight mismatches and form–function mapping challenges (Hanson & Carlson, Reference Hanson and Carlson2014; Lee & Fraundorf, Reference Lee and Fraundorf2022). Neuroscientific evidence indicates that greater TD increases processing load, eliciting heightened activation in the prefrontal cortex and amplified P600 responses (Durand-López, Reference Durand-López2024; Hopp, Reference Hopp2022). It is precisely this variation in load that LLM-derived metrics such as surprisal and entropy aim to capture.
Beyond directly increasing cognitive load, TD modulates the efficiency of cognitive resource allocation by constraining syntactic transfer and cue-weighting reconfiguration (Hopp, Reference Hopp2022; Hwang & Park, Reference Hwang and Park2025). Language pairs with higher similarity (e.g., English–German) facilitate positive transfer, thereby reducing processing costs (Sasson et al., Reference Sasson, Schiff and Zluf2025). In contrast, highly divergent pairs (e.g., English–Japanese) often induce cue mismatches, triggering compensatory engagement of cognitive control mechanisms that deplete resources otherwise available for prediction (Hopp & Lemmerth, Reference Hopp and Lemmerth2016). The Unified Competition Model (UCM) posits that L2 learners dynamically allocate attention based on cue reliability and salience, often favoring more salient lexical–semantic cues over abstract morphosyntactic ones, which can suppress syntactic prediction mechanisms (Clahsen & Felser, Reference Clahsen and Felser2006, Reference Clahsen and Felser2018; Hanson & Carlson, Reference Hanson and Carlson2014).
Critically, the impact of TD is not deterministic; rather, it is moderated and partially compensated for by an individual’s cognitive resources. Among these, WM is a core cognitive system responsible for the temporary maintenance and manipulation of information, with its principal functions encompassing the updating, shifting and inhibition of mental representations (Conway, Reference Conway2017). While WM encompasses multiple executive functions, the present study focuses specifically on its storage and maintenance component – verbal short-term storage (VSTS) – as this is most directly relevant to the incremental updating of contextual information during reading. Accordingly, we operationalized verbal storage capacity using a forward digit span task (see Method). Learners with higher storage capacity are expected to sustain predictive efficiency even under greater typological demands, whereas those with lower capacity may show different allocation patterns under higher processing demands (Hopp, Reference Hopp2022; Kim & Christianson, Reference Kim and Christianson2017).
Beyond WM, nonverbal intelligence – such as abstract reasoning and pattern recognition – offers additional cognitive support for language prediction, particularly in crosslinguistic comprehension and under conditions of typological divergence. By compensating for reduced availability or mismatch of language-specific cues, nonverbal cognitive pathways enable learners to integrate and infer information more effectively, thereby enhancing the adaptability and robustness of predictive processing while reducing reliance on real-time WM resources (Bice & Kroll, Reference Bice and Kroll2019; Ryskin & Nieuwland, Reference Ryskin and Nieuwland2023).
In addition to these cognitive capacities, learner-level differences in reading-task comprehension performance may also covary with predictive processing efficiency during L2 reading. In the present study, reading comprehension accuracy (RCA) is included as a learner-level index derived from task performance. Because this measure is likely shaped by multiple factors, including L2 proficiency, vocabulary knowledge and typological proximity, findings related to it are interpreted as reflecting overall comprehension performance.
Concurrently, language-specific skills provide the representational foundation for prediction. Vocabulary knowledge facilitates real-time processing and accelerates semantic integration (Roberts, Reference Roberts2012; Liu, Reference Liu2024), while literacy – encompassing phonological and morphological awareness – alleviates decoding load and mitigates difficulties associated with structural disparities (Lee & Kim, Reference Lee and Kim2023; Zhang & Koda, Reference Zhang and Koda2021). Within Robinson’s (Reference Robinson2001) Cognitive Resource Coordination Model, executive functions and statistical learning ability regulate predictive processing and facilitate crosslinguistic transfer, thereby compensating for resource limitations.
In summary, L2 predictive processing is best conceptualized as a trade-off between processing efficiency and cognitive load, wherein TD sets the baseline difficulty and cognitive resources dynamically modulate this load. However, several critical gaps persist in this integrative view. First, predictive processing research has rarely involved systematic comparisons among Asian languages. Second, the respective contributions of general cognitive resources (e.g., VSTS) and language-specific skills (e.g., literacy) to predictive processing remain underexplored. Most importantly, few studies have directly tested the key hypothesis arising from this framework: TD moderates the relationship between those cognitive abilities and processing efficiency.
2. Present study
The preceding review highlights a fundamental question: Under what conditions do LLM-based predictive metrics maintain their cognitive validity in crosslinguistic contexts? This study contends that L1–L2 TD serves as a key moderating variable that can explain such inconsistencies, shaping how cognitive resources interact with computational predictors during L2 reading. Grounded in this theoretical stance and addressing the identified research gaps, the present study investigates the cognitive mechanisms that underlie predictive processing in L2 reading, focusing on the interplay between TD, cognitive resources and LLM-based predictors.
To examine the cognitive validity of LLM-based metrics and their boundary conditions in crosslinguistic reading contexts, we conducted a single empirical study with a series of increasingly complex analytic models. We operationalized early cognitive load during passage reading using FPRT.
First, we established a baseline model (BM) to test the predictive validity of the LLM-derived measures within our sample and evaluated the incremental explanatory power of cognitive abilities. We hypothesized that:
H1: Higher surprisal and higher contextual entropy would significantly predict increased FPRT.
H2: Individual cognitive abilities (including VSTS, RCA, nonverbal intelligence, literacy skills and vocabulary size) would account for significant additional variance in FPRT beyond that explained by established linguistic predictors (e.g., word frequency, word length, proficiency and subjective familiarity), thereby modulating the effects of surprisal and contextual entropy.
A hierarchical linear model (HLM) was employed to test these hypotheses, controlling for key potential confounds.
Next, we extended the model to directly test the moderating role of TD, which served as a continuous variable (comparing Chinese–Lao, Chinese–Burmese, Chinese–Thai, Chinese–Vietnamese). TD was quantified using Hamming distance based on features drawn from the World Atlas of Language Structures (WALS). Chinese and Burmese represented a high-distance pair (TD = 0.45), while Chinese and Vietnamese constituted a low-distance pair (TD = 0.32) (see Supplementary Table S1). The same set of five learner-level measures was incorporated to examine their potential interaction with typological group. We hypothesized that:
H3: The predictive strength of LLM-derived measures on FPRT would be greater for the Vietnamese group (closer typology) than for the Burmese group (more distant typology).
H4: Cognitive abilities more strongly predict processing performance in language pairs with larger TD, indicating the activation of cognitive compensation mechanisms under conditions of greater linguistic distance.
2.1. Method
This study aimed to establish the baseline predictive validity of LLM-derived metrics (surprisal and contextual entropy) for L2 Chinese reading and to evaluate the incremental explanatory power of cognitive abilities beyond traditional linguistic predictors (H1 and H2), as well as to examine the moderating role of TD and the compensatory effects of cognitive abilities (H3 and H4). All analyses were performed on the same dataset using a series of increasingly complex mixed-effects models, as detailed in the “Results” section.
2.1.1. Participants
Data were obtained from the Chinese Learner Eye-tracking Corpus (CECO, Lu et al., Reference Lu, Lao, Zhang, You and Sun2025) – developed by the Language and Cognition Laboratory of Guangxi University, it primarily targets eye movement data of the Association of Southeast Asian Nations (ASEAN) learners while reading Chinese. A total of 246 non-native speakers of Chinese were included, comprising native speakers of Lao (n = 69), Burmese (n = 51), Thai (n = 59) and Vietnamese (n = 67), representing learner groups with different degrees of TD from Chinese. Participants were undergraduate or graduate students between 16 and 43 years old (Mage = 23.6, SD = 4.0; 102 females) who had attained certification on the Hanyu Shuiping Kaoshi (HSK) – the standardized test of Chinese proficiency for non-native speakers, administered by the Ministry of Education of the People’s Republic of China (Level 4: n = 129; Level 5: n = 68; Level 6: n = 17). All participants reported normal or corrected-to-normal vision and no history of neurological or psychiatric disorders. Written informed consent was obtained from all participants, who were compensated for their involvement. Table 1 presents detailed sample characteristics.
Information regarding participants in available samples
Table 1. Long description
The table consists of seven columns and five rows including the header.
Column headers from left to right:
1. L 1 (First Language)
2. n (Sample size)
3. Age (M, range)
4. Years of education (M, S D)
5. Compensation
6. Passages after trimming, percent
7. Word tokens after trimming
Data rows:
- Lao: n = 52; Age = 23.31 (19–38); Education = 17.33 (2.55); Compensation = 50 R M B / hr; Passages = 71 percent; Word tokens = 153,840.
- Burmese: n = 51; Age = 23.04 (16–35); Education = 16.18 (2.46); Compensation = 50 R M B / hr; Passages = 86 percent; Word tokens = 181,827.
- Thai: n = 51; Age = 22.76 (18–42); Education = 18.33 (3.97); Compensation = 50 R M B / hr; Passages = 81 percent; Word tokens = 172,830.
- Vietnamese: n = 60; Age = 23.68 (19–43); Education = 18.33 (3.17); Compensation = 50 R M B / hr; Passages = 82 percent; Word tokens = 206,632.
2.1.2. Cognitive ability measures
All participants completed the Language Experience and Proficiency Questionnaire (LEAP-Q; Marian et al., Reference Marian, Blumenfeld and Kaushanskaya2007) to provide detailed information on their language backgrounds. For consistency in analysis, Chinese was uniformly treated as the target second language (L2), regardless of its numerical order in their language repertoire (see Supplementary Tables S2 and S3).
The study was guided by a hierarchically moderated framework of L2 predictive processing. Within this model, information-theoretic measures (specifically, surprisal and contextual entropy) derived from LLMs are hypothesized to directly influence moment-to-moment processing effort during L2 comprehension. The effects of these metrics are further moderated by learner-internal cognitive resources, including VSTS, RCA, vocabulary knowledge, literacy skills and nonverbal intelligence. Moreover, L1–L2 TD is theorized to constrain the compensatory role of these cognitive resources, delineating the boundary conditions under which LLM-derived metrics remain predictive.
WM is hypothesized to play a central moderating role (Huettig & Janse, Reference Huettig and Janse2016). Individuals with greater WM capacity are posited to maintain and integrate more elaborate contextual representations, facilitating more robust prediction generation and verification (Ding et al., Reference Ding, Zhang, Liang and Li2023; Just & Carpenter, Reference Just and Carpenter1992). Conversely, limited WM increases susceptibility to cognitive overload and promotes shallower, “semantics-first” processing strategies (Durand-López, Reference Durand-López2024; Sagarra & Herschensohn, Reference Sagarra and Herschensohn2010; Hopp, Reference Hopp2022; Kim & Christianson, Reference Kim and Christianson2017). WM capacity also influences early lexical access and exerts cascading effects on downstream comprehension processes (Huang et al., Reference Huang, Ouyang and Jiang2022).
Additional cognitive capacities are theorized to support predictive processing: vocabulary knowledge and literacy provide representational quality that speeds semantic integration and narrows prediction space (Goodrich & Namkung, Reference Goodrich and Namkung2019; Grabe, Reference Grabe2009; Nation, Reference Nation2006); and nonverbal intelligence supports rule abstraction, pattern detection and cognitive flexibility, all relevant to real-time prediction (Horn & Cattell, Reference Horn and Cattell1967; Motallebzadeh & Yazdi, Reference Motallebzadeh and Yazdi2016). Thus, these abilities are expected to jointly moderate the influence of surprisal and contextual entropy on online L2 reading.
To capture these cognitive constructs, five core abilities were assessed (See Appendix A for more details):
-
1. VSTS: An automated forward digit span task adapted from Robinson (Reference Robinson2002) was administered to assess verbal storage capacity. Performance was scored as the maximum sequence length correctly recalled in order. The digit span score was treated as a continuous variable. We chose the forward version (rather than backward) because our theoretical focus was specifically on the storage component of verbal memory, not on executive processing or manipulation (Just & Carpenter, Reference Just and Carpenter1992).
-
2. RCA: RCA was calculated for each participant as the proportion of correctly answered comprehension questions across all passages, with the number of correct responses divided by the total number of comprehension questions completed. It reflects the combined influence of L2 proficiency, vocabulary knowledge, background knowledge, typological proximity, motivation and overall reading engagement during text processing.
-
3. Nonverbal intelligence: The Culture Fair Test 20 (CFT-20; Weiß, Reference Weiß2006) was used as a measure of fluid reasoning ability.
-
4. Literacy: A standardized Chinese character recognition test (Zhang et al., Reference Zhang, Zhang, Zhang and Wang2021), administered in paper format, assessed character decoding accuracy and orthographic knowledge.
-
5. Vocabulary knowledge: A computer-based Chinese vocabulary size test (Wen et al., Reference Wen, Qiu, Leong and van Heuven2024) estimated the breadth of participants’ lexical knowledge.
Descriptive statistics for the individual-difference measures are presented in Supplementary Table S4. With the exception of the paper-based literacy measure, all tests were administered online in a fixed order. Since the VSTS score reflects maximum performance on a standardized task, conventional internal consistency metrics (e.g., Cronbach’s α) are not applicable. All other psychometric instruments demonstrated acceptable reliability, with Spearman–Brown split-half coefficients exceeding 0.87 for each scale (see Table 2).
Reliability of the cognitive tests
Table 2. Long description
The table consists of four columns and five rows including the header. The columns are labeled Measure, Cronbach alpha, Split-half r sub h h, and Spearman-Brown r sub S B.
* Row 1: R C A measure shows Cronbach alpha 0.949, Split-half 0.974, and Spearman-Brown 0.987.
* Row 2: Intelligence measure shows Cronbach alpha 0.671, Split-half 0.773, and Spearman-Brown 0.872.
* Row 3: Literacy measure shows Cronbach alpha 0.911, Split-half 0.951, and Spearman-Brown 0.975.
* Row 4: Vocabulary size measure shows Cronbach alpha 0.820, Split-half 0.889, and Spearman-Brown 0.941.
A footer note defines alpha as Cronbach alpha, r sub h h as split-half reliability, and r sub S B as test reliability corrected by the Spearman-Brown formula.
Note: α = Cronbach’s alpha; rhh = split-half reliability; rSB = test reliability corrected by the Spearman–Brown formula.
2.1.3. Design and materials
The stimuli comprised 75 Chinese passages presented in a fixed pseudorandomized order. Fifteen passages were selected from HSK reading-comprehension materials (Levels 4–6), and 60 were drawn from naturalistic corpora. Genres were balanced to ensure ecological validity and topic diversity (20 narrative, 20 expository, 20 argumentative). Passage length ranged from 25 to 134 characters (M = 55.7, SD = 27.5). All materials were deemed appropriate for university-level readers. Each passage was followed by one or two multiple-choice comprehension questions assessing both factual recall and inferential understanding.
The dependent measure was FPRT, operationalized as the cumulative duration of all fixations within a word’s region of interest during the initial reading pass. FPRT is a well-established and sensitive indicator of early lexical and integrative processing effort (Rayner, Reference Rayner2009); longer FPRT reflects greater cognitive effort associated with word recognition, syntactic integration, semantic conflict or reduced predictability. The choice of FPRT as the primary dependent measure was motivated by theoretical alignment with the computational architecture of autoregressive LLMs. Gaze duration reflects processing costs incurred before the reader has an opportunity to regress to earlier text or reanalyze the input (Clifton et al., Reference Clifton, Staub, Rayner, van Gompel, Fischer, Murray and Hill2007; Inhoff & Radach, Reference Inhoff, Radach and Underwood1998; Rayner, Reference Rayner1998). Autoregressive LLMs are strictly incremental left-to-right predictors without any look-back or reanalysis mechanism. As FPRT captures initial word processing and is sensitive to left-context information, it closely aligns with the incremental prediction mechanisms of autoregressive language models (Haller et al., Reference Haller, Bolliger and Jäger2024; Wilcox et al., Reference Wilcox, Pimentel, Meister, Cotterell and Levy2023), making it a suitable behavioral correlate for evaluating information‑theoretic predictions. Crosslinguistic studies further demonstrate that greater L1–L2 TD reliably increases FPRT (β = 0.272, p < .001; Kuperman, Reference Kuperman2025), supporting its validity in the present typological comparison.
Primary predictors included (i) language background and (ii) token-level information-theoretic indices – notably surprisal and contextual entropy – derived from a fine-tuned Chinese GPT-2 model (trained on Chinese Wikipedia). These model-based metrics quantify expected lexical processing difficulty during incremental comprehension and serve as operational measures of lexical predictability.
The following covariates were included to account for potential confounding effects:
-
1. Word frequency (log-transformed counts from SUBTLEX-CH; Cai & Brysbaert, Reference Cai and Brysbaert2010; M = 4.24, SD = 1.25, ranging from 0.30 to 6.23);
-
2. Word length (number of characters);
-
3. Participant Chinese proficiency (standardized HSK scores);
-
4. Item-level vocabulary familiarity, normed via seven-point ratings from a separate sample of 55 L2 Chinese learners (M age = 23.5, SD = 3.7).
Raw cognitive test scores were converted to z-scores to ensure comparability across measures. Continuous variables were screened for outliers; values exceeding ±2.5 standard deviations from the mean were excluded. All predictors (primary variables and covariates) were subsequently z-standardized to aid model convergence and facilitate the interpretation of effect sizes across measures.
2.1.4. Procedure
After providing informed consent, participants completed the eye-tracking reading task. They were instructed to read each passage silently and at their own pace for comprehension. After reading each passage, participants advanced the display by pressing the spacebar and then responded to one or two multiple-choice comprehension items (keys 1–4) that probed both literal and inferential understanding. Following the reading task, participants completed the LEAP-Q (Marian et al., Reference Marian, Blumenfeld and Kaushanskaya2007) and a battery of cognitive assessments. The entire session lasted approximately 2.5 hours, including self-paced breaks.
2.1.5. Apparatus
Eye movements were recorded using a desktop-mounted EyeLink 1000 Plus system (SR Research, Canada) with a sampling rate of 1,000 Hz and a display refresh rate of 270 Hz. Viewing was binocular, but only the right eye was tracked. Stimuli were presented using Experiment Builder software (SR Research, Canada) on a 23.8-inch LCD monitor (1,920 × 1,080 pixels) positioned 70 cm from the participant; head position was stabilized with an adjustable chinrest. Passages were presented one per screen, double-spaced, in 22-point SimSun font; each character subtended approximately 0.8° of visual angle (≈ 29.3 × 29.3 pixels). Calibration was performed using a nine-point grid and repeated until the average spatial error fell below 1.0°. Drift correction was administered at the start of each trial.
2.1.6. Data cleaning and preprocessing
Raw eye-movement records were preprocessed in Data Viewer 4.4.1 (SR Research) and supplemented by manual inspection. Trials were excluded based on the following a priori criteria: (1) fixations that deviated substantially from the region of interest (ROI); (2) trials with >20% of fixations falling outside the ROI; (3) trials exhibiting abnormally dense fixation clusters indicative of unnatural re-reading (as confirmed by trajectory playback) or (4) trials with invalid data resulting from participant error, calibration failure or signal loss. Across all participants, a total of 2,670 trials were excluded due to substantial deviation from the ROI, 989 trials were excluded for having more than 20% of fixations outside the ROI, 409 trials were excluded due to unnatural re-reading and 201 trials were excluded due to invalid data. Sixteen participants with fewer than 32 valid passages (out of 75) were excluded from subsequent analyses. At the fixation level, we applied Data Viewer’s four-phase procedure to remove physiologically implausible or artifactual samples, fixations shorter than 80 ms or longer than 1,200 ms, fixations outside the ROI and pseudo-fixations produced by blinks or noise. The proportion of valid data retained at the passage and word levels is reported in Table 1. All statistical analyses were conducted in R 4.5.2 (R Core Team, 2025). Linear mixed-effects models were fitted using the lme4 package (v1.1-38) and evaluated with lmerTest (v3.1-3).
2.1.7. Analytical strategy
The BM included surprisal and contextual entropy (see Appendix B for more details) as fixed-effect predictors of FPRT, while controlling for word frequency, word length, HSK level and vocabulary familiarity; random intercepts were included for participants and items. Years of formal education was also tested as a potential covariate but was not retained, as its inclusion did not significantly improve model fit (all ps > .05) (see Supplementary Table S5). Target Model 1 (TM1) added standardized scores from the five cognitive ability tests. Model comparisons were conducted using likelihood-ratio tests (LRT) and changes in conditional R 2 (ΔR 2). All continuous predictors were standardized (converted to z-scores) prior to analysis, and FPRT was log-transformed to improve normality.
To examine the moderating role of TD and the compensatory effects of cognitive abilities, we extended the baseline mixed‑effects models by introducing interaction terms. Specifically, we employed a 4 (language typology: Lao, Burmese, Thai, Vietnamese) × 5 (cognitive abilities: VSTS, RCA, nonverbal intelligence, literacy, vocabulary size) between‑subjects design. Interaction terms between TD and each cognitive ability were added to form Target Model 2 (TM2). Subsequently, three‑way interaction models (TM3) including TD, cognitive abilities and the LLM‑derived metrics (surprisal and contextual entropy) were constructed to test whether cognitive resources buffer prediction‑related processing costs differently depending on TD. All models were fitted to the same dataset, and the results are presented in a single Results section organized by model complexity.
If the effect of contextual entropy extends beyond initial lexical access, it should also be detectable in later eye-movement measures. We therefore repeated our core analyses using total reading time (TRT), regression-path duration (RPD) and proportion of first-pass regressions (FPR) as dependent variables, following the same mixed-effects modeling framework and including all cognitive covariates.
2.2. Data analysis and results
2.2.1. Baseline predictive validity of LLM metrics (Testing H1)
To test H1, BMs were constructed separately for four learner groups. The results revealed a consistent effect of surprisal across all four national samples. Specifically, surprisal emerged as a significant positive predictor in the Lao (β = 0.07, SE = 0.01, t = 12.60, p < .001), Burmese (β = 0.05, SE = 0.01, t = 8.43, p < .001), Thai (β = 0.06, SE = 0.01, t = 10.78, p < .001) and Vietnamese (β = 0.07, SE = 0.005, t = 15.52, p < .001) groups. This pattern indicates that words that were more unexpected (i.e., associated with higher surprisal) elicited longer FPRT.
In contrast, the effect of contextual entropy differed across groups. No statistically significant effects were observed in the Lao, Burmese or Thai groups, whereas a significant positive effect emerged only in the Vietnamese group (β = 0.02, SE = 0.005, t = 3.32, p < .001), indicating that words associated with greater contextual uncertainty elicited longer FPRT. The fixed effects of control variables and covariates included in the BMs for each group are reported in Supplementary Table S6.
2.2.2. Robustness analysis controlling for baseline cognitive differences
To examine whether the previously observed group-specific entropy pattern remained after accounting for baseline cognitive differences, we fitted an additional linear mixed-effects model including all four L1 groups simultaneously. The model included an interaction between standardized contextual entropy and L1 group retaining surprisal and the same linguistic covariates as in the main analyses and controlling for the five baseline cognitive measures. Group-specific entropy slopes were then estimated from the fitted model.
After adjustment for baseline cognitive differences, the previously significant entropy effect in the Vietnamese group was no longer significant (β = 0.005, SE = 0.005, z = 1.04, p = .300), whereas a significant positive entropy effect emerged in the Burmese group (β = 0.019, SE = 0.005, z = 3.66, p < .001). No significant entropy effects were observed in the Thai (β = −0.001, SE = 0.005, z = −0.22, p = .830) or Lao groups (β = −0.004, SE = 0.005, z = −0.77, p = .442). Full estimates of the group-specific entropy slopes and their pairwise comparisons are reported in Supplementary Tables S7 and S8.
2.2.3. Incremental role of cognitive abilities (Testing H2)
To test H2, we expanded the BM by incorporating standardized scores from five cognitive ability measures, yielding TM1. Model comparisons (summarized in Table 3) revealed that for Vietnamese learners, TM1 yielded a statistically significant improvement in model fit relative to the baseline (χ 2(5) = 34.86, p < .01), although the increase in conditional R 2 was minimal (ΔR 2 = 0.10%), suggesting that while cognitive abilities contributed significantly to the model, their practical explanatory power was limited. For other learners, the improvement in model fit was not statistically significant. These results provide partial support for H2 and further suggest that the role of cognitive abilities in L2 reading processing may be modulated by L1–L2 TD.
Fit indices for the linear mixed-effects models
Table 3. Long description
The table contains nine columns: Dataset, Model, k, A I C, B I C, Log Lik, chi-squared (5), p, and Delta R super 2 sub con.
* Lao Dataset: B M model has k 10, A I C 242,148, B I C 242,242, and Log Lik minus 121,064. T M 1 model has k 15, A I C 242,154, B I C 242,295, Log Lik minus 121,062, chi-squared 4.09, p 0.537, and Delta R-squared minus 0.0002.
* Burmese Dataset: B M model has k 10, A I C 317,550, B I C 317,648, and Log Lik minus 158,765. T M 1 model has k 15, A I C 317,555, B I C 317,702, Log Lik minus 158,763, chi-squared 4.85, p 0.435, and Delta R-squared minus 0.0007.
* Thai Dataset: B M model has k 10, A I C 317,618, B I C 317,716, and Log Lik minus 158,799. T M 1 model has k 15, A I C 317,623, B I C 317,769, Log Lik minus 158,796, chi-squared 5.17, p 0.395, and Delta R-squared 0.0003.
* Vietnamese Dataset: B M model has k 10, A I C 360,697, B I C 360,795, and Log Lik minus 180,338. T M 1 model has k 15, A I C 360,672, B I C 360,820, Log Lik minus 180,321, chi-squared 34.86, p less than .001, and Delta R-squared minus 0.0010 with three asterisks indicating significance.
Note: K denotes the number of model parameters; AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion) are information criteria, with smaller values indicating better model fit. LogLik represents the model log-likelihood value. The χ 2 statistic refers to the likelihood ratio test, with degrees of freedom (df) = 5, and the p value indicates its significance level. ΔR 2 denotes the change in conditional R 2 (Nakagawa & Schielzeth, Reference Nakagawa and Schielzeth2013) after including the predictor variables. Significance levels are defined as: *p < .05, **p < .01, ***p < .001.
To further evaluate H2, we examined the unique contributions of each of the five learner-level measures to FPRT. Among Vietnamese learners, VSTS span emerged as a significant positive predictor (β = 0.10, SE = 0.03, t = 3.73, p < .001), and this effect remained robust after false discovery rate (FDR) correction (adj. p = .002). Although the variance explained was limited, this finding highlights the particular role of the VSTS measure in L2 Chinese reading under conditions of smaller TD. Notably, longer VSTS span was associated with longer FPRT, possibly reflecting deeper engagement of memory resources during the processing of typologically related languages. In addition, literacy (β = −0.09, SE = 0.04, t = −2.41, p = .032) and vocabulary size (β = −0.09, SE = 0.03, t = −2.94, p = .005) showed significant negative predictive effects, which also remained robust following FDR adjustment (adj. ps < .05). Neither RCA nor nonverbal intelligence reached statistical significance. In the Lao, Burmese and Thai learner groups, no cognitive ability significantly predicted FPRT. All observed effects were small in magnitude (|β| < 0.06), and the mean change in conditional R 2 attributable to cognitive abilities was low, suggesting that when TD is relatively large, individual cognitive dimensions alone are insufficient to account for variability in L2 Chinese reading performance. Standardized effects of individual cognitive abilities on FPRT are reported in Supplementary Table S9.
2.2.4. Moderation by typological distance (Testing H3)
To further examine whether TD accounted for the observed entropy pattern beyond baseline cognitive differences, we fitted a unified mixed-effects model in which standardized TD was entered as a continuous moderator while the five cognitive measures were retained as covariates. The interaction between TD and contextual entropy remained significant when compared with a model without the interaction term (χ 2(1) = 19.81, p < .001). The relationship between contextual entropy and processing difficulty varied systematically with TD even after baseline cognitive differences were taken into account.
To test H3, we incorporated interaction terms between TD and the five cognitive ability metrics into the TM2, with the other four cognitive abilities included as covariates, and evaluated their incremental contribution using likelihood ratio tests (LRTs). As summarized in Table 4, only the interactions involving VSTS (χ 2(1) = 5.63, β = −0. 04, p = .018) and RCA (χ 2(1) = 5.51, β = −0.03, p = .019) reached statistical significance, suggesting that TD modulated the relationship between these two cognitive abilities and FPRT. Specifically, higher levels of VSTS and RCA were associated with a weaker negative effect of TD on FPRT and significantly attenuated the processing load associated with increasing TD, as reflected in a reduced negative impact of TD on FPRT as these cognitive abilities increased. None of the other cognitive abilities yielded significant interaction effects. Notably, the conditional R 2 changes (ΔR 2) associated with all interaction terms were very small, indicating that although statistically significant interactions were observed, their overall effect sizes were limited, and the remaining cognitive abilities did not exhibit reliable modulatory effects in either statistical or practical terms. These results provide partial support for H3, indicating that the interaction between TD and cognitive abilities is ability-specific.
Likelihood Ratio Test results for the interaction effects between typological distance and cognitive abilities
Table 4. Long description
The table consists of six columns: Cognitive Test, chi-squared with 1 degree of freedom, p, Delta R super 2 sub con, beta sub int, and p sub int.
* V S T S: chi-squared 5.63, p 0.018, Delta R super 2 sub con 2.97 E minus 05, beta sub int minus 0.04, p sub int 0.018 asterisk.
* R C A: chi-squared 5.51, p 0.019, Delta R super 2 sub con minus 0.0002, beta sub int minus 0.03, p sub int 0.019 asterisk.
* Intelligence: chi-squared 0.32, p 0.573, Delta R super 2 sub con minus 1.50 E minus 05, beta sub int 0.01, p sub int 0.574.
* Literacy: chi-squared 2.48, p 0.115, Delta R super 2 sub con minus 4.17 E minus 05, beta sub int minus 0.02, p sub int 0.116.
* Vocabulary size: chi-squared 0.13, p 0.715, Delta R super 2 sub con minus 2.78 E minus 05, beta sub int minus 0.01, p sub int 0.715.
A note below the table explains that negative beta sub int values indicate that as cognitive ability increases, the effect of typological distance on F P R T weakens. Asterisks indicate significance at p less than 0.05.
Note: χ2 values and p values are derived from likelihood ratio tests, comparing the full model including the interaction terms with a reduced model including only main effects (df = 1). βint represents the standardized coefficient estimate of the interaction term, with its corresponding p value indicating the significance level. A negative βint indicates that as cognitive ability increases, the effect of typological distance on FPRT weakens. ΔR 2con denotes the change in conditional R 2 after adding the interaction term. Significance levels are defined as: *p < .05, **p < .01, ***p < 0.001.
Simple slope analyses (Figure 1) further elucidated the specific patterns underlying the interaction effects described above. In the Burmese learner group, which is typologically more distant from Chinese, higher VSTS was associated with shorter predicted FPRT, yielding a clear negative slope. In contrast, in the Vietnamese, Lao and Thai learner groups, which are typologically closer to Chinese, this relationship was substantially attenuated and even exhibited a slight positive slope. Moreover, the magnitude of the simple slopes increased in a stepwise manner as TD from Chinese decreased. A similar modulatory pattern was observed for RCA. In typologically more distant groups, RCA showed a stronger negative association with FPRT, whereas this relationship was weaker in typologically closer groups. By comparison, intelligence, literacy and vocabulary size exhibited largely consistent simple slope directions across language groups, in line with the absence of statistically significant interaction effects for these variables (see Table 4). This pattern indicates that the modulatory effect of TD is primarily manifested in cognitive abilities that are directly related to online processing resources.
Interaction effects between typological distance and cognitive abilities on FPRT. The x-axis represents standardized values of the cognitive abilities, and the y-axis shows model-predicted FPRT (in milliseconds), back-transformed from predictions on the log-transformed FPRT scale. In each panel, the solid lines in different colors represent simple slopes at the observed typological distance levels of the four learner groups (Vietnamese, Lao, Burmese and Thai), and shaded areas indicate 95% confidence intervals. All predictions were generated while controlling for linguistic and learner-related covariates and including random intercepts for subjects and words.
Figure 1. Long description
A multi-panel set of five line graphs arranged in two rows. The top row contains three panels and the bottom row contains two. All graphs share a Y-axis labeled F P R T (ms) ranging from 300 to 420. The X-axis for all graphs represents standardized cognitive values from negative 2 to 2. Each graph features four colored lines with shaded 95 percent confidence intervals: red (Vietnamese), blue (Lao), green (Burmese), and purple (Thai).
* Top-left panel (V S T S): Red and purple lines show a steep linear increase. Blue shows a slight increase. Green shows a linear decrease.
* Top-middle panel (R A C): Red and purple lines show a moderate linear increase. Blue shows a slight decrease. Green shows a steep linear decrease.
* Top-right panel (Intelligence): All four lines (red, blue, green, purple) show a similar upward linear trend, converging near the center.
* Bottom-left panel (Literacy): Red and purple lines show a very slight increase. Blue and green lines show a linear decrease, with green being steeper.
* Bottom-right panel (Vocabulary size): All four lines show a downward linear trend, with green showing the steepest decline and red the shallowest.
A legend at the bottom identifies the L 1 groups by color: Vietnamese (red), Lao (blue), Burmese (green), and Thai (purple).
2.2.5. Moderation by cognitive abilities (Testing H4)
To evaluate the compensatory mechanism proposed in H4, we fitted three-way interaction models (TM3) including TD, cognitive abilities (with the other four cognitive abilities included as covariates) and predictive language measures (surprisal and contextual entropy), in order to examine how TD modulates the buffering role of cognitive resources on the effects of language prediction. Specifically, all cognitive predictors were entered into TM3 as continuous standardized variables. For post hoc probing and visualization only, predicted effects were plotted at low (−1 SD), mean and high (+1 SD) values.
Table 5 summarizes the results of the three-way interaction terms between surprisal or contextual entropy, cognitive abilities and TD in the linear mixed-effects models. The results indicate significant three-way interactions among TD, cognitive abilities and predictive language measures, with patterns varying across cognitive domains. With the exception of vocabulary size, the inclusion of three-way interaction terms for the remaining four cognitive abilities significantly improved model fit, indicating that the modulatory effects of cognitive abilities on predictive processing vary as a function of TD between learners’ L1 and Chinese.
Likelihood ratio test results for the interaction effects among typological distance, cognitive abilities and surprisal/entropy
Table 5. Long description
The table consists of seven columns: Interaction term, chi-squared (2), p, Delta R super 2 sub con, beta sub int, p sub int, and adj.p.
* T D times surprisal times V S T S: chi-squared 9.68, p 0.008, Delta R super 2 sub con 1.68E-05, beta sub int 0.001, p sub int 0.405, adj.p 0.405.
* T D times entropy times V S T S: chi-squared 9.68, p 0.008, Delta R super 2 sub con 1.68E-05, beta sub int minus 0.004, p sub int 0.002, adj.p 0.004 (significant at p < .01).
* T D times surprisal times R C A: chi-squared 98.93, p < .001, Delta R super 2 sub con 0.0001, beta sub int minus 0.008, p sub int < .001, adj.p < .001 (significant at p < .001).
* T D times entropy times R C A: chi-squared 98.93, p < .001, Delta R super 2 sub con 0.0001, beta sub int minus 0.007, p sub int < .001, adj.p < .001 (significant at p < .001).
* T D times surprisal times intelligence: chi-squared 12.77, p 0.002, Delta R super 2 sub con 2.26E-05, beta sub int minus 0.002, p sub int 0.397, adj.p 0.405.
* T D times entropy times intelligence: chi-squared 12.77, p 0.002, Delta R super 2 sub con 2.26E-05, beta sub int minus 0.005, p sub int 0.004, adj.p 0.006 (significant at p < .01).
* T D times surprisal times literacy: chi-squared 85.93, p < .001, Delta R super 2 sub con 0.0001, beta sub int minus 0.009, p sub int < .001, adj.p < .001 (significant at p < .001).
* T D times entropy times literacy: chi-squared 85.93, p < .001, Delta R super 2 sub con 0.0001, beta sub int minus 0.006, p sub int < .001, adj.p < .001 (significant at p < .001).
* T D times surprisal times vocabulary: chi-squared 3.96, p 0.138, Delta R super 2 sub con minus 2.14E-06, beta sub int minus 0.001, p sub int 0.373, adj.p 0.405.
* T D times entropy times vocabulary: chi-squared 3.96, p 0.138, Delta R super 2 sub con minus 2.14E-06, beta sub int minus 0.002, p sub int 0.189, adj.p 0.270.
Note: χ2 values and p values are derived from likelihood ratio tests, comparing the full model including the three-way interaction terms with a reduced model excluding these terms but retaining all their constituent main effects and lower-order interactions (df = 2). βint represents the standardized coefficient estimate of the interaction term, with its corresponding p value (FDR-adjusted) indicating the significance level. ΔR 2con denotes the change in conditional R 2 after adding the interaction term. Significance levels are defined as: *p < .05, **p < .01 ***p < .001.
Specifically, significant three-way interactions were observed for RCA (χ 2(2) = 98.93, adj. p < .001) and literacy (χ 2(2) = 85.93, adj. p < .001), suggesting that RCA and literacy showed the most pronounced modulatory patterns on predictive language measures under different TD conditions. In contrast, VSTS (χ 2(2) = 9.68, adj. p = .004) and intelligence (χ 2(2) = 12.77, adj. p = .006) showed significant interactions only with contextual entropy, whereas their interactions with surprisal did not reach statistical significance. This pattern indicates that VSTS and intelligence were associated with partial modulation of processing costs related to contextual uncertainty, primarily buffering processing costs associated with contextual uncertainty, and that this effect is modulated by TD. Notably, although several interaction terms reached statistical significance, the associated changes in conditional R 2 (ΔR 2) were very small, indicating that the three-way interactions contributed only minimally to explaining variance in FPRT (Figure 2).
The three-way interaction effects among typological distance, cognitive abilities and surprisal/entropy on FPRT. Fixed-effect predictions from a linear mixed-effects model, showing how five learner-level measures moderate the effect of typological distance on language processing difficulty (surprisal, contextual entropy) in FPRT. Panels A–E represent VSTS, RCA, intelligence, literacy and vocabulary, respectively. Rows correspond to L1 groups (Vietnamese, Lao, Burmese, Thai); columns to cognitive levels (−1 SD, mean, +1 SD), which are used solely for visualization and simple slope analyses; all cognitive abilities were entered into the model as continuous variables. The x‑axis shows standardized cognitive scores; the y‑axis shows model‑predicted FPRT (ms), back‑transformed from log‑scale predictions. Colored lines depict simple slopes at observed typological distance levels for each group; shaded areas are 95% CIs. Predictions control for linguistic and learner covariates and include random intercepts for subjects and words.
Figure 2. Long description
The figure consists of five large panels labeled A through E, each containing a 2 by 3 grid of line graphs.
Common Axes and Legend:
* Y-axis: F P R T in milliseconds, ranging from 300 to 420.
* X-axis: Standardized predictor z, ranging from minus 2 to 2.
* Rows: The top row in each panel represents Entropy; the bottom row represents Surprisal.
* Columns: Represent cognitive levels at minus 1 S D, Mean, and plus 1 S D.
* Legend: Four colored lines represent L 1 groups: Vietnamese (red), Lao (blue), Burmese (green), and Thai (purple).
Panel Breakdown:
* Panel A (V S T S): In the Surprisal row, all groups show a steep linear increase in F P R T as the predictor increases. In the Entropy row, slopes are flatter, with Vietnamese showing a slight downward trend at minus 1 S D and a slight upward trend at plus 1 S D.
* Panel B (R C A): Similar to Panel A, Surprisal shows consistent positive linear slopes across all columns. Entropy slopes are nearly horizontal at the Mean but diverge slightly at the extremes.
* Panel C (Intelligence): Surprisal rows show tight clustering of positive slopes. Entropy rows show very little variation across the standardized predictor range.
* Panel D (Literacy): Surprisal slopes remain positive and steep. In the Entropy row, the Burmese group (green) shows a more pronounced positive slope at minus 1 S D compared to other groups.
* Panel E (Vocabulary): Surprisal slopes are consistent with other panels. In the Entropy row, there is a noticeable downward trend for all groups at the plus 1 S D cognitive level, indicating faster processing as entropy increases for high-vocabulary learners.
2.2.6. Extension to later processing stages: TRT, RPD and FPR
To examine whether the effects of LLM-derived metrics extend to later processing stages, we analyzed TRT, RPD and FPR. After controlling for baseline cognitive differences, contextual entropy significantly predicted TRT, RPD and FPR in the typologically most distant Burmese group, consistent with the revised FPRT pattern. However, entropy effects were less stable in RPD and FPR, and the crucial interaction between entropy and TD was not significant in any late measure. These patterns are consistent with the view that FPRT is the most theoretically appropriate index for the incremental, left-to-right predictions of autoregressive LLMs, as it is uncontaminated by rereading or strategic reanalysis (Haller et al., Reference Haller, Bolliger and Jäger2024). Full model outputs are provided in Supplementary Tables S10–S25.
3. Discussion
3.1. Effectiveness of LLM metrics in low-resource language pairs and their boundary conditions
By comparing learner groups differing in estimated L1–L2 TD and integrating computational modeling with behavioral measures, the present study systematically examined the predictive utility of LLM-generated surprisal and contextual entropy in second language reading among learners of Chinese with Lao, Burmese, Thai and Vietnamese as their first languages. The findings indicate that associations between contextual entropy and reading time vary across learner groups, remain related to TD and are sensitive to adjustment for baseline cognitive characteristics. This pattern identifies L1–L2 TD as a candidate boundary condition governing the psychological realism of computational predictors in language processing research.
Specifically, the BMs showed that among Chinese–Vietnamese learners, both surprisal and contextual entropy significantly predicted early processing effort (as indexed by FPRT), thereby replicating patterns previously observed in Indo-European language studies (Haller et al., Reference Haller, Bolliger and Jäger2024; Wilcox et al., Reference Wilcox, Pimentel, Meister, Cotterell and Levy2023). Across the four learner groups, surprisal showed a relatively stable predictive effect, whereas contextual entropy exhibited greater group-level variability.
The additional covariate-controlled analysis further qualified the entropy results. After VSTS, RCA, nonverbal intelligence, literacy and vocabulary size were included as covariates, the entropy effect was no longer significant in the Vietnamese group, whereas a significant positive effect was observed in the Burmese group. The initial Vietnamese-specific effect therefore cannot be attributed to typological proximity alone. Nevertheless, the interaction between contextual entropy and TD remained significant after all five cognitive measures were controlled, showing that the relationship between contextual entropy and FPRT continued to vary systematically with TD. In the present models, contextual entropy exhibited greater variation across groups and model specifications than surprisal.
Several mechanisms may account for this pattern. First, insufficient training data and limited model generalization may reduce the accuracy of LLM-derived estimates for such language pairs (Ranathunga et al., Reference Ranathunga, Lee, Prifti Skenduli, Shekhar, Alam and Kaur2023). Second, increasing TD may reduce representational similarity between languages and limit the availability of predictive cues (Von Grebmer Zu Wolfsthurn et al., Reference Von Grebmer Zu Wolfsthurn, Pablos and Schiller2026). Third, fundamental differences in syntactic structure may hinder learners’ ability to effectively use contextual information for predictive processing.
Beyond merely identifying this boundary condition, our findings point to a compensatory role for cognitive resources. Individual differences in VSTS capacity (as measured by forward digit span) and RCA were associated with processing effort under different typological conditions. This finding suggests that ultimate processing efficiency emerges from a dynamic interaction between macro-linguistic constraints and micro-cognitive resources.
In summary, through systematic typological comparison, our study delineates the boundary conditions for LLM-based predictive metrics in low-resource Asian language pairs. We show that the alignment between computational models and human cognition is conditional rather than uniform, with its predictive strength varying systematically as a function of L1–L2 TD across learner groups, as reflected in differential surprisal- and entropy-based effects on early reading time.
These findings effectively reframe the central question from “Are LLM metrics effective?” to “Under what conditions, and for which learner populations, are these metrics effective?” This shift underscores the necessity of language-specific validation of computational models. Future research should prioritize architectural improvements that explicitly incorporate typological features (Bankula & Bankula, Reference Bankula and Bankula2025), leverage internal representations derived from parallel corpora (Li et al., Reference Li, Wang, Zhang and Zong2024) and enhance cross-lingual transfer from high-resource languages (Yu et al., Reference Yu, Tian and Li2025), thereby strengthening the cognitive realism of LLM-derived estimates across diverse linguistic contexts.
3.2. TD as a predictive moderating mechanism: Cognitive ability reorganization and compensation
Our findings indicate that the contributions of cognitive abilities to L2 reading are not uniform; rather, their predictive power varies with L1–L2 TD, such that cognitive resources play a more discernible role in early processing under conditions of greater crosslinguistic divergence. This pattern underscores a process of cognitive resource reorganization and compensation, through which learners adapt their cognitive resources to the specific processing demands associated with crosslinguistic differences.
For Vietnamese learners, whose L1 is typologically closer to Chinese, cognitive abilities provided modest yet significant incremental explanatory power. Among these, VSTS capacity emerged as a robust independent predictor of longer FPRTs. This may suggest that in contexts of higher structural overlap, superior memory resources are engaged more efficiently for online semantic integration and prediction (Bråten et al., Reference Bråten, Haverkamp and Anmarkrud2025), though potential facilitation from crosslinguistic transfer cannot be ruled out (Momenian et al., Reference Momenian, Privitera and Weekes2023). The Lao and Thai groups, which occupy an intermediate position in terms of TD, showed predictive patterns and effect sizes that fell between those observed for the Chinese–Vietnamese and Chinese–Burmese groups, further supporting a continuous modulatory effect of TD.
In stark contrast, for Burmese learners facing a greater TD, none of the cognitive abilities significantly predicted reading performance in the main-effects model. The generally small effect sizes (|β| < 0.06) suggest that in highly divergent language pairs, the explanatory power of general cognitive abilities is diminished at the early stages of lexical processing. Instead, comprehension may rely more heavily on other resources, such as deeper lexical-semantic knowledge, pragmatic inferences and real-world knowledge.
The critical insight comes from the interaction analysis: VSTS and RCA were the only learner-level measures to show significant interactions with TD. Simple slope analyses confirmed that high VSTS capacity and RCA attenuated the processing disadvantage associated with greater L1–L2 distance. This finding highlights a specific compensatory mechanism: learners with greater VSTS and RCA appear better equipped to handle the form–function mapping challenges and syntactic mismatches characteristic of distant language pairs. This storage capacity likely facilitates the maintenance and integration of predictive information (Cheng et al., Reference Cheng, Yan, Yang and Lin2024; Ding et al., Reference Ding, Zhang, Liang and Li2023) while potentially suppressing crosslinguistic interference (Tan et al., Reference Tan, Martin and Van Dyke2017). Nevertheless, because our measure tapped only storage, the role of executive control processes in this effect remains to be examined.
Notably, the negative direction of all cognitive coefficients for the Burmese group, though nonsignificant, invites interpretation. This pattern may signal a strategic reallocation of cognitive resources: under high typological load, learners might invest more effort – drawing upon lexical knowledge and literacy skills – for deeper semantic encoding and successful comprehension, reflecting an adaptive, compensatory response to systemic linguistic challenges (Chung-Fat-Yim et al., Reference Chung-Fat-Yim, Calvo and Grundy2022; J. Yang et al., Reference Yang, Qi, Wang, Sun and Zheng2022).
In addition, complex synergistic relationships were observed among TD, cognitive abilities and LLM-based predictive measures. Across different levels of TD, RCA showed pronounced modulatory effects on the relationship between surprisal and contextual entropy, whereas VSTS capacity significantly modulated only contextual entropy, suggesting that its compensatory role is primarily expressed in alleviating processing costs associated with contextual uncertainty. These findings indicate that the impact of cognitive abilities on predictive processing is not only modulated by TD, but is also closely tied to specific stages of information processing, such as local prediction violations versus global contextual uncertainty. This pattern further supports the view that cognitive compensation mechanisms are both conditional and hierarchical, highlighting the dynamic roles of different cognitive resources in crosslinguistic predictive processing.
Collectively, these findings depict L2 reading efficiency as an emergent property of dynamic adaptation. Under conditions of closer typological proximity, the role of VSTS capacity and overall comprehension performance is more directly discernible in online processing. Under conditions of greater divergence, the system suggests its flexibility: learners reorganize their available resources, and storage capacities play a more pronounced role in navigating structural divergence. These results are broadly consistent with Cognitive Resource Coordination Model (Robinson, Reference Robinson2001), which proposes that language performance emerges from the coordination of multiple cognitive resources under varying processing demands. Extending this perspective, the present findings show that L2 reading reflects the joint influence of learner-level cognitive resources and crosslinguistic conditions, with L1–L2 TD shaping how these resources support predictive processing. Although based on adult L2 learners, these findings provide testable hypotheses for developmental research on bilingual children, while direct conclusions about children’s predictive processing require evidence from child populations.
3.3. Limitations and future directions
This study represents a systematic investigation into the interplay between LLM-derived predictive metrics and cognitive resources among understudied Asian language pairs, thereby broadening the crosslinguistic scope of predictive processing research. Nevertheless, several limitations suggest productive avenues for future research to extend these findings.
First, although FPRT serves as a reliable index of early lexical processing effort, it does not fully capture later-stage processes such as semantic integration, inference generation and comprehension monitoring. Our supplementary analyses of later eye-movement measures – TRT, RPD and proportion of FPR – showed that contextual entropy effects can emerge beyond initial lexical access. Notably, these later-stage effects were less systematically modulated by TD than those observed in FPRT. This divergence is consistent with the strictly incremental architecture of autoregressive LLMs, which lack mechanisms for look-back or reanalysis, and reinforces the theoretical rationale for prioritizing FPRT as the primary dependent measure.
Second, our measure of VSTS – a forward digit span task – captures VSTS rather than full WM, which requires simultaneous storage and processing (Conway, Reference Conway2005). Therefore, findings related to this measure should be interpreted as reflecting storage capacity, not executive control. Future studies should use backward digit span or complex span tasks (e.g., reading span, operation span) to assess WM more comprehensively.
Third, RCA is a composite measure influenced by multiple linguistic and cognitive processes, including lexical knowledge, inferential processing and text integration. Accordingly, the relevant findings should be interpreted as reflecting overall comprehension performance rather than any single mechanism. Future studies could use more targeted measures to distinguish the component processes involved.
Fourth, the psychological validity of surprisal and contextual entropy derived from a monolingual Chinese LLM remains an open question, particularly for lower-resource pairs such as Chinese–Burmese. Enhancing the crosslinguistic generalizability and cognitive plausibility of these metrics represents a crucial next step. Future research could explore hybrid modeling approaches that integrate explicit linguistic features (e.g., syntactic parses, semantic frames) into LLM-based estimates, or employ massively multilingual or expertly fine-tuned models trained on larger bilingual corpora to improve the transparency and consistency of computational predictions across diverse typological contexts.
Finally, although the present sample exhibited variability in both linguistic background and cognitive abilities, the overall sample size was relatively limited, which may constrain statistical power and the generalizability of the findings. Future research could enhance robustness and theoretical coverage by expanding sample sizes, adopting multi-site collaborative data collection and including a broader range of language pairings.
An additional limitation concerns the linguistic ecology of Southeast Asia, which is characterized by intensive language contact. Many participants in the present study had trilingual or multilingual backgrounds. Although they were treated analytically as learners of Chinese as a second language, experience with additional languages may have influenced their reading strategies, allocation of cognitive resources and sensitivity to TD. This factor may complicate interpretations that attribute observed effects solely to L1–L2 relationships.
In summary, this work introduces an integrative analytic approach for examining how TD modulates cognitive resource allocation during L2 reading. Promising directions emerging from this work include: (1) formally integrating linguistic theory with computational metrics to enhance the interpretability of LLMs; (2) adopting longitudinal designs to trace the co-development of cognitive abilities and predictive processing efficiency and (3) translating these insights into adaptive language instruction systems that are personalized to learners’ native-language backgrounds and cognitive profiles. By addressing these future directions, we can move toward a more nuanced and predictive science of second language acquisition.
4. Conclusion
This study examined the cognitive mechanisms underlying L2 predictive processing by comparing L1–L2 TD across four learner groups and integrating LLM-derived metrics with individual cognitive abilities. The results indicated that surprisal significantly accounted for processing difficulty across all L1 groups, whereas contextual entropy was initially significant only in the Vietnamese group. After adjustment for baseline cognitive differences, the entropy effect was no longer significant and a positive effect emerged in the Burmese group. Furthermore, cognitive abilities contributed incrementally to L2 reading efficiency beyond traditional linguistic predictors, with their effects systematically moderated by TD. Specifically, learners facing greater structural divergence relied more heavily on verbal storage capacity, whereas those with typologically closer L1s drew more efficiently on RCA and vocabulary knowledge. Critically, higher cognitive capacities enhanced processing efficiency under conditions of greater typological demand, revealing an ability-dependent compensatory mechanism. These findings refine Robinson’s (Reference Robinson2001) Cognitive Resource Coordination Model by demonstrating that cognitive compensation is highly specific to both the cognitive domain and the linguistic context. Together, these findings establish TD as a key moderating variable in L2 predictive processing and highlight the potential for adaptive instructional systems that are sensitive to learners’ crosslinguistic and cognitive profiles.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/S1366728926101904.
Data availability statement
All data needed to evaluate the conclusions of this article are available in the article and/or the Supplementary Material. The analysis code and a partial sample of the data supporting the findings of this study have been deposited in a public repository (https://osf.io/8hgbz). Due to confidentiality agreements, the full dataset is not publicly accessible but can be requested from the corresponding author for legitimate academic purposes.
Funding statement
This research was supported by the project “Construction of the Language and Cognition Laboratory at Guangxi University and Development of a Cognitive Processing Database for ASEAN Chinese Language Learners” (ZX02020032125003).
Competing interests
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Ethics approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.
Appendix A: Cognitive ability tests
A.1. Literacy
The literacy measure was adapted from the Character Literacy Scale for international students developed by Zhang et al. (Reference Zhang, Zhang, Zhang and Wang2021). The original scale comprises ten sets of items, with ten Chinese characters in each set (100 characters in total). In the present study, an equal-interval sampling procedure was applied: from each set, the first, fifth and tenth characters in the original test order were selected, yielding a final set of 30 characters for administration. Participants were asked to provide the pinyin and the meaning of each character. Meanings could be expressed either by writing a Chinese word containing the target character or by providing a pinyin-based compound. Scoring was conducted independently by two trained raters. The scoring criteria followed Zhang et al. (Reference Zhang, Zhang, Zhang and Wang2021). For each item, 1 point was awarded only when both pronunciation and meaning were correct; responses with only one component correct, both incorrect, or missing received 0 points. The maximum score was 30. For pronunciation scoring, an item was considered correct if both the onset (initial) and rime (final) were accurate; tone was not taken into account. For polyphonic characters, any correct pronunciation was accepted, provided that it was compatible with the meaning supplied by the participant. Meaning scoring adopted a relatively lenient criterion: responses were judged correct as long as the intended meaning was appropriate, and no points were deducted for errors in pinyin spelling or character writing. Colloquial expressions, formal written forms, internet slang, popular expressions, proper names or word-order reversals were all accepted if the meaning was judged correct. Completing this section of the test takes approximately 10 minutes.
A.2. Vocabulary size
Vocabulary size was assessed using LexCHI, a simplified-Chinese vocabulary test developed by Wen et al. (Reference Wen, Qiu, Leong and van Heuven2024). LexCHI is a lexical decision task designed to estimate learners’ Chinese vocabulary knowledge through their ability to recognize real words. The test includes 60 items in total, consisting of 40 two-character words and 20 pseudowords, presented in a fixed order. No time limit was imposed. Participants were instructed to judge, for each visually presented item, whether it was an existing Chinese word (e.g., 涟漪; response options: “Yes” vs. “No”). Scoring was based on accuracy: each correct judgment received 5 points, incorrect responses received 0 points and the maximum score was 300. Completing this section of the test takes approximately 10 minutes.
A.3. Nonverbal intelligence
Nonverbal intelligence was measured using the Culture Fair Intelligence Test (CFT 20-R; Weiß, Reference Weiß2006). The test is suitable for children and adolescents aged 8.5–19 years as well as adults aged 20–64 years and is designed to assess general nonverbal intelligence (Cattell, Reference Cattell1963). This construct reflects the ability to identify visuospatial relations of varying complexity and to solve formal reasoning problems under time constraints. The CFT 20-R is language-free and relies on intuitive task formats, thereby minimizing the influence of linguistic knowledge and cultural background. The full test comprises two parallel parts, each containing four subtests (series completion, classification, matrices and topological reasoning). Part 1 includes 56 items, whereas Part 2 includes 45 items and is more difficult. Depending on research purposes, the test can be administered in full or in a shortened version. In the present study, 12 figural reasoning items were selected for administration. Participants were required to infer the correct missing figure based on the patterns in the given stimuli. Each correct response was awarded 1 point; incorrect responses received 0 points. The maximum score was 12. Completing this section of the test takes approximately 10 minutes.
A.4. VSTS
VSTS was assessed with an adapted digit span task based on Robinson (Reference Robinson2002), which measures short-term memory capacity. In this task, participants were required to recall sequences of digits presented serially on the screen, and performance was indexed by the maximum span recalled correctly. Digit sequences were randomly generated by the program, such that each participant received a unique set of sequences. Digits were presented one at a time. After each sequence, participants immediately typed the digits in the same order. The task consisted of three practice trials and 14 test trials and followed an adaptive staircase procedure. The starting sequence length was three digits. If a participant responded correctly, the subsequent sequence increased by one digit; if the response was incorrect, the sequence length remained unchanged. If a participant made two consecutive errors at a given length, the subsequent sequence length was reduced by one digit (with a minimum of three digits). Scoring was based on trial-level accuracy (1 point per correctly recalled sequence, 0 otherwise). The primary outcome measure was the maximum digit length successfully recalled in the test trials. Completing this section of the test takes approximately 10 minutes.
Appendix B: Surprisal and entropy
B.1. Surprisal
Surprisal quantifies the amount of information conveyed by a word in a given context, that is, how unexpected or unpredictable the word is. It is typically operationalized as the negative log probability of the word given its preceding context (Hale, Reference Hale2001; Levy, Reference Levy2008). Surprisal has been widely used as an index of processing difficulty in language comprehension: higher surprisal values indicate that the word is less probable in the current context and therefore is assumed to require greater cognitive resources to process.
Following Wilcox et al. (Reference Wilcox, Pimentel, Meister, Cotterell and Levy2023), we defined word surprisal as
where
$ {w}_t $
denotes the word at position
$ t $
(i.e., the current word),
$ {w}_{<t} $
denotes the sequence of all preceding words (i.e.,
$ {w}_1,{w}_2,\dots, {w}_{t-1} $
) and
$ p\left({w}_t\mid {w}_{<t}\right) $
is the conditional probability of observing
$ {w}_t $
given the preceding context, typically estimated using a language model. The logarithm is usually computed either in the natural base
$ e $
or in base 2 (yielding units in bits).
B.2. Contextual entropy
Contextual entropy captures the expected difficulty of processing upcoming material before encountering the next word. It reflects the degree of uncertainty about what will follow in a given context and is typically computed from the conditional probability distribution over possible next words (Shannon, Reference Shannon1948). Higher entropy indicates that a larger set of continuations is plausible in the given context and that the probability mass is distributed more evenly across alternatives, thereby implying greater uncertainty and increased processing demands. Following Wilcox et al. (Reference Wilcox, Pimentel, Meister, Cotterell and Levy2023), we defined contextual entropy as
where
$ H\left({W}_t\mid {w}_{<t}\right) $
denotes the contextual entropy of the lexical variable at position
$ t $
given the preceding context,
$ {W}_t $
is a random variable ranging over possible words at position
$ t $
,
$ {w}_{<t} $
denotes the sequence of all preceding words and
$ V $
is the vocabulary (i.e., the set of all possible words). Here,
$ p\left(w\mid {w}_{<t}\right) $
denotes the conditional probability of word
$ w $
given the context.
B.3. Supplement
Following Haller et al. (Reference Haller, Bolliger and Jäger2024), word-level surprisal was computed by summing the surprisal values of the subword units that constitute the word. Specifically, for a word consisting of
$ k $
subwords
$ {w}_t,{w}_{t+1},\dots, {w}_{t+k} $
, its surprisal can be expressed as
By the chain rule, this can be rewritten as
which is equivalent to
For a word consisting of
$ k $
subword-level random variables
$ {W}_t,{W}_{t+1},\dots, {W}_{t+k} $
, their joint entropy can in principle be defined as
where
$ \varSigma $
denotes the set of possible subword units.
Open access
