R4.1. Processing: Sign us up for cognitive xenobiology!

Resnik and Nair and Phillips offer measured skepticism about what large language models can tell us about human language processing. They do not deny that LMs exhibit striking linguistic behavior, nor do they dismiss them as mere string-based heuristics. Rather, their concern is about the kinds of explanations LMs can legitimately support. Drawing on Marr’s levels of analysis (Marr, Reference Marr1982) in both cases, these commentaries emphasize that matching human behavior at the computational level does not by itself license conclusions about the algorithmic or implementational levels that underlie human sentence processing.

Resnik doubts the usefulness of LMs as cognitive models at all three of Marr’s levels. He says the implementation is so obviously different as to not be useful. There are indeed important differences, and we certainly agree that figuring out the right way to abstract away from implementation differences between models and humans is important work for the kind of program we lay out. Yet results from vision show striking parallels between, for example, neural networks and the visual system, as we discussed in our target article, and different architectures can converge to similar representational ideas, which then form good hypotheses for brain representations (Hosseini et al., Reference Hosseini, Casto, Zaslavsky, Conwell, Richardson and Fedorenko2024; Huh et al., Reference Hu, Mahowald, Lupyan, Ivanova and Levy2024). Furthermore, recent results with LMs show intriguing ways that details of the transformer architecture are more closely related than one might have expected to models of working memory in sentence processing (Ryu & Lewis, Reference Ryu and Lewis2025). So, while we agree with Resnik’s claim that modern AI engineering has moved away from its biologically inspired connectionist roots, we think the new implementations can still be informative.

At the computational level, Resnik seems to acknowledge that there may be behavioral parallels between models and people. But that by itself is, he says, unsatisfying because it might not be linkable to the algorithmic level: “There remains the burden of mapping any such computational theory to a biologically plausible algorithmic account of real-time processes distributed over diverse, specialized components.” So we take his most pressing concern to be about the algorithmic level. At the algorithmic level, he particularly worries about mechanistic exceptions that would undermine our claim that we should expect convergence to similar mechanisms. Nair and Phillips share this worry about mapping potentially powerful computational-level parallels onto algorithmic and implementational ones. They see LMs as (usefully) supercharging the project of using predictive measures to estimate linguistic difficulty and complexity. But, like Resnik, they see that project as distinct from the psycholinguists’ goal of “characterizing the underlying mental computations from each step of incremental processing.”

Resnik raises a useful thought experiment: imagine aliens come to Earth, learn to produce language by reading all the text ever produced, and we can probe their brains. He asks: “[W]hy in the world would one expect to find mechanistic correspondences between what’s happening in [the aliens’] boxes and what’s happening in the human cognitive apparatus?” We agree that the thought experiment is a good one for our current situation. But we don’t share the intuition that the alien brain would be so different from ours that it would have nothing to teach us. Partly, it seems that we have a different level of confidence in the expectation of convergence. Resnik doubts the contravariance principle in part because “[U]nlike visual object recognition, a key part of what makes “language” hard is that it decomposes into very distinct subproblems.” In contrast, we think the parallel to vision is apt and that the focus on breaking language down into sub-problems (e.g., first sound-level, then syntax, then semantics, then pragmatics) has been overemphasized in psycholinguistics, with much evidence pointing towards a much more unified processing of these sub-problems (MacDonald et al., Reference MacDonald, Pearlmutter and Seidenberg1994; Spivey & Tanenhaus, Reference Spivey and Tanenhaus1998; Tanenhaus et al., Reference Tanenhaus, Spivey-Knowlton, Eberhard and Sedivy1995; Trueswell, Tanenhaus, & Garnsey, Reference Trueswell, Tanenhaus and Garnsey1994).

Of course, just because convergence is possible in theory doesn’t mean that it really does happen. This caution motivates us to want to do more work in this domain, as we think both Resnik and Nair and Phillips would endorse, and proceed with scientific rigor. And we agree that one must be explicit about which level of explanation one is targeting and what evidence would bear on it. But, partly, it seems that we have a different reading of the last few years of linguistics-focused LM research. We think there is already a growing and important body of work (summarized in our target article) finding interesting parallels between the kinds of representations posited in linguistics and the kinds of representations emergent in neural models. As we discuss in Section R4.3.3, there is philosophical and cognitive work to do to learn how to think about the mappings across these very different systems. But we think a picture is emerging.

As with any scientific project, there are likely to be unexpected findings and developments along the way. But we find it hard to imagine a scenario where probing the linguistically competent aliens’ brains didn’t give us insight, one way or another, into human cognition. That can be true even though there will surely be divergences. Resnik cites as a cautionary tale the divergent mechanisms for prey capture in sharks (electroreception) and dolphins (echolocation). But, if there were dolphin scientists studying prey capture, and suddenly a bunch of sharks with highly probeable brains landed in the dolphin scientists’ laps, we think they would learn a lot by studying the sharks and modeling how a similar computational goal is realized through a different mechanism. It would force them to interrogate their assumptions about what is required for prey capture, how different mechanisms can work, whether there is a level of abstraction at which the dolphin mechanism can be mapped onto the shark one, and whether the different mechanisms lead to different prey-capture behavior. The discovery of sharks would be a great opportunity for the dolphin scientists, although we would recommend they proceed with care.

In short, now that the aliens are here, we think it would be scientifically negligent if no team of scientists tried to probe their language ability – and futile if that team did not involve linguists. So, sign us up for cognitive xenobiology!

R4.2. Learning: Reshaping, not denying, the learning problem

The responses also give us a chance to clarify our stance on innateness and inductive bias. Neural LMs succeed in learning language in ways that no previous system has, and so – even if the trajectory of their learning does not match humans’ (McDermott-Hinman and Feiman, Howitt and Lidz) – understanding how they learn is relevant for understanding how language learning is possible. We wrote: “Far from demoting inductive bias as a concern, language models open up the range of (possibly innate) inductive biases that we might look for in humans. The main point for linguistic theory is not to demote the importance of the learning problem, but to reshape it: away from explanation by constrained description, and toward a broader landscape of approaches and hypotheses.”

The central point is not that LMs argue against innate biases, but rather that those biases can look very different from what much of the generative linguistics literature assumed they had to look like. Inductive biases in large neural networks are less like an elegant discrete grammar formalism where the learner just needs to flip a few categorical switches and the rest follows deductively (Berwick et al., Reference Berwick, Pietroski, Yankama and Chomsky2011; Chomsky, Reference Chomsky1981; Gibson & Wexler, Reference Gibson and Wexler1994). They are more like a collection of soft constraints that arise from a variety of sources. Some of these sources are domain-specific to language, and some of them are domain-general constraints that are useful for a variety of tasks. At the computational level, the inductive biases in large neural networks look less like a hard delimitation of possible hypotheses and more like the kinds of soft Bayesian priors that have appeared in computational modeling work on language acquisition (Hsu, Chater, & Vitányi, Reference Hsu, Chater and Vitányi2011; Perfors et al., Reference Perfors, Tenenbaum and Regier2013) – which may well be innate and language-specific (Pearl & Sprouse, Reference Pearl, Sprouse, Sprouse and Hornstein2013)!

Rawski and Heinz doubt that this idea of soft versus hard constraints in learning makes sense – indeed they question the idea that constraints can be domain-general or domain-specific at all. They point out, for example, that any learner operates under the hard constraint that it has a fixed computable hypothesis class and that the inductive biases we surveyed are specific to certain neural architectures, making them putatively domain-specific. Yet their discussion seems to use these terms in nonstandard ways that make them trivial. Indeed, neural learners cannot entertain uncomputable hypotheses – in this sense, all possible learners trivially have a hard constraint. But what is interesting about learning in large neural networks, and what was surprising even to statistical learning theorists (Zhang et al., Reference Zhang, Bengio, Hardt, Recht and Vinyals2021), is that the hard limits of their hypothesis space are less important for determining their inductive biases than the soft constraints that are imposed inside that space. This was a nontrivial discovery about how learning can work – although it has since been understood retroactively in terms of existing theories (Wilson, Reference Wilson2025) – and it should cause us to broaden our view of how human learning can work as well. Similarly, neural LMs clearly have biases that depend on their architectures, but nonetheless, they are domain-general learners in the standard sense of the term from cognitive science: their biases do not limit them to learning language (Clark & Lappin, Reference Clark and Lappin2010a; Pearl & Sprouse, Reference Pearl, Sprouse, Sprouse and Hornstein2013; Wilcox et al., Reference Wilcox, Futrell and Levy2024). If we take “domain-specific” to mean dependent on architecture as Rawski and Heinz seem to mean it, then all learners are trivially domain specific. This way of framing things does not seem productive.

Indeed, the domain-generality of neural LMs forms another objection to the idea that they can be useful for how we think about learning. Some would have it that if an LM could learn an “impossible language” – one violating constraints like a bias against grammar rules based on linear order – just as well as an attested human language, it would suggest LMs are so deeply unhuman-like that they are uninformative about human language. Bowers and Mitchell, Bolhuis et al., Murphy et al., Wilcox and Newport, and Kallini and Potts address this view from different perspectives.

Bowers and Mitchell claim that Kallini et al. (Reference Kallini, Papadimitriou, Futrell, Mahowald, Potts, Ku, Martins and Srikumar2024) found only two impossible languages harder than English and concluded no linguistic priors are needed to account for LM learning. This mischaracterizes both the findings and the claims in Kallini et al. (Reference Kallini, Papadimitriou, Futrell, Mahowald, Potts, Ku, Martins and Srikumar2024): they found that all tested impossible variants were harder to learn than English and explicitly attributed the gap to linguistic inductive biases – that is, inductive biases that are aligned with the structure of language, which may be either specific to language or general to many different tasks. Similarly, we claimed in the target article that LMs have inductive biases that are aligned with the structure of language and that discovering what these biases are is a fruitful endeavor for linguistics.

None of this implies that current LMs share all inductive biases with humans or that they can only acquire human language. Nor is that a requirement for them to be informative for linguistics. The inductive biases we discuss in the target article (information locality and low sensitivity) – as well as other inductive biases in neural LMs – seem to match real properties of language that were either not known before or not emphasized and that enable learning. Contra Bowers and Mitchell, Murphy et al., and Bolhuis et al., learning only possible languages is not a prerequisite for a model and its inductive biases to be informative. It is not even clear that humans have this property; subjects in Musso et al. (Reference Musso, Moro, Glauche, Rijntjes, Reichenbach, Büchel and Weiller2003) learned the impossible variant of Italian to a similar level of accuracy and at a similar rate as the real language. There is no tension between being a domain-general learner and having biases that align with language: the biases that make LMs good at language, such as information locality, may be useful for many other tasks.

On the “impossible languages” question, we favor the characterization in Kallini and Potts (disclosing that we are coauthors with them on Kallini et al. (Reference Kallini, Papadimitriou, Futrell, Mahowald, Potts, Ku, Martins and Srikumar2024), a key study in this debate): the productive question is not whether LMs can or cannot learn impossible languages but what the pattern of their successes and failures reveals about which inductive biases support language learning. This reframing turns the impossible-language debate from a litmus test into a research program.

The upshot for theories of learning is that LMs do not argue against innate biases, but they argue against the view that the biases required for language learning must be narrowly linguistic and categorically hard. The success of domain-general learners at acquiring core properties of language tells us that the prior overgrammars can be softer – and broader in origin – than previously assumed. Understanding exactly what that prior looks like is, we think, one of the most exciting questions that LM research opens up for linguistics.

R4.3. Structure: Generative AI isn’t anti-generative – but it doesn’t do generative linguistics any favors

R5.1. Learning: Data size, data diversity, and acquisition-relevance

We agree with Ambridge that “disruption is mutually assured” between LMs and research in language acquisition. Ambridge and a number of other commentaries draw our attention to data used to train models and how it compares (unfavorably in richness, although favorably in number of words) to the input a child gets. Lampinen highlights data diversity and the relative lack of diversity in internet text data compared to the kind of data children learn from. Wilcox and Newport call for a need for “human-sized data” and discuss the ways that LMs can help us understand how “graded, fuzzy constraints” interact during language acquisition, leading to a richly realized complex system. Howitt and Lidz and particularly McDermott-Hinman and Feiman discuss the ways that LM learning trajectories fundamentally differ from those of human learners.

Studying these aspects of how models learn is a promising research program. Indeed, we think efforts, inspired by child language learning, to train smaller models (Warstadt et al., Reference Warstadt, Mueller, Choshen, Wilcox, Zhuang, Ciro, Mosquera, Paranjabe, Williams, Linzen and Cotterell2023) under various manipulations (Misra & Mahowald, Reference Misra, Mahowald, Al-Onaizan, Bansal and Chen2024; Patil et al., Reference Patil, Jumelet, Chiu, Lapastora, Shen, Wang, Willrich and Steinert-Threlkeld2024; Yao et al., Reference Yao, Misra, Weissweiler and Mahowald2025) and study learning have been one of the most promising contributions of linguistics to LM research. ^{Footnote 4} The commentators lay out promising avenues for further work in these domains. If it turns out that certain linguistic structures are fundamentally not amenable to being learned on human-scale data or that learning trajectories are fundamentally not recoverable by models, that will be an informative data point.

R5.2. Beyond standard written English: Multilinguality, dialects, interaction

Several commentators suggest that, by arguing for the role of LMs in linguistics, we privilege the role of standard written English, particularly syntax and semantics. Tripp writes, “LMs as understood in this frame effectively erase ways of languaging which are not readily captured in large repositories of text strings. Multimodal signals, admixtures of codes and the use of sign languages are all ignored, implicitly treated as complexity which is justifiably lost in message ‘compression.”

While much of this is a fair criticism of current LM research, the arguments we make in our target article also apply in principle to future models trained on non-standardized dialects, on audio, on sign language, on admixtures of codes, and on multimodal interactive systems. Model building can and should encompass these domains. And we see the body of work on written English not as the endpoint, but as a playbook for future work in all of these areas.

We do agree with Tripp that, in our review of the literature, we overfocused on syntax at the expense of other aspects of “languaging.” We did so largely both because of the historical importance of syntax in linguistics and cognitive science and also because that’s been the locus of most linguistic research on LMs (probably for the same reason: historical import). We enthusiastically endorse the pleas to push beyond the current boundaries. Below, we outline a few ways we think that can and will happen.

Non-English languages, like Tripp and Ármannsson et al., point to failures of LMs in other languages, particularly highlighting Icelandic. Modern LM architectures likely favor English (Blasi, Anastasopoulos, & Neubig, Reference Blasi, Anastasopoulos, Neubig, Muresan, Nakov and Villavicencio2022), and more engineering effort has gone into English-language models. These are genuine shortcomings. We see expanding the scope of languages under study as one of the highest-priority items on the research agenda, both as an ethical imperative (since we want systems that work for everyone) and a scientific one (since we want to know how generalizable LM-based results are across languages and how typological differences interact with inductive biases, model performance, learning trajectories, and the like).

Having said that, we do not think that the current narrowness of LM-based results should necessarily be taken to make the extant body of LM work uninformative about language in general. For instance, consider the claim that English LMs deliver proof of concept that language in general can be learned by general-purpose learners with weak inductive biases. It would be scientifically surprising if English were learnable by large neural systems while typologically different languages were not learnable even in principle (even though other languages might benefit from different architectures or training regimes). A long-held tenet of linguistics is that all languages are learnable by children on roughly similar timescales. Thus, if English were uniquely amenable to statistical approaches, whereas other world languages were impervious to statistical learning, we would find that extremely surprising and thus extremely informative.

Audio We agree with de Heer Kloots et al. that linguists should also learn to love audio-based LMs. Doing so would extend the insights of this research program into the auditory domain, with import for phonetics and phonology. There is a lot of work left to be done towards this goal. At the same time, while text data lacks the multimodal richness of child input, we think it’s sometimes understated how rich it is. While traditional corpora used in corpus linguistics came from news text and books, modern LMs are often trained on almost the whole internet. This includes all kinds of formal writing but also lots of informal writing: texts, web posts, and so on. In many cases, these data sources approximate informal real-time conversations, using emoji or other cues. So, while extending LM work to audio is an important future goal, we again do not think the extant literature is radically undermined by being text-focused.

Interaction and Communication Several commentators pointed out that there is something sterile about LMs: they lack the social, embodied, goal-laden usage that is the hallmark of human language. In that vein, we are sympathetic to much of Bunzeck et al.’s argument that LMs find patterns in data but are not fundamentally as “usage-based” as we say since they lack interaction. Culbertson et al. make a related point, drawing on the language evolution literature to argue that communication provides a crucial counteracting pressure that prevents language from collapsing into highly predictable but non-functional systems. The extent to which LMs are subject to these pressures is unclear. We find this perspective compelling and agree that the communicative function of language is essential to any complete explanatory account.

There is something off about merely learning from co-occurrence statistics, without having rich communicative goals. And it’s true that classical pretrained models fall short in that way. But, as we move past the era of text-based pretrained models (as we largely have), we are optimistic that these limitations can be overcome.

As such, we see the lack of interaction and communication not as a strict limit, but as a demonstration that the right training regime matters. Regimes incorporating communicative objectives – whether through reinforcement learning from human feedback, multi-agent interaction, or grounded language tasks – are the path forward. For instance, with the rise of AI agents, we expect to see a sharp increase in the amount of linguistics-focused work exploring AI behavior in much richer settings. While simple next-word predictors might not have goals or intentions of their own, an AI agent may well inhabit a setting that requires them to have goals (e.g., helping a user write successful code or complete a task). These agents may well have the kind of interactive, communicative setting that some commentators found lacking. Progress in AI is happening so fast that studying linguistic behavior only in non-communicative next-word prediction machines may soon look quaint.

In that vein, we share Sripada et al.’s optimism about the ability of LM-based linguistics to make contact with not just how an agent says something but also how it knows what to say when. They revive an old and somewhat dormant idea in linguistics: the Creative Aspect of Language Use (CALU), the ability of language users to select what they will say out of all the possible utterances they could say. We share Sripada et al.’s intuition that CALU has lain dormant, in part, because the problem was just too hard for the methods we had. And we agree modern AI models may change that. While there has been a lot of focus on text-based LMs, AI models in general can (and now often do) have visual components, information from various other modalities, and rich representations of the world. They are ready-made for grounding since all of these inputs can exist in a shared vector space, right along with the language. In that sense, “saying the right thing” can become a matter of saying the right thing conditioned on a particular non-linguistic state. We think this plausibly makes much richer modeling of choosing utterances “appropriate to the situation and linguistic context” far more tractable.