The supposed “end of linguistic theory,” allegedly superseded by language models (LMs), has been a recurring topic of debate. Futrell and Mahowald (F&M) offer a different perspective, one that is more conciliatory and readily acceptable: LMs do not constitute a complete theory of language, but they provide a wealth of information about it. Given this, what should be the scope and aims of linguistic theory in the era of “triumphing LMs,” and how should the two domains be properly related?
F&M point out that LMs are “models of the stream of language produced and comprehended by humans.” However, strictly speaking, they are even less than that: they are just models of the stream of linguistic forms produced by humans, to which they attach semantic content to the extent this is recoverable from distributional statistics. After all, as they correctly say, LMs are the most advanced product of distributional semantics (Lenci & Sahlgren, Reference Lenci and Sahlgren2023). On the other hand, the goal of linguistics should be more than form sequence modelling. Its aim has always been to model natural language as a potentially infinite system of signs intended as form-meaning pairs (de Saussure, Reference de Saussure1916). Linguistic theory must therefore be a theory of how meaning representations are mapped back and forth onto linguistic forms for the purpose of communication. Similarly, the goal of a theory of language learning is to model how this mapping is acquired from a finite and relatively small amount of communicative experiences. Generative linguistics might be limited in pursuing this goal only with reference to language as an abstract system, as F&M argue, but other linguistic theories that take a very different perspective, such as the Parallel Architecture (Jackendoff, Reference Jackendoff2002) and Construction Grammar (Goldberg, Reference Goldberg2019), also assume that the aim of linguistics is modelling language as form-meaning mappings. Therefore, however substantial the potential contribution of LMs may be, it must be understood within this broader goal of linguistic theory.
F&M advocates a “decentralised” approach to linguistic inquiry, according to which partial insights drawn from other sciences and technologies, such as LMs, will gradually reveal the contours of the linguistic object, like an unknown continent to be charted or a jigsaw puzzle to be assembled. As they write, “We will not answer the most fundamental questions about language by focusing only on language, using only methods that are tailor-made for linguistics.” Within this perspective, the role of linguistic theory might be intended as just one of reverse engineering: The inner working of the first “language machines” will lead to understanding how language works. Although we concur with the claim that some insights about language will come from the analysis of LMs, we believe this is not the best possible setting for a fruitful dialogue between linguistics and LMs, given the very nature of linguistics and its goals, as stated above.
First of all, linguistics occupies a special status among the sciences. According to De Saussure, what distinguishes linguistics is precisely that its theoretical viewpoint constitutes its own object of inquiry. This principle is also shared by Zellig Harris (Reference Harris1991), one of the founders of the distributional paradigm that grounds the way in which LMs acquire linguistic categories. Unlike physics or biology, linguistics does not confront pre-existing natural objects, it creates its own epistemic domain through concepts such as sign, structure, and meaning. The search for physiological or psychological correlates of linguistic phenomena in the brain and mind already presupposes a prior definition of the linguistic entity for which the correlate is sought. Whatever physiological or psychological phenomenon may be observed, it cannot, by definition, constitute a linguistic phenomenon, which is linguistic in nature by the very premises of the inquiry. Similarly, the reverse engineering of LMs requires the analysis of how linguistic categories are encoded in their architectures, which in turn presupposes a prior definition of the linguistic entities and structures we aim to seek in the models’ internal states.
Secondly, deriving a scientific theory about a certain function from a machine’s behaviour is only possible if this function has been truly realised by the machine. The physical theory of aerodynamics has greatly benefited from the study of airplanes, but this was possible because such machines were indeed able to fly. As we said above, LMs are just machines that produce strings of linguistic forms that closely approximate the ones produced by humans, but do not really master language in the true sense of performing a mapping between forms and meanings. For instance, as Mahowad et al. (Reference Mahowald, Ivanova, Blank, Kanwisher, Tenenbaum and Fedorenko2024) convincingly argue, there is a mismatch between the formal and the functional competences of LMs. Actually, this mismatch is not a secondary issue, but a necessary consequence of there being distributional devices that piggyback all meaning onto form co-occurrences. We might regard it as another case of the “embers of autoregression” of LMs qua LMs (McCoy et al., Reference McCoy, Yao, Friedman, Hardy and Griffiths2023).
What then is the role of LMs for linguistic theory? We agree with F&M that LMs have great potential for linguistics. We think that LMs provide a new “tribunal of experience” (Quine, Reference Quine and Quine1953) to which linguistic theory must be submitted. However, it does not follow from this that we should develop linguistic theories just by reverse engineering LMs. It is instead necessary that independently developed hypotheses and theories about language be validated on computational models like LMs. In this light, LMs can become a powerful means for falsifying core conceptual systems that make linguistics a science in the first place. However, this is possible only under two conditions: 1.) linguistic theory keeps its autonomy from LMs, very much like theoretical physics is independent from the experimental methods validating it; 2.) linguistics and LMs must grow closer than they currently are. This calls for a reciprocal effort: Linguistics must accept LMs as empirical judges of its theories, but LMs must increase their plausibility as experimental testbeds for natural language models in the way they are trained and designed. If this encounter happens, the dialogue will be incredibly beautiful.
You have
Access
The supposed “end of linguistic theory,” allegedly superseded by language models (LMs), has been a recurring topic of debate. Futrell and Mahowald (F&M) offer a different perspective, one that is more conciliatory and readily acceptable: LMs do not constitute a complete theory of language, but they provide a wealth of information about it. Given this, what should be the scope and aims of linguistic theory in the era of “triumphing LMs,” and how should the two domains be properly related?
F&M point out that LMs are “models of the stream of language produced and comprehended by humans.” However, strictly speaking, they are even less than that: they are just models of the stream of linguistic forms produced by humans, to which they attach semantic content to the extent this is recoverable from distributional statistics. After all, as they correctly say, LMs are the most advanced product of distributional semantics (Lenci & Sahlgren, Reference Lenci and Sahlgren2023). On the other hand, the goal of linguistics should be more than form sequence modelling. Its aim has always been to model natural language as a potentially infinite system of signs intended as form-meaning pairs (de Saussure, Reference de Saussure1916). Linguistic theory must therefore be a theory of how meaning representations are mapped back and forth onto linguistic forms for the purpose of communication. Similarly, the goal of a theory of language learning is to model how this mapping is acquired from a finite and relatively small amount of communicative experiences. Generative linguistics might be limited in pursuing this goal only with reference to language as an abstract system, as F&M argue, but other linguistic theories that take a very different perspective, such as the Parallel Architecture (Jackendoff, Reference Jackendoff2002) and Construction Grammar (Goldberg, Reference Goldberg2019), also assume that the aim of linguistics is modelling language as form-meaning mappings. Therefore, however substantial the potential contribution of LMs may be, it must be understood within this broader goal of linguistic theory.
F&M advocates a “decentralised” approach to linguistic inquiry, according to which partial insights drawn from other sciences and technologies, such as LMs, will gradually reveal the contours of the linguistic object, like an unknown continent to be charted or a jigsaw puzzle to be assembled. As they write, “We will not answer the most fundamental questions about language by focusing only on language, using only methods that are tailor-made for linguistics.” Within this perspective, the role of linguistic theory might be intended as just one of reverse engineering: The inner working of the first “language machines” will lead to understanding how language works. Although we concur with the claim that some insights about language will come from the analysis of LMs, we believe this is not the best possible setting for a fruitful dialogue between linguistics and LMs, given the very nature of linguistics and its goals, as stated above.
First of all, linguistics occupies a special status among the sciences. According to De Saussure, what distinguishes linguistics is precisely that its theoretical viewpoint constitutes its own object of inquiry. This principle is also shared by Zellig Harris (Reference Harris1991), one of the founders of the distributional paradigm that grounds the way in which LMs acquire linguistic categories. Unlike physics or biology, linguistics does not confront pre-existing natural objects, it creates its own epistemic domain through concepts such as sign, structure, and meaning. The search for physiological or psychological correlates of linguistic phenomena in the brain and mind already presupposes a prior definition of the linguistic entity for which the correlate is sought. Whatever physiological or psychological phenomenon may be observed, it cannot, by definition, constitute a linguistic phenomenon, which is linguistic in nature by the very premises of the inquiry. Similarly, the reverse engineering of LMs requires the analysis of how linguistic categories are encoded in their architectures, which in turn presupposes a prior definition of the linguistic entities and structures we aim to seek in the models’ internal states.
Secondly, deriving a scientific theory about a certain function from a machine’s behaviour is only possible if this function has been truly realised by the machine. The physical theory of aerodynamics has greatly benefited from the study of airplanes, but this was possible because such machines were indeed able to fly. As we said above, LMs are just machines that produce strings of linguistic forms that closely approximate the ones produced by humans, but do not really master language in the true sense of performing a mapping between forms and meanings. For instance, as Mahowad et al. (Reference Mahowald, Ivanova, Blank, Kanwisher, Tenenbaum and Fedorenko2024) convincingly argue, there is a mismatch between the formal and the functional competences of LMs. Actually, this mismatch is not a secondary issue, but a necessary consequence of there being distributional devices that piggyback all meaning onto form co-occurrences. We might regard it as another case of the “embers of autoregression” of LMs qua LMs (McCoy et al., Reference McCoy, Yao, Friedman, Hardy and Griffiths2023).
What then is the role of LMs for linguistic theory? We agree with F&M that LMs have great potential for linguistics. We think that LMs provide a new “tribunal of experience” (Quine, Reference Quine and Quine1953) to which linguistic theory must be submitted. However, it does not follow from this that we should develop linguistic theories just by reverse engineering LMs. It is instead necessary that independently developed hypotheses and theories about language be validated on computational models like LMs. In this light, LMs can become a powerful means for falsifying core conceptual systems that make linguistics a science in the first place. However, this is possible only under two conditions: 1.) linguistic theory keeps its autonomy from LMs, very much like theoretical physics is independent from the experimental methods validating it; 2.) linguistics and LMs must grow closer than they currently are. This calls for a reciprocal effort: Linguistics must accept LMs as empirical judges of its theories, but LMs must increase their plausibility as experimental testbeds for natural language models in the way they are trained and designed. If this encounter happens, the dialogue will be incredibly beautiful.
Acknowledgments
None.
Financial support
We acknowledge financial support under the PRIN 2022 Project Title “Computational and linguistic benchmarks for the study of verb argument structure” – CUP I53D23004050006 – Grant Assignment Decree No. 1016 adopted on 07/07/2023 by the Italian Ministry of University and Research (MUR). This work was also supported under the PNRR – M4C2 – Investimento 1.3, Partenariato Esteso PE00000013 – FAIR – Future Artificial Intelligence Research – Spoke 1 “Human-centered AI,” funded by the European Commission under the NextGeneration EU programme.
Competing interests
None.