Dennett’s idea of real patterns connects emergent ontology with projectible regularities in the phenomena. In Dennett’s example of the Game of Life, there are emergent “entities” (or patterns), such as eaters and gliders, whose behaviour is predictable. In human behaviour, there are real patterns too in the form of the entities of folk psychology – for example, beliefs and desires – and according to Dennett, the question of whether such things exist is exhausted by the question of whether there are explanatorily and predictively successful causal or lawlike regularities expressed in terms of them (in which they play a non-redundant role). Dennett’s idea has been applied much more widely recently, for example, by David Wallace to the emergent ontology of fluid mechanics (Wallace, Reference Wallace, Millhouse, Petersen and Ross2026), and Margarida Hermida to biological classification (Hermida, Reference Hermida, Millhouse, Petersen and Ross2026). It has been applied to linguistic phenomena by Ryan Nefdt (Reference Nefdt2023). The key aspect of real patterns for present purposes is that they depend on there being a modal structure to the relevant phenomena that is represented by a scientific theory or model. Philosophers of science have studied scientific modelling of many kinds in great depth, and there are two important lessons that have been learned pertinent to the debate about large language models and natural language. The first is that models can represent modal structure approximately and incompletely, without there being an isomorphism between the structure of the domain and the model. The second, which is a corollary of the first, is that there can be multiple models of the same domain that represent different aspects of its modal structure.
Futrell and Mahowald ingeniously apply the idea of real patterns to deep learning language models (LMs), prominent in current natural language processing (NLP). They argue that these models can reveal linguistic structure via simple compressed representations of past regularities found in natural language data. Their account aims to circumnavigate the charybdis of formal linguistics, which has persistently rejected the importance of statistical methods, and the scylla of modern NLP, which shows an inclination to interpret the successes of LMs as refutations of said formal linguistic theory. We commend their effort, but one important question remains unaddressed namely, what is the domain of these real patterns? In particular, are they patterns in linguistic outputs or are they patterns in cognition?
Formal linguists, following Chomsky, emphasise the cognitive-explanatory nature of the science above all else. If there are real patterns to be found, they likely lurk surreptitiously in the mind of cognisers. Compressibility will then require formal, discrete mathematical tools (like proof-theoretic generative grammars). In contrast, computational linguistics has always been performance-based, and in fact, Futrell and Mahowald do see LMs as vindications of usage-based theories of language. Here, big data, statistics and more recently, deep neural networks reveal structure, of which the categories of linguistic theory might only be useful and lossy representations. But Futrell and Mahowald seem to see the Dennettian picture as that of linguistic structure (e.g. “grammatical subject”) being at the level of belief or folk psychology and the messy underlying circuitry living at the level of neural nets, big data, and statistics. However, if we return to the Game of Life, it seems to be the other way around, the underlying reality is characterised by simple discrete rules (like a cell stays alive if it has 2 or 3 live neighbours on a 2D grid), and the emergent complexity of gliders, eaters, and pulsars is to be found at higher levels. It marks a genuine phase transition. If this analogy applies, then their argument is reversed. Formal linguistic structures are not necessarily emergent real patterns but deeper characterisations of a deterministic universe. This actually corresponds with the idea that LMs are prediction machines. What makes intentional vocabulary (beliefs, desires, intentions) useful and ineliminable (for Dennett) is that it provides us with mechanisms for the successful prediction of behaviour. The formal theory furnishes the fundamental physics. But here too there is a caveat since LMs often do not provide their predications in a human transparent manner (although perhaps neither do brains, see Hasson, Nastase, & Goldstein, Reference Hasson, Nastase and Goldstein2020), and formal linguistics, unlike physics, does not always deliver on the prediction or implementation side.
Of course, it is possible that nature endowed human beings with a redundant linguistic architecture such that both formal grammar rules and the statistical approximations of LMs can represent linguistic structure to different degrees. This would obviate the need to find the actual or underlying modal base of linguistic structure (i.e. is natural language discrete or continuous in essence?). Competence and performance would be bound up in a complex system. Perhaps, as F&M suggest, LMs can serve as proofs of concept or delimiters of the modal space of possible linguistic structure, within which human representational capabilities operate. It really depends on what kinds of models LMs are. It is also possible that, with relation to language, we simply have “multiple models idealisation” (Weisberg, Reference Weisberg2013), similar to modelling practices in sciences like climate science. Multiple incompatible representations and resources track different patterns in accordance with distinct scientific purposes and goals. Without a clear path to connecting the two paradigms in linguistic theorising, it seems unclear how real patterns can unify or unite formal linguistics with NLP and engineering.
This all comes back to our initial question: where do the real patterns reside? Initially, both formal symbolic grammar and the subsymbolic nature of connectionism were searching for them within cognition, such that classic proposals like Smolensky (Reference Smolensky1986) aim at reconciliation within one framework. Or perhaps something akin to autoregression is at the heart of human language competence as well (Caucheteux & King, Reference Caucheteux and King2022)? Either way, we have to contend with the fact that modern LMs are not designed to explain or uncover cognitive structure explicitly; they are engineering devices created to optimise a next token prediction task. It remains to be seen if the patterns they so expertly and superhumanly survey are the real ones in the heads of language users.
You have
Access
Dennett’s idea of real patterns connects emergent ontology with projectible regularities in the phenomena. In Dennett’s example of the Game of Life, there are emergent “entities” (or patterns), such as eaters and gliders, whose behaviour is predictable. In human behaviour, there are real patterns too in the form of the entities of folk psychology – for example, beliefs and desires – and according to Dennett, the question of whether such things exist is exhausted by the question of whether there are explanatorily and predictively successful causal or lawlike regularities expressed in terms of them (in which they play a non-redundant role). Dennett’s idea has been applied much more widely recently, for example, by David Wallace to the emergent ontology of fluid mechanics (Wallace, Reference Wallace, Millhouse, Petersen and Ross2026), and Margarida Hermida to biological classification (Hermida, Reference Hermida, Millhouse, Petersen and Ross2026). It has been applied to linguistic phenomena by Ryan Nefdt (Reference Nefdt2023). The key aspect of real patterns for present purposes is that they depend on there being a modal structure to the relevant phenomena that is represented by a scientific theory or model. Philosophers of science have studied scientific modelling of many kinds in great depth, and there are two important lessons that have been learned pertinent to the debate about large language models and natural language. The first is that models can represent modal structure approximately and incompletely, without there being an isomorphism between the structure of the domain and the model. The second, which is a corollary of the first, is that there can be multiple models of the same domain that represent different aspects of its modal structure.
Futrell and Mahowald ingeniously apply the idea of real patterns to deep learning language models (LMs), prominent in current natural language processing (NLP). They argue that these models can reveal linguistic structure via simple compressed representations of past regularities found in natural language data. Their account aims to circumnavigate the charybdis of formal linguistics, which has persistently rejected the importance of statistical methods, and the scylla of modern NLP, which shows an inclination to interpret the successes of LMs as refutations of said formal linguistic theory. We commend their effort, but one important question remains unaddressed namely, what is the domain of these real patterns? In particular, are they patterns in linguistic outputs or are they patterns in cognition?
Formal linguists, following Chomsky, emphasise the cognitive-explanatory nature of the science above all else. If there are real patterns to be found, they likely lurk surreptitiously in the mind of cognisers. Compressibility will then require formal, discrete mathematical tools (like proof-theoretic generative grammars). In contrast, computational linguistics has always been performance-based, and in fact, Futrell and Mahowald do see LMs as vindications of usage-based theories of language. Here, big data, statistics and more recently, deep neural networks reveal structure, of which the categories of linguistic theory might only be useful and lossy representations. But Futrell and Mahowald seem to see the Dennettian picture as that of linguistic structure (e.g. “grammatical subject”) being at the level of belief or folk psychology and the messy underlying circuitry living at the level of neural nets, big data, and statistics. However, if we return to the Game of Life, it seems to be the other way around, the underlying reality is characterised by simple discrete rules (like a cell stays alive if it has 2 or 3 live neighbours on a 2D grid), and the emergent complexity of gliders, eaters, and pulsars is to be found at higher levels. It marks a genuine phase transition. If this analogy applies, then their argument is reversed. Formal linguistic structures are not necessarily emergent real patterns but deeper characterisations of a deterministic universe. This actually corresponds with the idea that LMs are prediction machines. What makes intentional vocabulary (beliefs, desires, intentions) useful and ineliminable (for Dennett) is that it provides us with mechanisms for the successful prediction of behaviour. The formal theory furnishes the fundamental physics. But here too there is a caveat since LMs often do not provide their predications in a human transparent manner (although perhaps neither do brains, see Hasson, Nastase, & Goldstein, Reference Hasson, Nastase and Goldstein2020), and formal linguistics, unlike physics, does not always deliver on the prediction or implementation side.
Of course, it is possible that nature endowed human beings with a redundant linguistic architecture such that both formal grammar rules and the statistical approximations of LMs can represent linguistic structure to different degrees. This would obviate the need to find the actual or underlying modal base of linguistic structure (i.e. is natural language discrete or continuous in essence?). Competence and performance would be bound up in a complex system. Perhaps, as F&M suggest, LMs can serve as proofs of concept or delimiters of the modal space of possible linguistic structure, within which human representational capabilities operate. It really depends on what kinds of models LMs are. It is also possible that, with relation to language, we simply have “multiple models idealisation” (Weisberg, Reference Weisberg2013), similar to modelling practices in sciences like climate science. Multiple incompatible representations and resources track different patterns in accordance with distinct scientific purposes and goals. Without a clear path to connecting the two paradigms in linguistic theorising, it seems unclear how real patterns can unify or unite formal linguistics with NLP and engineering.
This all comes back to our initial question: where do the real patterns reside? Initially, both formal symbolic grammar and the subsymbolic nature of connectionism were searching for them within cognition, such that classic proposals like Smolensky (Reference Smolensky1986) aim at reconciliation within one framework. Or perhaps something akin to autoregression is at the heart of human language competence as well (Caucheteux & King, Reference Caucheteux and King2022)? Either way, we have to contend with the fact that modern LMs are not designed to explain or uncover cognitive structure explicitly; they are engineering devices created to optimise a next token prediction task. It remains to be seen if the patterns they so expertly and superhumanly survey are the real ones in the heads of language users.
Financial support
No funding to report.
Competing interests
There are no competing interests.