Hostname: page-component-7b7567c8c-lpg2g Total loading time: 0 Render date: 2026-11-15T23:24:38.922Z Has data issue: false hasContentIssue false
  • English
  • Français

Fusion is Great, and Interpretable Fusion could be Exciting for Theory Generation: Response to Pater

Published online by Cambridge University Press:  01 January 2026

Lisa S. Pearl
Lisa S. Pearl*
Affiliation:
University of California, Irvine
*
[lpearl@uci.edu]
Get accessRights & Permissions [Opens in a new window]

Abstract

From my perspective, Pater's (2019) target article does a great service both to researchers who work in generative linguistics and to researchers who utilize neural networks—and especially to researchers who might find themselves wanting to do both by harnessing the insights of each tradition. The fusion of theories of linguistic representation and probabilistic learning techniques has certainly led to many interesting and valuable insights about the nature of both linguistic representation and the language acquisition process. However, I feel that the most exciting aspect of Pater's article is the increasing interpretability of neural network models, especially when combined with insights from the theoretical framework of generative linguistics. This allows for the possibility that neural networks could be used to actually generate new theories of representation. I describe how I think this theory-generation process might work with interpretable neural networks.

Keywords

generative linguisticsinterpretable neural networksneural networksprobabilistic learning modelssymbolic modelstheory evaluationtheory generation

Information

Type
Perspectives
Information
Language , Volume 95 , Issue 1 , March 2019, pp. e109 - e114
Copyright
Copyright © 2019 Linguistic Society of America

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Abend, Omri, Kwiatkowski, Tom, Smith, Nathaniel J., Goldwater, Sharon; and Steedman, Mark. 2017. Bootstrapping language acquisition. Cognition 164. 116-43. DOI: 10.1016/j.cognition.2017.02.009.CrossRefGoogle ScholarPubMed
Avery, Michael C., and Krichmar, Jeffrey L.. 2017. Neuromodulatory systems and their interactions: A review of models, theories, and experiments. Frontiers in Neural Circuits 11:108. DOI: 10.3389/fncir.2017.00108.CrossRefGoogle ScholarPubMed
Back, Thomas. 1996. Evolutionary algorithms in theory and practice: Evolution strategies, evolutionary programming, genetic algorithms. Oxford: Oxford University Press.CrossRefGoogle Scholar
Beyeler, Michael, Rounds, Emily, Carlson, Kristofor, Dutt, Nikil; and Krichmar, Jeffrey L.. 2017. Sparse coding and dimensionality reduction in cortex. bioRxiv 149880 (preprint). DOI: 10.1101/149880.CrossRefGoogle Scholar
Dewar, Kathryn M., and Xu, Fei. 2010. Induction, overhypothesis, and the origin of abstract knowledge: Evidence from 9-month-old infants. Psychological Science 21. 1871-77. DOI: 10.1177/0956797610388810.CrossRefGoogle ScholarPubMed
Khan, Gul Muhammad. 2018. Evolutionary computation. Evolution of artificial neural development: In search of learning genes, 29–37. Dordrecht: Springer. DOI: 10.1007/978-3-319-67466-7_3.CrossRefGoogle Scholar
Koza, John R. 1994. Genetic programming as a means for programming computers by natural selection. Statistics and Computing 4. 87–112. DOI: 10.1007/BF00175355.CrossRefGoogle Scholar
Koza, John R. 1995. Survey of genetic algorithms and genetic programming. Proceedings of WESCON'95, 589-94. DOI: 10.1109/WESCON.1995.485447.CrossRefGoogle Scholar
Krichmar, Jeffrey L., Conradt, Jörg; and Asada, Minoru. 2015. Neurobiologically inspired robotics: Enhanced autonomy through neuromorphic cognition. Neural Networks 72. 1–2. DOI: 10.1016/j.neunet.2015.11.004.CrossRefGoogle ScholarPubMed
Li, Bingdong, Li, Jinlong, Tang, Ke; and Yao, Xin. 2015. Many-objective evolutionary algorithms: A survey. ACM Computing Surveys (CSUR) 48(1):13. DOI: 10.1145/2792984.CrossRefGoogle Scholar
Marr, David. 1982. Vision: A computational investigation into the human representation and processing of visual information. San Francisco: W. H. Freeman.Google Scholar
Neftci, Emre O., Augustine, Charles, Paul, Somnath; and Detorakis, Georgios. 2017. Event-driven random back-propagation: Enabling neuromorphic deep learning machines. Frontiers in Neuroscience 11:324. DOI: 10.3389/fnins.2017.00324.CrossRefGoogle ScholarPubMed
Neftci, Emre O., Binas, Jonathan, Rutishauser, Ueli, Chicca, Elisabetta, Indiveri, Giacomo; and Douglas, Rodney J.. 2013. Synthesizing cognition in neuromorphic electronic systems. Proceedings of the National Academy of Sciences of the United States of America 110(37).E3468–E3476. DOI: 10.1073/pnas.1212083110.Google ScholarPubMed
Palangi, Hamid, Smolensky, Paul, He, Xiaodong; and Deng, Li. 2017. Question-answering with grammatically-interpretable representations. arXiv:1705.08432 [cs.CL]. Online: https://arxiv.org/abs/1705.08432.CrossRefGoogle Scholar
Pater, Joe. 2019. Generative linguistics and neural networks at 60: Foundation, friction, and fusion. Language 95(1). e41–e74.CrossRefGoogle Scholar
Pearl, Lisa. 2017. Evaluation, use, and refinement of knowledge representations through acquisition modeling. Language Acquisition 24(2). 126-47. DOI: 10.1080/10489223.2016.1192633.CrossRefGoogle Scholar
Pearl, Lisa. 2019. Modeling syntactic acquisition. The Oxford handbook of experimental syntax, ed. by Sprouse, Jon. Oxford: Oxford University Press, to appear.Google Scholar
Pearl, Lisa, and Goldwater, Sharon. 2016. Statistical learning, inductive bias, and Bayesian inference in language acquisition. The Oxford handbook of developmental linguistics, ed. by Lidz, Jeffrey, Snyder, William, and Pater, Joe, 664-95. Oxford: Oxford University Press. DOI: 10.1093/oxfordhb/9780199601264.013.28.Google Scholar
Pearl, Lisa, Ho, Timothy; and Detrano, Zephyr. 2017. An argument from acquisition: Comparing English metrical stress representations by how learnable they are from child-directed speech. Language Acquisition 24(4). 307-42. DOI: 10.1080/10489223.2016.1194422.CrossRefGoogle Scholar
Pearl, Lisa, and Mis, Benjamin. 2016. The role of indirect positive evidence in syntactic acquisition: A look at anaphoric one. Language 92(1). 1–30. DOI: 10.1353/lan.2016.0006.CrossRefGoogle Scholar
Pearl, Lisa, and Sprouse, Jon. 2013. Computational models of acquisition for islands. Experimental syntax and island effects, ed. by Sprouse, Jon and Hornstein, Norbert, 109-31. Cambridge: Cambridge University Press. DOI: 10.1017/CBO9781139035309.006.Google Scholar
Pearl, Lisa, and Sprouse, Jon. 2018a. Comparing solutions to the linking problem using an integrated quantitative framework of language acquisition. Irvine: University of California, Irvine, ms. Online: https://ling.auf.net/lingbuzz/003913.Google Scholar
Pearl, Lisa, and Sprouse, Jon. 2018b. The acquisition of linking theories: A tolerance principle approach to learning UTAH and rUTAH. Irvine: University of California, Irvine, ms. Online: https://ling.auf.net/lingbuzz/004088.Google Scholar
Perfors, Amy. 2012. Bayesian models of cognition: What's built in after all? Philosophy Compass 7(2). 127-38. DOI: 10.1111/j.1747-9991.2011.00467.x.CrossRefGoogle Scholar
Perfors, Amy, Tenenbaum, Joshua B.; and Regier, Terry. 2011. The learnability of abstract syntactic principles. Cognition 118. 306-38. DOI: 10.1016/j.cognition.2010.11.001.CrossRefGoogle ScholarPubMed
Yang, Charles D. 2002. Knowledge and learning in natural language. Oxford: Oxford University Press.Google Scholar
Yang, Charles D. 2004. Universal grammar, statistics or both? Trends in Cognitive Science 8(10). 451-56. DOI: 10.1016/j.tics.2004.08.006.CrossRefGoogle ScholarPubMed
Yang, Charles D. 2005. On productivity. Linguistic Variation Yearbook 5. 265–302. DOI: 10.1075/livy.5.09yan.CrossRefGoogle Scholar
Yang, Charles D. 2012. Computational models of syntactic acquisition. WIREs Cognitive Science 3. 205-13. DOI: 10.1002/wcs.1154.CrossRefGoogle ScholarPubMed
Yang, Charles D. 2016. The price of linguistic productivity: How children learn to break the rules of language. Cambridge, MA: MIT Press.CrossRefGoogle Scholar