Hostname: page-component-7b7567c8c-jswmv Total loading time: 0 Render date: 2026-11-15T09:53:58.853Z Has data issue: false hasContentIssue false
  • English
  • Français

On the Informativity of Different Measures of Linguistic Acceptability

Published online by Cambridge University Press:  19 February 2026

Thomas Weskott and
Glsbert Fanselow
Thomas Weskott*
Affiliation:
University of Potsdam
Glsbert Fanselow*
Affiliation:
University of Potsdam
*
University of Potsdam, Department of Linguistics Komplex II, Karl-Liebknecht-Str. 24–25 Haus 35, D-14476 Golm, Germany [weskott@uni-potsdam.de]
[fanselow@uni-potsdam.de]
Get accessRights & Permissions [Opens in a new window]

Abstract

This article deals with the claim that the MAGNITUDE ESTIMATION (ME) method of gathering acceptability judgments produces data that are more informative for linguists than binary or n-point scale judgments. We performed three acceptability-rating experiments that directly compared ME data to binary and seven-point scale data. The results clearly falsify the hypothesis that data gathered by the ME method carry a larger amount of information about the acceptability of a given linguistic phenomenon. The three measures are largely equivalent with respect to informativity. Moreover, ME judgments are shown to be more liable to producing spurious variance under certain circumstances.

Keywords

acceptability judgmentsempirical syntaxmagnitude estimationinformativity

Information

Type
Research Article
Information
Language , Volume 87 , Issue 2 , June 2011, pp. 249 - 273
Copyright
Copyright © 2011 Linguistic Society of America

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Augustin, Thomas, and Maier, Katja. 2008. Empirical evaluation of the axioms of multiplicativity, commutativity, and monotonicity in ratio production of area. Acta Psychologica 129.208–16.CrossRefGoogle ScholarPubMed
Bader, Markus, and Häussler, Jana. 2009. Toward a model of grammaticality judgments. Journal of Linguistics 46.273–330.CrossRefGoogle Scholar
Bard, Ellen Gurman, Robertson, Dan; and Sorace, Antonella. 1996. Magnitude estimation of linguistic acceptability. Language 72.32–68.CrossRefGoogle Scholar
Cowart, Wayne. 1997. Experimental syntax: Applying objective methods to sentence judgments. Thousand Oaks, CA: Sage.Google Scholar
Ellermeier, Wolfgang, and Faulhammer, Günther. 2000. Empirical evaluation of axioms fundamental to Stevens's ratio-scaling approach: I. Loudness production. Perception & Psychophysics 62.1505–11.CrossRefGoogle ScholarPubMed
Fanselow, Gisbert. 2001. Features, theta-roles, and free constituent order. Linguistic Inquiry 32.405–37.CrossRefGoogle Scholar
Fanselow, Gisbert, Féry, Caroline, Vogel, Ralf; and Schlesewsky, Matthias (eds.) 2006. Gradience in grammar. Oxford: Oxford University Press.CrossRefGoogle Scholar
Featherston, Sam. 2005. Magnitude estimation and what it can do for your syntax: Some wh-constraints in German. Lingua 115.1525–50.CrossRefGoogle Scholar
Featherston, Sam. 2009. A scale for measuring well-formedness: Why syntax needs boiling and freezing points. The fruits of empirical linguistics, vol. 1: Process, ed. by Featherston, Sam and Winkler, Susanne, 47–73. Berlin: Mouton de Gruyter.CrossRefGoogle Scholar
Haider, Hubert. 2007. As a matter of facts: Comments on Featherston's sticks and carrots. Theoretical Linguistics 33.381.-94.CrossRefGoogle Scholar
Haider, Hubert, and Rosengren, Inger. 1998. Scrambling. (Sprache and Pragmatik 49.) Lund: Lund University.Google Scholar
Jaeger, T. Florian. 2008. Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language 59.434–46.CrossRefGoogle ScholarPubMed
Keller, Frank. 2000. Gradience in grammar: Experimental and computational aspects of degrees of grammaticality. Edinburgh: University of Edinburgh dissertation.Google Scholar
Keller, Frank. 2003. A psychophysical law for linguistic judgments. Proceedings of the 25th annual conference of the Cognitive Science Society, ed. by Alterman, Richard and Kirsh, David, 652–57. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
Lodge, Milton. 1981. Magnitude scaling: Quantitative measurement of opinions. Newbury Park, CA: Sage.CrossRefGoogle Scholar
Manning, Christopher D. 2003. Probabilistic syntax. Probabilistic linguistics, ed. by Bod, Rens, Hay, Jennifer, and Jannedy, Stefanie, 289–341. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
McMurray, Bob, Tanenhaus, Michael K.; and Aslin, Richard N.. 2002. Gradient effects of within-category phonetic variation on lexical access. Cognition 86.B33–B42.CrossRefGoogle Scholar
Narens, Louis. 1996. A theory of ratio magnitude estimation. Journal of Mathematical Psychology 40.109–29.CrossRefGoogle Scholar
Pechmann, Thomas, Uszkoreit, Hans, Engelkamp, Johannes; and Zerbst, Dieter. 1994. Word order in the German Middlefield. (Computerlinguistik an der Universität des Saarlandes technical report 43.) Saarbrücken: Universität des Saarlandes.Google Scholar
Pierce, Charles A., Block, Richard A.; and Aguinis, Herman. 2004. Cautionary note on reporting eta-squared values from multifactor ANOVA designs. Educational and Psychological Measurement 64.916–24.CrossRefGoogle Scholar
Schaeffer, Nora Cate, and Bradburn, Norman M.. 1989. Respondent behavior in magnitude estimation. Journal of the American Statistical Association 84.402–13.CrossRefGoogle Scholar
Schütze, Carson T. 1996. The empirical base of linguistics: Grammaticality judgments and linguistic methodology. Chicago: University of Chicago Press.Google Scholar
Sorace, Antonella, and Keller, Frank. 2005. Gradience in linguistic data. Lingua 115.1497–524.CrossRefGoogle Scholar
Sprouse, Jon. 2007a. Experimental syntax: What does it get you? Paper presented at the 20th annual CUNY Conference on Human Sentence Processing, San Diego, CA.Google Scholar
Sprouse, Jon. 2007b. A program for experimental syntax: Finding the relationship between acceptability and grammatical knowledge. College Park: University of Maryland dissertation.Google Scholar
Sprouse, Jon. 2008. Magnitude estimation and the non-linearity of acceptability judgments. Proceedings of the West Coast Conference on Formal Linguistics (WCCFL) 27.397–403.Google Scholar
Sprouse, Jon. 2011. A test of the cognitive assumptions of magnitude estimation: Commutativity does not hold for acceptability judgments. Language 87.274–88.CrossRefGoogle Scholar
Stevens, Stanley Smith. 1946. On the theory of scales of measurement. Science 103.677–80.CrossRefGoogle ScholarPubMed
Stevens, Stanley Smith. 1956. The direct estimation of sensory magnitudes: Loudness. American Journal of Psychology 69.1–25.CrossRefGoogle ScholarPubMed
Stevens, Stanley Smith. 1975. Psychophysics: Introduction to its perceptual, neural, and social prospects. New York: Wiley.Google Scholar
Wegener, Bernd. 1983. Category-rating and magnitude estimation scaling techniques: An empirical comparison. Sociological Methods and Research 12.31–75.CrossRefGoogle Scholar
Weskott, Thomas, and Fanselow, Gisbert. 2008. Variance and informativity in different measures of linguistic acceptability. Proceedings of the West Coast Conference on Formal Linguistics (WCCFL) 27.431–39.Google Scholar
Weskott, Thomas, and Fanselow, Gisbert. 2009. Scaling issues in the measurement of linguistic acceptability. The fruits of empirical linguistics, vol. 1: Process, ed. by Featherston, Sam and Winkler, Susanne, 231–45. Berlin: Mouton de Gruyter.Google Scholar