Hostname: page-component-7b7567c8c-2rsb6 Total loading time: 0 Render date: 2026-11-12T09:22:05.904Z Has data issue: false hasContentIssue false
  • English
  • Français

Real-Time Speaker Evaluation: How Useful is it, and What Does it Measure?

Published online by Cambridge University Press:  01 January 2026

Martha Austen and
Kathryn Campbell-Kibler
Martha Austen*
Affiliation:
The Ohio State University
Kathryn Campbell-Kibler*
Affiliation:
The Ohio State University
*
(Austen), [martha.m.austen@gmail.com]
Campbell-Kibler, Department of Linguistics, Ohio State University, Oxley Hall, 1712 Neil Ave., Columbus, OH 43210, [campbell-kibler.1@osu.edu]
Get accessRights & Permissions [Opens in a new window]

Abstract

Indexical associations are a crucial construct in third-wave variationist work, but little is understood about how perceivers incorporate indexical information over the course of sociolinguistic perception. In classic speaker evaluation, participants listen to a stimulus and report evaluations after listening, limiting our access to the moment-to-moment process of updating social percepts. Studies developing in-the-moment tools have combined methods development with substantive theoretical questions, hindering assessment. We test a continuous evaluation tool using a gestalt style shift and the English variables (ING) and like. The tool captures the expected reactions but has poor time granularity and very high variability. Divergence between slider responses and after-the-fact ratings suggests that the tasks may depend on a different mix of processes, underlining the multiplicity of sociolinguistic cognition processes.

Keywords

sociolinguistic cognitionsociolinguistic perceptionspeaker evaluationmethodssociolinguistic monitor(ING)vernacular like

Information

Type
Research Report
Information
Language , Volume 98 , Issue 2 , June 2022, pp. e108 - e130
Copyright
Copyright © 2022 Linguistic Society of America

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

*

We'd like to thank our research participants for their time and research assistants Nick Bednar, Shasteny Cabrera Roldan, Ben Elsbrock, Taylor Gifford, Yvon Gonzalez-Bergner, Jubileen Kombe, Hannah Mackenzie, and Navya Nalla for their help. Comments from Lynn Clark, Lauren Hall-Lew, and an anonymous referee substantially improved this work, and we are grateful to them as well. This research was supported by the National Science Foundation, grant 1655014. Supplementary materials for this research report can be accessed at http://muse.jhu.edu/resolve/150.

References

Asch, Solomon E. 1946. Forming impressions of personality. The Journal of Abnormal and Social Psychology 41(3). 258–90. DOI: 10.1037/h0055756.CrossRefGoogle ScholarPubMed
Austen, Martha. 2020. The role of listener experience in perception of conditioned dialect variation. Columbus: The Ohio State University dissertation. Online: http://rave.ohiolink.edu/etdc/view?acc_num=osu159532560325774.Google Scholar
Belmore, Susan M. 1987. Determinants of attention during impression formation. Journal of Experimental Psychology: Learning, Memory, and Cognition 13(3). 480–89. DOI: 10.1037/0278-7393.13.3.480.Google Scholar
Buchstaller, Isabelle. 2006. Social stereotypes, personality traits and regional perception displaced: Attitudes towards the ‘new’ quotatives in the U.K. Journal of Sociolinguistics 10. 362–81. DOI: 10.1111/j.1360-6441.2006.00332.x.CrossRefGoogle Scholar
Campbell-Kibler, Kathryn. 2007. Accent, (ING), and the social logic of listener perceptions. American Speech 82. 32–64. DOI: 10.1215/00031283-2007-002.CrossRefGoogle Scholar
Campbell-Kibler, Kathryn. 2009. The nature of sociolinguistic perception. Language Variation and Change 21. 135–56. DOI: 10.1017/S0954394509000052.CrossRefGoogle Scholar
Campbell-Kibler, Kathryn. 2016. Towards a cognitively realistic model of meaningful sociolinguistic variation. Awareness and control in sociolinguistic research, ed. by Babel, Anna, 123–51. Cambridge: Cambridge University Press. DOI: 10.1017/CBO9781139680448.008.Google Scholar
Campbell-Kibler, Kathryn. 2021. The cognitive structure behind indexicality: Correlations in tasks linking /s/ variation and masculinity. Social meaning and linguistic variation: Theorizing the third wave, ed. by Hall-Lew, Lauren, Moore, Emma, and Podesva, Robert J., 127–50. Cambridge: Cambridge University Press. DOI: 10.1017/9781108578684.006.Google Scholar
D'Arcy, Alexandra. 2007. Like and language ideology: Disentangling fact from fiction. American Speech 82. 386–419. DOI: 10.1215/00031283-2007-025.CrossRefGoogle Scholar
Dailey-O'Cain, Jennifer. 2000. The sociolinguistic distribution of and attitudes toward focuser like and quotative like. Journal of Sociolinguistics 4. 60–80. DOI: 10.1111/1467-9481.00103.CrossRefGoogle Scholar
Diamond, Adele. 2013. Executive functions. Annual Review of Psychology 64. 135–68. DOI: 10.1146/annurev-psych-113011-143750.CrossRefGoogle ScholarPubMed
Drager, Katie. 2006. Social categories, grammatical categories, and the likelihood of ‘like‘ monophthongisation. Proceedings of the 11th Australasian International Conference on Speech Science & Technology, 384–87. Online: https://assta.org/proceedings/sst/2006/sst2006-81.pdf.Google Scholar
Drager, Katie. 2009. A sociophonetic ethnography of Selwyn Girls' High. Canterbury: University of Canterbury dissertation. DOI: 10.26021/4745.Google Scholar
Fischer, John L. 1958. Social influences on the choice of a linguistic variant. Word 14. 47–56. DOI: 10.1080/00437956.1958.11659655.CrossRefGoogle Scholar
Fredrickson, Barbara, and Kahneman, Daniel. 1993. Duration neglect in retrospective evaluations of affective episodes. Journal of Personality and Social Psychology 65. 45–55. DOI: 10.1037/0022-3514.65.1.45.CrossRefGoogle ScholarPubMed
García, Christy, Walker, Abby; and Campbell-Kibler, Kathryn. 2016. Effects of linguistic environment on the social meaning of /s/ aspiration. Poster presented at the annual meeting of the Linguistic Society of America, Washington, DC.Google Scholar
Hazen, Kirk. 2008. (ING): A vernacular baseline for English in Appalachia. American Speech 83. 116–40. DOI: 10.1215/00031283-2008-008.CrossRefGoogle Scholar
Hesson, Ashley, and Shellgren, Madeline. 2015. Discourse marker like in real time: Characterizing the time-course of sociolinguistic impression formation. American Speech 90. 154–86. DOI: 10.1215/00031283-3130313.CrossRefGoogle Scholar
Janse, Esther, and Quené, Hugo, 2004. On measuring multiple lexical activation using the cross-modal semantic priming technique. LOT Occasional Series 2. 105–14. Online: https://dspace.library.uu.nl/handle/1874/295437.Google Scholar
Johnson, Keith. 2006. Resonance in an exemplar-based lexicon: The emergence of social identity and phonology. Journal of Phonetics 34(4). 485–99. DOI: 10.1016/j.wocn.2005.08.004.CrossRefGoogle Scholar
Jones, L. Keeta. 2016. Updating evaluations of sociolinguistic variations in real time. Columbus: The Ohio State University qualifying paper.Google Scholar
Kahneman, Daniel. 1999. Objective happiness. Well-being: Foundations of hedonic psychology, ed. by Kahneman, Daniel, Diener, Edward, and Schwarz, Norbert, 3–25. New York: Russell Sage Foundation. Online: http://www.jstor.org/stable/10.7758/9781610443258.5.Google Scholar
Kahneman, Daniel, Fredrickson, Barbara L., Schreiber, Charles A.; and Redelmeier, Donald A.. 1993. When more pain is preferred to less: Adding a better end. Psychological Science 4. 401–5. DOI: 10.1111/j.1467-9280.1993.tb00589.x.CrossRefGoogle Scholar
Labov, William. 1966. The social stratification of English in New York City. Washington, DC: Center for Applied Linguistics.Google Scholar
Labov, William. 1993. The unobservability of structure and its linguistic consequences. Paper presented at New Ways of Analyzing Variation (NWAV) 22, University of Ottawa.Google Scholar
Labov, William. 2001. Principles of linguistic change, vol. 2: Social factors. Oxford: Wiley-Blackwell.Google Scholar
Labov, William, Ash, Sharon, Ravindranath, Maya, Weldon, Tracey, Baranowski, Maciej; and Nagy, Naomi. 2011. Properties of the sociolinguistic monitor. Journal of Sociolinguistics 15. 431–63. DOI: 10.1111/j.1467-9841.2011.00504.x.CrossRefGoogle Scholar
Levon, Erez, and Fox, Sue. 2014. Social salience and the sociolinguistic monitor: A case study of ING and TH-fronting in Britain. Journal of English Linguistics 42. 185–217. DOI: 10.1177/0075424214531487.CrossRefGoogle Scholar
Maddeaux, Ruth, and Dinkin, Aaron. 2017. Is like like like?: Evaluating the same variant across multiple variables. Linguistics Vanguard 3(1):20150032. DOI: 10.1515/lingvan-2015-0032.CrossRefGoogle Scholar
Montgomery, Chris, and Moore, Emma. 2018. Evaluating S(c)illy voices: The effects of salience, stereotypes, and co-present language variables on real-time reactions to regional speech. Language 94(3). 629–61. DOI: 10.1353/lan.2018.0038.CrossRefGoogle Scholar
Newman, Leonard S., and Uleman, James S.. 1993. When are you what you did? Behavior identification and dispositional inference in person memory, attribution, and social judgment. Personality and Social Psychology Bulletin 19(5). 513–25. DOI: 10.1177/0146167293195004.CrossRefGoogle Scholar
Pitt, Mark A., Johnson, Keith, Hume, Elizabeth, Kiesling, Scott; and Raymond, William. 2005. The Buckeye corpus of conversational speech: Labeling conventions and a test of transcriber reliability. Speech Communication 45. 89–95. DOI: 10.1016/j.specom.2004.09.001.CrossRefGoogle Scholar
Sumner, Meghan, Kim, Seung Kyung, King, Ed; and McGowan, Kevin B.. 2014. The socially weighted encoding of spoken words: A dual-route approach to speech perception. Frontiers in Psychology 4:1015. DOI: 10.3389/fpsyg.2013.01015.CrossRefGoogle ScholarPubMed
Tamminga, Meredith. 2017. Matched guise effects can be robust to speech style. The Journal of the Acoustical Society of America 142(1).EL18–EL23. DOI: 10.1121/1.4990399.CrossRefGoogle ScholarPubMed
Wagner, Dylan D., and Heatherton, Todd F.. 2015. Self-regulation and its failure: The seven deadly threats to self-regulation. APA handbook of personality and social psychology, vol. 1: Attitudes and social cognition, ed. by Mikulincer, Mario, Shaver, Phillip R., Borgida, Eugene, and Bargh, John A., 805–42. Washington, DC: American Psychological Association. DOI: 10.1037/14341-000.Google Scholar
Wagner, Suzanne Evans, and Hesson, Ashley. 2014. Individual sensitivity to the frequency of socially meaningful linguistic cues affects language attitudes. Journal of Language and Social Psychology 33(6). 651–66. DOI: 10.1177/0261927X14528713.CrossRefGoogle Scholar
Watson, Kevin, and Clark, Lynn. 2013. How salient is the nurse-square merger? English Language and Linguistics 17(2). 297–323. DOI: 10.1017/S136067431300004X.CrossRefGoogle Scholar
Watson, Kevin, and Clark, Lynn. 2014. Exploring listeners' real-time reactions to regional accents. Language Awareness 24(1). 38–59. DOI: 10.1080/09658416.2014.882346.CrossRefGoogle Scholar
Wolfram, Walt, and Christian, Donna. 1976. Appalachian speech. Arlington, VA: Center for Applied Linguistics.Google Scholar
Young, Andrew W., and Bruce, Vicki. 2011. Understanding person perception. British Journal of Psychology 102. 959–74. DOI: 10.1111/j.2044-8295.2011.02045.x.CrossRefGoogle ScholarPubMed
Zhang, Qing. 2005. A Chinese yuppie in Beijing: Phonological variation and the construction of a new professional identity. Language in Society 34. 431–66. DOI: 10.1017/S0047404505050153.CrossRefGoogle Scholar
Zimman, Lal. 2013. Hegemonic masculinity and the variability of gay-sounding speech: The perceived sexuality of transgender men. Journal of Language and Sexuality 2(1). 1–39. DOI: 10.1075/jls.2.1.01zim.CrossRefGoogle Scholar
Supplementary material: File

Austen and Campbell-Kibler supplementary material

Austen and Campbell-Kibler supplementary material
Download Austen and Campbell-Kibler supplementary material(File)
File 5.3 MB