Highlights
-
• Bilinguals accurately identify emotions in both their L1 and highly proficient L2
-
• Emotion recognition in L2 correlates with L2 proficiency
-
• In-group advantage for vocal emotion recognition extends to a highly proficient L2
-
• L2 in-group cultural membership is not essential for high emotion recognition
1. Introduction
The expression of emotion is a vital aspect of human communication. Previous research has established that basic emotions such as anger, fear, sadness, joy/happiness and disgust (Ekman & Friesen, Reference Ekman and Friesen1971) can be conveyed through behavior, facial expressions and vocally (Elfenbein & Ambady, Reference Elfenbein and Ambady2002). In fact, prosodic and acoustic features such as intonation, pitch and rhythm can be used to convey emotion through the voice alone (Banse & Scherer, Reference Banse and Scherer1996; Juslin & Laukka, Reference Juslin and Laukka2003; Scherer, Reference Scherer2003), playing a crucial role in social communication by helping listeners interpret a speaker’s intention and emotional state. For detailed discussions of the specific acoustic features associated with different emotions and their recognition across languages, see Ekberg et al. (Reference Ekberg, Stavrinos, Andin, Stenfelt and Dahlström2023), Larrouy-Maestri et al. (Reference Larrouy-Maestri, Poeppel and Pell2025) and Yildirim et al. (Reference Yildirim, Bulut, Lee, Kazemzadeh, Deng, Lee, Narayanan and Busso2004).
Research has found that vocal emotion (just as facial emotion) has universal qualities that can be recognized cross-culturally at levels greater than chance (e.g., Paulmann & Uskul, Reference Paulmann and Uskul2014; Pell et al., Reference Pell, Monetta, Paulmann and Kotz2009a; Scherer et al., Reference Scherer, Banse and Wallbott2001; Thompson & Balkwill, Reference Thompson and Balkwill2006; Zhang & Pell, Reference Zhang and Pell2022; for other perspectives, such as those emphasizing facial emotion recognition being largely context-dependent, see e.g., Barrett, Reference Barrett2011; Barrett et al., Reference Barrett, Mesquita and Gendron2011; Gendron et al., Reference Gendron, Roberson, Van Der Vyver and Barrett2014; Nelson & Russell, Reference Nelson and Russell2013). While some aspects of vocal emotion recognition appear to be universal, research additionally indicates that it may be easier to interpret emotional prosody in a listener’s native or first language (L1) compared to unfamiliar foreign languages (Elfenbein & Ambady, Reference Elfenbein and Ambady2002; Laukka & Elfenbein, Reference Laukka and Elfenbein2021; Paulmann & Uskul, Reference Paulmann and Uskul2014; Pell et al., Reference Pell, Monetta, Paulmann and Kotz2009a; Scherer et al., Reference Scherer, Banse and Wallbott2001; Thompson & Balkwill, Reference Thompson and Balkwill2006; Zhang & Pell, Reference Zhang and Pell2022; but see McCluskey & Albas, Reference McCluskey and Albas1981; Zhu, Reference Zhu2013). This benefit, known as the in-group advantage, is thought to arise due to language- or culture-specific elements of emotional communication aiding in emotion recognition within a cultural community (Elfenbein et al., Reference Elfenbein, Beaupré, Lévesque and Hess2007; Elfenbein & Ambady, Reference Elfenbein and Ambady2002).
However, the idea of an in-group advantage may not uniformly apply to all individuals. Bilinguals, for example, speak two languages and may have varying degrees of language proficiency or exposure to the cultural environment of their second language (L2). Little is known about how bilinguals recognize emotions in both languages, especially for a later-learned L2 where they may not belong to the cultural in-group. This study compares emotion recognition in participants’ L1, L2 and unfamiliar languages to explore whether the in-group advantage extends to highly proficient bilinguals’ L2, despite their lack of in-group membership.
1.1. L1 advantage in vocal emotion recognition
Previous studies investigating the recognition of vocal emotions have identified both universal as well as culture-specific aspects of emotion recognition. Across a range of recognition-based task types, languages and emotions, research has found that listeners can recognize vocal emotion in both their own language and unfamiliar foreign languages at a level greater than chance (Paulmann & Uskul, Reference Paulmann and Uskul2014; Pell et al., Reference Pell, Monetta, Paulmann and Kotz2009a; Sauter et al., Reference Sauter, Eisner, Calder and Scott2010a; Scherer et al., Reference Scherer, Banse and Wallbott2001; Thompson & Balkwill, Reference Thompson and Balkwill2006; Zhang & Pell, Reference Zhang and Pell2022; see Laukka & Elfenbein, Reference Laukka and Elfenbein2021 for a recent meta-analysis). Listeners’ ability to recognize emotion conveyed through unfamiliar foreign languages supports the “universality” of emotional prosody recognition (Scherer et al., Reference Scherer, Banse and Wallbott2001; but note that universality claims have been challenged due to being primarily derived from vocal-based forced-choice recognition studies; for discussion see Gendron et al., Reference Gendron, Roberson, Van Der Vyver and Barrett2014; Lorette & Dewaele, Reference Lorette and Dewaele2024). Furthermore, above-chance emotion recognition accuracy has been observed regardless of whether the unfamiliar languages are similar to the listeners’ native language (e.g., within the same language family) or more distantly related (e.g., Pell et al., Reference Pell, Monetta, Paulmann and Kotz2009a; Thompson & Balkwill, Reference Thompson and Balkwill2006).
Importantly, while vocal emotion recognition has been found to exceed chance in both a listener’s own language and unfamiliar languages, research shows that individuals better recognize vocal emotions from another speaker of their native language (i.e., a member of their own cultural group) compared to recognizing emotions spoken in an unfamiliar foreign language (that is cross-cultural emotion recognition; Cho & Dewaele, Reference Cho and Dewaele2021; Paulmann & Uskul, Reference Paulmann and Uskul2014; Pell et al., Reference Pell, Monetta, Paulmann and Kotz2009a; Sauter et al., Reference Sauter, Eisner, Calder and Scott2010a; Thompson & Balkwill, Reference Thompson and Balkwill2006; Zhang & Pell, Reference Zhang and Pell2022). This effect has been described as an “in-group advantage” in emotion recognition (Elfenbein & Ambady, Reference Elfenbein and Ambady2002). For instance, Thompson and Balkwill (Reference Thompson and Balkwill2006) investigated how native English listeners from Canada recognized joy, anger, sadness and fear in semantically neutral sentences produced in both their native language (English) and four unfamiliar languages: Chinese, German, Japanese and Tagalog. Emotion recognition was above chance across all languages, suggesting that some emotional prosodic cues are recognizable across languages, even for languages that the listener does not speak and is not familiar with. However, emotion recognition accuracy was significantly higher for English than for the unfamiliar languages, supporting the in-group advantage. Further supporting this in-group advantage, Laukka et al. (Reference Laukka, Elfenbein, Thingujam, Rockstuhl, Iraki, Chui and Althoff2016) found that even among native speakers of the same language (English), there was an emotion recognition advantage when the speaker was a member of the listener’s own cultural group (i.e., Australia, India, Kenya, Singapore or the United States) as well as differences in emotional production by English speakers from these different cultural backgrounds. Together, these findings highlight the role of shared listener-speaker cultural membership on the in-group advantage, suggesting that it may stem from learned, culturally specific patterns of emotion production in a culturally defined group of language users (i.e., theory of cultural learning in emotional behavior, Elfenbein & Ambady, Reference Elfenbein and Ambady2002; dialect theory, Elfenbein, Reference Elfenbein2013; Laukka et al., Reference Laukka, Elfenbein, Thingujam, Rockstuhl, Iraki, Chui and Althoff2016).
To better compare emotion recognition across known and unfamiliar languages and isolate emotional prosody apart from semantic meaning, studies have used pseudo-utterance stimuli – non-word sentences that follow the grammatical and phonological rules of a particular language (e.g., “The fector egzullin the boshent”; Banse & Scherer, Reference Banse and Scherer1996; Paulmann & Uskul, Reference Paulmann and Uskul2014; Pell et al., Reference Pell, Monetta, Paulmann and Kotz2009a; Pell & Kotz, Reference Pell and Kotz2011; Scherer et al., Reference Scherer, Banse and Wallbott2001; see Lorette & Dewaele, Reference Lorette and Dewaele2019, Reference Lorette and Dewaele2020, Reference Lorette and Dewaele2024, for studies examining emotion recognition of semantically meaningful stimuli). For instance, Pell et al. (Reference Pell, Monetta, Paulmann and Kotz2009a) examined how Argentine Spanish listeners identified emotions in pseudo-utterances presented in their native language, Spanish, as well as in three foreign languages they did not speak (English, German and Arabic). Results showed that emotion recognition was above chance for all language conditions, with higher accuracy in identifying emotions in the listeners’ native language than in the three unfamiliar languages, again supporting the in-group advantage. Within their native language, listeners identified certain emotions more accurately than others, with higher recognition accuracy for joy, anger and fear and lowest accuracy for neutral, disgust and sadness. This suggests that while the in-group advantage enhances overall recognition in a native language, its effects vary by emotion. Some emotions may be more universally recognizable, whereas others rely more on language-specific prosodic patterns, making them harder to identify in unfamiliar languages.
Much of the existing work on vocal emotion recognition typically compares listeners’ identification of emotions in their native language (where cultural “in-group” membership is inferred implicitly based on the language spoken) with emotion identification in unfamiliar foreign languages (e.g., Liang et al., Reference Liang, Goudbeek, Konopka, Choi and Broersma2025; Pell et al., Reference Pell, Monetta, Paulmann and Kotz2009a; Sauter et al., Reference Sauter, Eisner, Calder and Scott2010a; Scherer et al., Reference Scherer, Banse and Wallbott2001; Thompson & Balkwill, Reference Thompson and Balkwill2006). However, the majority of the world’s population is bilingual, speaking more than one language. While the “in-group” advantage in emotion recognition is well-replicated, it remains unclear how it extends to L2 users – particularly those who have reached high L2 proficiency but are not themselves members of the L2 cultural environment. Patterns of emotion recognition in an L2 can offer insights into how language familiarity (apart from in-group membership) may contribute to vocal emotion recognition. For instance, highly accurate L2 emotion recognition may suggest that cultural membership is not required for accurate emotion recognition, whereas reduced recognition relative to the L1 may reflect the added influence of broader in-group cultural experience. In this way, bilingualism serves not only as a population of interest but also as a useful context for examining how language proficiency shapes the in-group advantage in vocal emotion recognition.
1.2. L2 vocal emotion recognition
Only a small subset of research on vocal emotion recognition has included bilingual participants, and those that do typically test participants in the L2 environment. These studies have replicated the “in-group” advantage effects to varying degrees (Alqarni & Dewaele, Reference Alqarni and Dewaele2020; Altrov, Reference Altrov2013; Bhatara et al., Reference Bhatara, Laukka, Boll-Avetisyan, Granjon, Elfenbein and Bänziger2016; Dromey et al., Reference Dromey, Silveira and Sandor2005; Graham et al., Reference Graham, Hamblin and Feldstein2001; Lorette & Dewaele, Reference Lorette and Dewaele2015; Paulmann & Uskul, Reference Paulmann and Uskul2014; Zhang & Pell, Reference Zhang and Pell2022; Zhu, Reference Zhu2013). For example, Altrov (Reference Altrov2013) examined recognition of emotion (i.e., anger, joy, sadness and neutral emotion) in Estonian sentences for three listener groups: Estonians, Russians and Russian-Estonian bilinguals (living in Estonia). Overall, they observed that the Russian-Estonian group had increased Estonian (L2) emotion recognition accuracy compared to native Russians (living in Russia, with no Estonian proficiency). The Russian-Estonian group was just as accurate as native Estonians on anger and sadness, but performed more poorly on joy and neutral emotion. These results were interpreted to indicate that Estonian language ability as well as living and working in an Estonian environment improved emotion recognition of Estonian L2 users. This was particularly true for anger and sadness, indicating that certain emotions may be learned by an out-group or L2 user more easily than other emotions.
Similar conclusions were drawn by Paulmann and Uskul (Reference Paulmann and Uskul2014), who compared the recognition of emotions (i.e., anger, disgust, fear, happiness, sadness, surprise and neutral emotion) in English and Chinese for two participant groups: native British English listeners and Chinese-English bilinguals living in the UK. They found in-group advantages for both groups: overall, the native-English listener group was more accurate at recognizing English emotions, while the Chinese-English bilingual listener group was more accurate at recognizing emotions in Mandarin. Importantly, the Chinese-English bilingual listener group performed slightly better on their out-group language (L2 English) than the native English listener group did with their out-group language (unfamiliar Mandarin). The authors suggest that the elevated out-group performance of the bilingual listeners could be attributed to their increased exposure to English while living in their L2 environment.
Zhang and Pell (Reference Zhang and Pell2022) extended these findings by adding an unfamiliar language (Hindi). Consistent with Paulmann and Uskul (Reference Paulmann and Uskul2014), Zhang and Pell (Reference Zhang and Pell2022) found in-group advantages for both cultural groups – native English listeners performed better in English; Chinese-English bilingual listeners performed better in Mandarin. Additionally, both groups showed higher accuracy in English and Chinese compared to Hindi. Like Paulmann and Uskul (Reference Paulmann and Uskul2014) and Altrov (Reference Altrov2013), the authors largely interpreted their results on the basis of culture, suggesting that for the Chinese-English bilingual listeners, more familiarity with English culture may have contributed to their improved recognition accuracy for English stimuli. However, they additionally suggest that their Chinese-English bilinguals’ L2 emotion recognition performance may be attributed to their level of English proficiency. Greater language proficiency may enhance emotion recognition due to increased familiarity with relevant acoustic-phonetic and prosodic features associated with emotional expression, thereby improving sensitivity to these cues. However, they did not explicitly examine the relationship between emotion recognition and language proficiency in their study.
To our knowledge, only two studies have explicitly attempted to examine the effect of L2 proficiency on L2 vocal emotion recognition (for studies investigating the effect of L2 proficiency on emotion recognition of semantically meaningful stimuli with vocal, audiovisual and semantic cues, see Lorette & Dewaele, Reference Lorette and Dewaele2019, Reference Lorette and Dewaele2020, Reference Lorette and Dewaele2024). In their study of English language learners living in the US, Graham et al. (Reference Graham, Hamblin and Feldstein2001) found that Japanese-English and Spanish-English bilingual listeners performed more poorly than native English listeners on English emotion recognition. English emotion recognition did not differ for beginner/low-proficiency listeners compared to more advanced/high-proficiency listeners. Similarly, Bhatara et al. (Reference Bhatara, Laukka, Boll-Avetisyan, Granjon, Elfenbein and Bänziger2016) examined the effect of English language proficiency on emotion recognition in French-English bilinguals living in France and found no overall effect of proficiency. Interestingly, recognition of positive emotions (such as interest, pride, relief and happiness) in the L2 was negatively correlated with English proficiency, although the use of real words in the stimuli may have drawn more proficient participants’ attention to the lexical items rather than the prosodic features essential for accurately detecting positive emotions.
Overall, results on L2 emotion recognition are mixed. Some studies suggest that bilinguals exhibit better emotion recognition in the L1 than L2, but that L2 emotion recognition may be improved by language or cultural familiarity (Altrov, Reference Altrov2013; Paulmann & Uskul, Reference Paulmann and Uskul2014; Zhang & Pell, Reference Zhang and Pell2022). On the other hand, there is little evidence to indicate that this benefit is directly related to differences in L2 proficiency levels (Bhatara et al., Reference Bhatara, Laukka, Boll-Avetisyan, Granjon, Elfenbein and Bänziger2016; Graham et al., Reference Graham, Hamblin and Feldstein2001). Moreover, much of the existing literature is limited by unclear definitions of listeners’ language backgrounds and a lack of comparisons to performance of first, second and unfamiliar languages. In addition, most studies (i.e., Bhatara et al., Reference Bhatara, Laukka, Boll-Avetisyan, Granjon, Elfenbein and Bänziger2016; Graham et al., Reference Graham, Hamblin and Feldstein2001; Paulmann & Uskul, Reference Paulmann and Uskul2014; Zhang & Pell, Reference Zhang and Pell2022) have been conducted in the native cultural environment of the listeners’ L2, with bilinguals who have spent considerable time living there, making it difficult to disentangle the impact of L2 proficiency from direct experience with the L2 culture.
1.3. Present study
This study examines how the in-group advantage in vocal emotional prosody recognition – typically observed when listeners identify emotions more accurately in their L1 than in unfamiliar foreign languages – extends to a bilingual’s second language (L2). We explore the role of language proficiency on emotion recognition by testing listeners who are highly proficient in a second language but are not a member of the L2 cultural in-group. In doing so, this study advances theories of emotion recognition by highlighting the role of language experience without in-group membership. The present study asks two key questions: (a) do in-group advantages in emotion recognition, typically associated with an L1, also extend to a highly proficient L2, even when listeners do not belong to the L2’s native cultural in-group? And (b) how does emotion recognition in unfamiliar foreign languages – languages for which the listener neither speaks nor has cultural affiliation with – compare to emotion recognition in the L1 and L2?
We address these questions by examining how accurately highly proficient Dutch-English bilinguals recognize four emotions (happiness, sadness, anger and fear) in pseudo-sentences spoken in their L1 (Dutch), L2 (English) and two unfamiliar foreign languages (Arabic and Hindi). Importantly, this participant group is highly proficient in L2 English and has regular exposure to nonnative English as part of daily life in the Netherlands (L1 Dutch environment). Additionally, participants were tested in their L1 cultural setting and had little to no prior direct immersion in a native English-speaking cultural environment (i.e., were not part of a native English-speaking cultural “in-group”).
Building on the theoretical frameworks reviewed in the Introduction, three predictions can be outlined. Prediction (a), which we call the language-proficiency advantage hypothesis, predicts that the in-group advantage extends to the highly proficient L2, resulting in similar emotion recognition in L1 and L2, with listeners performing better in the two languages they speak than in the two unfamiliar foreign languages. If the language-proficiency advantage hypothesis is supported, it would imply that highly proficient bilinguals are as effective at detecting emotions in their later-learned L2 as in their L1 – even in the absence of in-group membership in the L2 environment – and would show a clear advantage for both languages they speak over unfamiliar foreign languages.
Prediction (b), which we call the cultural-membership hypothesis, implies that the in-group advantage does not extend to the L2, leading to greater emotion recognition in the L1 compared to the L2. This hypothesis ascribes an important role to cultural membership for developing emotion recognition skills comparable to those in the L1. In this case, emotion recognition performance in the L2 would be similar to that in the unfamiliar foreign languages, supporting the notion that proficiency alone – without native cultural experience – does not facilitate the same level of emotion recognition as in the L1.
Prediction (c), which we call the L2 modulation hypothesis, suggests that the L2 proficiency modulates the in-group advantage, such that emotion recognition is highest in the L1, followed by the L2, with the lowest performance observed in the unfamiliar languages. This would indicate that being a member of the cultural in-group supports superior recognition accuracy for the L1. High L2 proficiency would offer a moderate advantage over unfamiliar languages, albeit less than that of the L1.
2. Methods
2.1. Participants
Fifty-seven Dutch-English bilinguals were recruited from Radboud University in Nijmegen, the Netherlands. Data from nine participants were discarded: one due to not following task instructions, one due to reporting English as their most proficient language, three due to exceeding the intended age range and four due to living in an English-speaking environment for 10 or more years. The final sample included 48 participants (27 women, 15 men, 6 nonbinary; mean age = 22.75 years, min = 18, max = 40). Our target sample size was determined based on previous studies on emotion recognition using similar experimental designs, and our final sample size is comparable to that of similar studies in this field (i.e., Altrov, Reference Altrov2013; Bhatara et al., Reference Bhatara, Laukka, Boll-Avetisyan, Granjon, Elfenbein and Bänziger2016; Cho & Dewaele, Reference Cho and Dewaele2021; Liang et al., Reference Liang, Goudbeek, Konopka, Choi and Broersma2025).
All included participants spoke Dutch as their first language and learned English as a second language (mean age of English acquisition = 8.92 years; min = 2, max = 12), in a school setting. Only two participants can be considered simultaneous bilinguals, reporting L2 acquisition before the age of four; the rest are sequential bilinguals.Footnote 1 Participants were highly proficient English speakers with an average score on the LexTALE English lexical decision task of 85.6 (SD = 10.69; max score is 100). The average self-rated speaking proficiency for Dutch (9.67, SD = 0.56; max score = 10) was greater than the average self-rated speaking proficiency for English (7.54, SD = 0.97; t(47) = −12.9, p < .001). Furthermore, participants reported how many years they have spent living in English- and Dutch-speaking environments (categorical response options ranged from “zero years” to “ten or more years” in one-year increments). All participants reported that they had spent ten or more years living in a Dutch-speaking country. Most participants reported spending little to no time living in an English-speaking country (35 spent zero years, 11 spent one year or less and 2 spent one to two years).
Participants had no or limited experience speaking or listening to Arabic or Hindi. As self-reported on a language history questionnaire, 47 participants had no experience with Hindi (1 reported having traveled to India). Forty-two participants had no experience with Arabic (with 6 reporting exposure through movies/TV/music/pop culture or having an Arabic-speaking friend). Prior to testing, all participants provided informed consent, as approved by the Institutional Review Board at The Pennsylvania State University and the Humanities Ethics Review Board at Radboud University. Participants were compensated at a rate of 10 EUR per hour for their participation.
2.2. Materials
Materials for this study consisted of Dutch, English, Hindi and Arabic pseudo-sentence recordings, spoken with angry, sad, happy, fearful and neutral emotion. Critically, all experimental stimuli were pseudo-sentences in which content words were replaced by semantically meaningless but phonotactically legal words (for example, in English, The fector egzullin the boshent). This format retains phonological and some grammatical properties of the language, allowing speakers to easily produce each sentence with different emotional inflections. However, critical to the experimental design of the present study, all words were semantically meaningless; thus, there was no risk of participants accessing semantic information or emotional associations, and decisions were made purely on prosodic cues.
The English, Hindi and Arabic stimuli used in this study were carefully created and validated by Pell et al. (Reference Pell, Paulmann, Dara, Alasseri and Kotz2009b). For each language, pseudo-sentences were first constructed by a native speaker of each language. Then these pseudo-sentences were produced and recorded by native speakers of each language (English speakers from Quebec/Ontario, Hindi speakers from Central India and Jordanian/Syrian dialect Arabic speakers from the Middle East) using the same emotional elicitation procedure (see Pell et al., Reference Pell, Paulmann, Dara, Alasseri and Kotz2009b). Each speaker produced each pseudo-sentence with different emotional inflections including anger, disgust, fear, sadness, happiness, pleasant surprise and neutral emotion. The stimuli for each language were then entered into perceptual rating studies, where they were rated by native speakers of that language. For full details on the stimuli creation, emotional elicitation and recording procedures and results from the perceptual rating studies for the English, Hindi and Arabic stimuli sets, see Pell et al. (Reference Pell, Paulmann, Dara, Alasseri and Kotz2009b).
2.2.1. Dutch stimuli creation
The Dutch stimuli were created by carefully following the methods outlined in Pell et al. (Reference Pell, Paulmann, Dara, Alasseri and Kotz2009b), using the same approach as was used for creation and validation of the English, Arabic and Hindi stimuli sets. A native Dutch speaker (author JGV) created a list of 30 Dutch pseudo-sentences, which were phonotactically and syntactically legal in Dutch, but in which all content words (nouns and verbs) were replaced by semantically anomalous pseudo-words (e.g., Hij belierde een parf). On average, the sentences were 4.5 words long (min = 3, max = 6). Two female native speakers of Dutch (ages 21 and 24), not affiliated with or familiar with the nature and goals of the study, recorded the pseudo-sentences in four emotional inflections: anger, fear, happiness and sadness, and non-emotional neutral fillers. In line with the procedure used by Pell et al. (Reference Pell, Paulmann, Dara, Alasseri and Kotz2009b), before recording the pseudo-sentences, the speakers practiced eliciting each emotion with lexicalized Dutch sentences to facilitate later production of pseudo-sentences in a way that accurately reflected natural prosodic inflections for each emotion. While recording the pseudo-sentence stimuli, speakers were encouraged to speak naturally, avoiding exaggeration. Each sentence was repeated three times, with all sentences for one emotion recorded before proceeding to the next emotion. The order of emotions was randomized between speakers. Recordings were conducted with a digital recorder (TASCAM DR-100 MK III) and an external microphone (STAGG IMP-600 Dynamic Microphone). Sentences were segmented into individual 16-bit WAV files and adjusted to a root-mean-square amplitude of 70 dB sound pressure level. The sampling frequency was set at 44.1 kHz using Praat software (Boersma & Weenink, Reference Boersma and Weenink2022). Dutch stimuli created for this experiment are available on OSF.
2.2.2. Dutch stimuli validation
A validation study was conducted with 30 native Dutch speakers (21 male; 9 female; mean age = 28.53, SD = 9.94) all residing in the Netherlands, to identify the most accurate stimuli for use in the current experiment. Participants (none of whom participated in the main experiment) were recruited via Prolific© and completed the study online via Qualtrics (Qualtrics, Provo, UT). They listened to all pseudo-sentences produced by both speakers in a randomized order and answered a multiple-choice question indicating which emotion they thought the speaker was expressing: angry, fearful, happy, sad or neutral. Data were then used to calculate the average accuracy for each sentence.
2.2.3. Stimuli selection
Perceptual data from the Dutch validation study, along with the validation data for the English, Arabic and Hindi stimuli sets from Pell et al. (Reference Pell, Paulmann, Dara, Alasseri and Kotz2009b), were used to select the final sentences for this experiment. The highest-rated female speaker of each language was chosen as the exemplar speaker for that language. For each emotion, the top 12 most accurately rated sentences for each speaker were selected, resulting in 60 sentences per speaker. See Table 1 for validation study accuracy ratings for each language and emotion. Validation study accuracy rates are controlled for in the statistical analysis to account for potential baseline stimuli differences (see Accuracy analysis below).
Average percentage (SD) of native speakers who correctly identified the emotion of the final stimuli in their respective validation studies (English, Hindi and Arabic from Pell et al., Reference Pell, Paulmann, Dara, Alasseri and Kotz2009b)
Table 1. Long description
The table consists of five columns: Emotion, Dutch, English, Arabic, and Hindi. Data is presented as a percentage followed by the S D in brackets.
* Angry: Dutch 95.00 percent (3.47 percent), English 71.11 percent (8.37 percent), Arabic 78.28 percent (6.27 percent), Hindi 89.17 percent (5.72 percent).
* Fearful: Dutch 83.06 percent (2.53 percent), English 89.21 percent (4.31 percent), Arabic 58.82 percent (12.49 percent), Hindi 42.92 percent (5.94 percent).
* Sad: Dutch 93.33 percent (3.60 percent), English 97.92 percent (2.09 percent), Arabic 69.61 percent (10.70 percent), Hindi 96.25 percent (4.15 percent).
* Happy: Dutch 65.28 percent (3.96 percent), English 95.49 percent (3.17 percent), Arabic 50.00 percent (11.90 percent), Hindi 86.67 percent (4.72 percent).
* Neutral: Dutch 87.22 percent (4.48 percent), English 89.86 percent (3.61 percent), Arabic 70.13 percent (8.14 percent), Hindi 20.42 percent (6.61 percent).
2.3. Experimental procedure
Experimental procedures closely followed those of Pell et al. (Reference Pell, Monetta, Paulmann and Kotz2009a). Participants were tested individually in sound-attenuated booths. Participants listened to each sentence presented one at a time; after each sentence, they were asked to judge how the speaker feels based on their voice by choosing one of five options (angry, fearful, happy, sad or neutral) using a mouse click. The experiment was blocked by language. For each language, participants first listened and responded to all 60 sentences in quiet and then the same sentences with background babble. Only the quiet listening conditions are reported here. The order of the languages was counterbalanced across participants, and the sentences were randomized. Prior to the main task, participants completed a practice session with five sentences per speaker (one for each emotion), with two in quiet and three in background babble.
The experiment was programmed and presented with E-Prime 3.0 software (Psychological Software Tools, Pittsburgh, PA). Stimuli were delivered through headphones (Edifier Hi-Fi-Stereo H840), with volume set at a consistent level between participants. Accuracy of responses was recorded throughout the experiment.
After the experiment, participants completed a series of questionnaires and tasks. First, they completed a debriefing survey related to the emotion recognition experiment. This survey was designed to gather feedback on which speakers and emotions the participants found easiest and most challenging to recognize. They then completed the LexTALE lexical decision task to measure receptive vocabulary in L2 English (Lemhöfer & Broersma, Reference Lemhöfer and Broersma2012). Lastly, they completed a language history questionnaire and the Schutte Self-Report Emotional Intelligence Test (Schutte et al., Reference Schutte, Malouff, Hall, Haggerty, Cooper, Golden and Dornheim1998) to measure emotional intelligence.
2.4. Accuracy analysis
Data analyses were conducted in R (Version 4.4.1, R Core Team, 2024). Generalized linear mixed-effects models with a binomial family function were fitted to behavioral accuracy data (1 = accurate, 0 = inaccurate) with the lme4 package (Bates et al., Reference Bates, Mächler, Bolker and Walker2015). Language (Dutch, English, Arabic, Hindi), Emotion (anger, fear, sadness, happiness) and their interaction were included as fixed effects. Item and Participant were included as random intercepts. Language and Emotion were coded using sum-to-zero contrasts. Note that non-emotional neutral filler stimuli were excluded from the statistical analyses, as in Pell et al. (Reference Pell, Monetta, Paulmann and Kotz2009a). Additionally, although care was taken to include the most highly rated stimuli in the experiment, the validation ratings for the stimuli differed between the four languages. Thus, mean-centered validation rating for each item was included as a covariate. Model fitting began with random by-participant slopes for language and emotion. Non-convergence was addressed by iteratively simplifying the random effects structure, first by removing the interaction between the random slopes, then removing the individual random slope for emotion, then for language. The final model reflects the maximally supported model (Barr et al., Reference Barr, Levy, Scheepers and Tily2013). To test the statistical significance of the fixed effects, type-III analysis-of-deviance tables were calculated, and Wald chi-square tests were conducted with the car package (Fox & Weisberg, Reference Fox and Weisberg2019). To evaluate the simple effects of each fixed effect, estimated marginal means were calculated, and pairwise comparisons were conducted with the emmeans package (Lenth and Piaskowski, Reference Lenth and Piaskowski2026); pairwise p-values were adjusted with the Tukey method.
3. Results
Overall, emotion recognition rates were high, and the target response was the most frequently selected response for each emotion and language combination. To more closely examine how listeners classified each emotion within each language, a stimulus–response confusion matrix was calculated from the responses, as seen in Figure 1. The resulting matrices revealed clear response patterns, indicating that classification was not random. Listeners demonstrated high overall accuracy for recognizing emotions (81.15%; over 4x 20% chance accuracy) and high accuracy for each language condition (all ps < .0001, calculated using emmeans from the model below).
Confusion matrix describing participant responses (%) for each language and emotion stimuli condition. Cells on the diagonal represent accurate emotion recognition; all other cells represent inaccurate recognition.
Figure 1. Long description
The figure contains four panels, each representing a different language stimulus. Each panel has a Y-axis labeled Stimulus Type (Neutral, Happy, Sad, Fearful, Angry) and an X-axis labeled Participant Response (Angry, Fearful, Sad, Happy, Neutral).
1. Dutch Stimuli (Blue color scale): The diagonal accuracy percentages are Angry 94.48, Fearful 93.98, Sad 83.73, Happy 90.72, and Neutral 71.29. A notable error is Happy stimuli being perceived as Neutral 20.80 percent of the time.
2. English Stimuli (Teal color scale): The diagonal accuracy percentages are Angry 95.42, Fearful 95.20, Sad 92.32, Happy 89.80, and Neutral 82.99.
3. Arabic Stimuli (Red color scale): The diagonal accuracy percentages are Angry 93.72, Fearful 94.61, Sad 89.30, Happy 85.45, and Neutral 61.22.
4. Hindi Stimuli (Orange color scale): The diagonal accuracy percentages are Angry 70.33, Fearful 73.54, Sad 51.17, Happy 93.67, and Neutral 46.90. Neutral was often confused for Sad (40.18 percent) and Angry was often confused for Fearful (24.55 percent).
We observed significant effects of Language
$ ({\chi}^2 $
(3) = 50.89, p < .001), Emotion
$ ({\chi}^2 $
(3) = 18.64, p < .001) and their interaction
$ ({\chi}^2 $
(9) = 47.39, p < .001) on accuracy. To probe the two-way interaction between Language and Emotion, we compared the levels of Language within each level of Emotion as shown in Figure 2.
Predicted emotion recognition accuracies by language and emotion type. Error bars represent 95% confidence intervals. Overall mean for each language indicated by black diamond, with bars indicating significant differences (** for p < .01 and *** for p < .001; dotted line represents chance performance at 20%).
Figure 2. Long description
The y-axis represents Predicted Accuracy from 0 percent to 100 percent, with a horizontal dashed line at 20 percent labeled Chance equals 20 percent. The x-axis lists four languages: Dutch, English, Arabic, and Hindi. A legend on the right identifies four emotions by color: Angry (blue), Fearful (orange), Sad (red), and Happy (teal).
For each language, four colored dots with vertical error bars represent the accuracy for each emotion, and a black diamond represents the overall mean.
Statistical significance bars at the top indicate that Hindi’s mean accuracy is significantly lower than Dutch (p less than .01), English (p less than .001), and Arabic (p less than .001).
For angry stimuli, predicted accuracies for Dutch (M = 0.95, 95% CI [0.89, 0.97], p = .003), English (M = 0.95, 95% CI [0.92, 0.97], p = .001) and Arabic (M = 0.94, 95% CI [0.9, 0.97], p = .003) were greater than for Hindi (M = 0.8, 95% CI [0.69, 0.88]). Dutch, English and Arabic accuracies did not differ (ps > .05). For fearful stimuli, predicted accuracy for Arabic (M = 0.97, 95% CI [0.95, 0.99]) was greater than for Dutch (M = 0.85, 95% CI [0.77, 0.91], p < .001) or Hindi (M = 0.77, 95% CI [0.57, 0.89], p < .001), which did not differ from each other. Predicted accuracies for English (M = 0.93, 95% CI [0.87, 0.96]) did not differ from the other three languages (ps > .05). For sad stimuli, predicted accuracies for Dutch (M = 0.9, 95% CI [0.83, 0.95], p = .006) and English (M = 0.95, 95% CI [0.89, 0.97], p < .001) were greater than for Hindi (M = 0.72, 95% CI [0.57, 0.83]). Accuracy for English was greater than Arabic (M = 0.81, 95% CI [0.71, 0.89], p = .04), which did not differ from the other two languages (ps > .05). Predicted accuracies for happy stimuli did not differ between the four languages (ps > .05).
We also probed the Language-by-Emotion interaction by comparing levels of Emotion within each level of Language. Within Dutch stimuli, predicted accuracy for anger was greater than for happiness (p = .03). Predicted accuracy for sad and fear did not differ from each other or from the other emotions within Dutch stimuli (ps > .05). Within English stimuli, predicted accuracy for anger, sadness and fear were greater than for happiness (M = 0.81, 95% CI [0.69, 0.89]; p = .008, p = .007, p = .03), but did not differ from one another (ps > .05). Within Arabic stimuli, predicted accuracy for anger and fear were greater than for sadness (p = .005, p < .001), and predicted accuracy for fear was also greater than for happiness (p = .02). None of the other comparisons within Arabic stimuli was significant (ps > .05). Within Hindi stimuli there were no significant differences between conditions (ps > .05).
To follow up on the main effects of Language and Emotion, we investigated their simple effects. For Language, averaging across levels of Emotion, predicted accuracies for Dutch (M = 0.89, 95% CI [0.85, 0.92], p = .002), English (M = 0.92, 95% CI [0.89, 0.95], p < .001) and Arabic (M = 0.93, 95% CI [0.89, 0.95], p < .001) were significantly higher than for Hindi (M = 0.79, 95% CI [0.73, 0.84]). No other pairwise comparisons within Language were significant (ps > .05). For Emotion, averaging across levels of Language, predicted accuracies were greater for angry (M = 0.93, 95% CI [0.9, 0.95]) than for sad (M = 0.87, 95% CI [0.82, 0.91]; p = .02) or happy (M = 0.85, 95% CI [0.81, 0.89]; p = .003) stimuli. Predicted accuracies were greater for fearful (M = 0.91, 95% CI [0.87, 0.94]) than for happy (p = .04) stimuli. No other pairwise comparisons within Emotion were significant (ps > .05).
In addition to the effects of Language and Emotion, there was a significant effect of validation rating such that an increase in validation rating was associated with an increase in accuracy
$ ({\chi}^2 $
(1) = 13.33, p < .001).
3.1. Exploratory correlations with language proficiency measures
To further explore the relationship between listeners’ emotion recognition and language ability, we calculated Pearson’s correlations between raw emotion recognition accuracy for L1 Dutch and L2 English and various language measures, including English LexTALE scores, self-rated speaking proficiency in both Dutch and English and English age of acquisition. We corrected for multiple correlations using Benjamini–Hochberg correction for false discovery rate. For simplicity, only significant correlations between emotion recognition accuracy and language measures will be reported here, with the full correlation matrix shown in Figure 3. English LexTALE scores were positively correlated with English emotion recognition accuracy (r = 0.38, p = .008, p adj = .04).
Pearson’s correlations between Dutch and English emotion recognition accuracy and language proficiency measures, including self-rated Dutch proficiency, self-rated English proficiency, English LexTALE score and English age of acquisition (AoA). *p adj < .05, **p adj < .01, ***p adj < .001.
Figure 3. Long description
The heatmap displays correlation values in a grid where blue indicates positive correlation and red indicates negative correlation.
Row 1, Dutch Accuracy:
- With English Accuracy: 0.43 asterisk.
- With Dutch Proficiency: minus 0.01.
- With English Proficiency: 0.08.
- With English Lex T A L E: 0.32.
- With English A o A: minus 0.03.
Row 2, English Accuracy:
- With Dutch Proficiency: 0.03.
- With English Proficiency: 0.21.
- With English Lex T A L E: 0.38 asterisk.
- With English A o A: minus 0.08.
Row 3, Dutch Proficiency:
- With English Proficiency: minus 0.05.
- With English Lex T A L E: minus 0.20.
- With English A o A: minus 0.02.
Row 4, English Proficiency:
- With English Lex T A L E: 0.49 double asterisk.
- With English A o A: minus 0.36 asterisk.
Row 5, English Lex T A L E:
- With English A o A: minus 0.25.
A vertical color bar on the right scales from 1 at the top in dark blue, through 0 in white, to minus 1 at the bottom in dark red.
3.2. Subjective ratings of emotion recognition difficulty
In a debriefing questionnaire, participants were asked to rate their perceived difficulty of recognizing emotions in each language condition on a scale of 1 (extremely easy) to 5 (extremely difficult). The percentage of participants who selected each response for each language is presented in Table 2. For both English and Dutch stimuli, the majority of participants indicated that emotions were somewhat easy to recognize. In contrast, for Arabic and Hindi stimuli, the majority of participants reported that emotions were somewhat difficult to recognize.
Participants’ subjective difficulty ratings for recognizing Dutch, English, Arabic and Hindi emotions and overall emotion recognition difficulty (% of participants)
Table 2. Long description
The table consists of six columns. The first column lists the Language or category, followed by five columns under the heading Response: Extremely easy (%), Somewhat easy (%), Neither easy nor difficult (%), Somewhat difficult (%), and Extremely difficult (%).
* Dutch: 22.92 Extremely easy, 54.17 Somewhat easy, 6.25 Neither, 12.5 Somewhat difficult, 4.17 Extremely difficult.
* English: 20.83 Extremely easy, 54.17 Somewhat easy, 10.42 Neither, 10.42 Somewhat difficult, 4.17 Extremely difficult.
* Arabic: 4.17 Extremely easy, 16.67 Somewhat easy, 14.58 Neither, 50.00 Somewhat difficult, 14.58 Extremely difficult.
* Hindi: 0.00 Extremely easy, 20.83 Somewhat easy, 12.50 Neither, 35.42 Somewhat difficult, 31.25 Extremely difficult.
* Overall task difficulty: 0.00 Extremely easy, 27.08 Somewhat easy, 18.75 Neither, 54.17 Somewhat difficult, 0.00 Extremely difficult.
4. Discussion
This study examined to what extent the commonly found “in-group” advantage in emotion recognition (i.e., greater accuracy for emotion recognition in the L1 compared to foreign languages) extends to a bilingual’s L2, in bilinguals who are not immersed in their L2 and are not a member of the cultural in-group. Additionally, we explored to what extent emotion recognition for unfamiliar foreign languages – languages for which the listener neither speaks nor has cultural affiliation with – compared to recognition in the L1 and L2. To do this, we examined Dutch-English bilinguals’ recognition of vocal anger, sadness, fear, happiness and neutral emotion in pseudo-utterances in their L1 (Dutch), highly proficient L2 (English) and two unfamiliar languages (Hindi and Arabic).
This study tested three hypothesized outcomes: the language-proficiency advantage hypothesis in which the in-group advantage extends to the highly proficient L2, resulting in similar L1 and L2 emotion recognition with both exceeding unfamiliar languages; the cultural-membership hypothesis in which the in-group advantage does not extend to the L2, resulting in L2 accuracy similar to unfamiliar accuracy or the L2 modulation hypothesis in which the L2 modulates the in-group advantage, such that emotion recognition is highest in the L1, followed by the L2, with the lowest performance observed in the unfamiliar languages. The findings are discussed in detail below.
4.1. Universality of emotion recognition
Overall, listeners’ predicted accuracy for all languages and emotions was high, exceeding chance accuracy by at least three times. These findings replicate previous research demonstrating high accuracy in emotion recognition for both familiar and unfamiliar languages (i.e., Elfenbein & Ambady, Reference Elfenbein and Ambady2002; Laukka & Elfenbein, Reference Laukka and Elfenbein2021; Paulmann & Uskul, Reference Paulmann and Uskul2014; Pell et al., Reference Pell, Monetta, Paulmann and Kotz2009a; Scherer et al., Reference Scherer, Banse and Wallbott2001; Thompson & Balkwill, Reference Thompson and Balkwill2006; Zhang & Pell, Reference Zhang and Pell2022). Given that our stimuli were pseudo-utterances – with no linguistic or semantic information – listeners demonstrated the ability to recognize emotion across both known and unfamiliar languages from prosodic features alone, supporting the universality of vocal emotion recognition (Elfenbein & Ambady, Reference Elfenbein and Ambady2002; Scherer et al., Reference Scherer, Banse and Wallbott2001).
4.2. Findings on L2 emotion recognition
As expected, listeners were highly accurate at identifying emotions in their L1, Dutch. Listeners were also highly accurate at recognizing emotions in their L2, English. In fact, we did not find significant differences in emotion recognition accuracy between L1 Dutch and L2 English overall, or for any specific emotions. These results support the language-proficiency advantage hypothesis – we find the in-group advantage for emotion recognition extends to a highly proficient L2.
Critically, these findings suggest that even bilinguals living and tested in their L1 cultural environment (who are not members of the L2 cultural in-group, and the majority of whom have spent no time living in the L2 native cultural environment) can learn to recognize emotion in their L2 at a level comparable to emotion recognition in their L1. These results inform existing theories of emotion recognition such as the theory of cultural learning in emotional behavior (Elfenbein & Ambady, Reference Elfenbein and Ambady2002) and the dialect theory (Elfenbein et al., Reference Elfenbein, Beaupré, Lévesque and Hess2007; Laukka et al., Reference Laukka, Elfenbein, Thingujam, Rockstuhl, Iraki, Chui and Althoff2016), which emphasize the need for culturally specific aspects of emotional behavior to be learned via direct cultural membership.
Our results suggest, at least for highly proficient L1 Dutch speakers of L2 English who learned their L2 in a school setting, that cultural in-group membership is not a prerequisite for achieving native-like levels of emotion recognition and point to language proficiency as an important factor. For highly proficient L2 listeners, interactions from classroom learning and engagement with other L2 English or native English speech (e.g., via social media or television) in their L1 Dutch environment are sufficient for learning the acoustic-prosodic cues necessary for emotion recognition, even without L2 cultural membership. This conclusion aligns with research that emphasizes that vocal expressions of emotion can be processed without extensive cultural learning (Sauter et al., Reference Sauter, Eisner, Ekman and Scott2010b). In other words, listeners can extract and interpret emotional prosodic cues across languages, with greater language proficiency promoting greater basic emotion recognition accuracy in that language. Consequently, bilinguals without L2 cultural membership are still able to learn and extract key acoustic-prosodic cues through language learning and interaction alone, achieving recognition performance comparable to that of their L1.
Further supporting this role of language proficiency as a crucial factor in L2 emotion recognition, we find significant positive correlations between English accuracy and language proficiency (the LexTALE test of vocabulary knowledge). While previous studies (Bhatara et al., Reference Bhatara, Laukka, Boll-Avetisyan, Granjon, Elfenbein and Bänziger2016; Graham et al., Reference Graham, Hamblin and Feldstein2001) did not find significant correlations between L2 proficiency and emotion recognition, here we tested highly proficient L2 users (rather than lower-level L2 learners as in Graham et al., Reference Graham, Hamblin and Feldstein2001) and used semantically meaningless pseudo sentences (eliminating potential negative effects of proficiency as seen in Bhatara et al., Reference Bhatara, Laukka, Boll-Avetisyan, Granjon, Elfenbein and Bänziger2016 who used stimuli with real words). Positive correlations found here suggest that L2 emotion recognition can be modulated by language proficiency, giving further evidence that L2 prosodic cues can be learned without in-group cultural membership.
It is important to note, however, that it is difficult to fully disentangle language familiarity with cultural exposure. More specifically, in the L1 environment of the present study’s listener group (the Netherlands), English is widely used as a second language. A large majority of the population (approximately 90%) is proficient in English (European Commission, 2012). This high level of proficiency is reflected in the country’s high ranking in the English Proficiency Index (Education First, 2024), scoring above the global average. Within the country, English is commonly used in education, business and tourism, and many Dutch people are fluent in English, especially in urban areas and among younger generations (Edwards, Reference Edwards2016). Although our participants are not L2 English in-group members, they likely have substantial exposure to English-speaking individuals or communities through various channels such as academic settings, interactions with other L2 or L1 English speakers, and via virtual channels such as English-language content on the internet, non-dubbed films, online gaming or social media. L2 language classes themselves often include authentic materials from English-speaking countries and expose students to cultural “mandates” or “scripts” (i.e., Goddard & Wierzbicka, Reference Goddard and Wierzbicka2004; Mesquita et al., Reference Mesquita, Boiger and De Leersnyder2017). As a result, participants’ proficiency in English most likely also provides them with considerable cultural context for successful emotion recognition in their L2, and this cultural exposure likely increases with increased language proficiency.
Disentangling language proficiency and cultural exposure is empirically difficult, particularly in today’s highly globalized, interconnected world. “In-group” membership as typically described (i.e., Elfenbein et al., Reference Elfenbein, Beaupré, Lévesque and Hess2007; Elfenbein & Ambady, Reference Elfenbein and Ambady2002) may no longer be a relevant categorization as individuals around the world are increasingly exposed to other cultures virtually via easily accessible content on the internet and social media. Nonetheless, future research could attempt to more directly examine the differential effects of language proficiency versus cultural exposure in at least two ways. First, studies could include a questionnaire to characterize participants’ prior cultural exposure or cultural characteristics (for an example, see De Groot et al., Reference De Groot, Schaafsma, Castelain, Malinowska, Mann, Ohtsubo, Wulandari, Bataineh, Fry, Goudbeek and Suryani2021). Second, future studies could explicitly compare a group learning an L2 with no or little exposure to the native culture versus a group learning an L2 with exposure to or immersion in a native cultural environment to better isolate the effects of language versus culture (see, for example, Hess et al., Reference Hess, Thibault, Levesque, Fontaine, Scherer and Soriano2013).
4.3. Findings on unfamiliar language emotion recognition
Further supporting the language-proficiency advantage hypothesis, we find an overall advantage for both Dutch and English compared to Hindi. This effect replicates in-group advantages found by Zhang and Pell (Reference Zhang and Pell2022), who found that Chinese-English bilinguals immersed in their L2 environment were better at emotion recognition in both their L1 and L2 compared to Hindi as a foreign language. Specifically, we find that this effect was driven by negative emotions, anger and sadness. Interestingly, although Arabic and Hindi are both unfamiliar languages to our Dutch-English bilingual participants, emotion recognition was more accurate for Arabic than Hindi. In fact, overall results indicate that Arabic accuracy did not significantly differ from English or Dutch. These results join only a few studies that have not found clear in-group advantages in vocal emotion recognition (i.e., McCluskey & Albas, Reference McCluskey and Albas1981; Zhu, Reference Zhu2013).
A potential explanation for better performance in one unfamiliar language over the other may be related to indirect exposure and the population demographics of the participants’ home country. The Netherlands is home to large numbers of Arabic-speaking immigrant groups (i.e., from Morocco and Syria; ETIAS, 2024). Although our participants may not have had direct experience with or exposure to Arabic (in fact, only six participants reported having any level of exposure to Arabic), it is conceivable that our participants may often overhear Arabic conversations in their daily lives. There is ample evidence that passive exposure to a foreign language (without being able to speak or use that language) leads to better auditory perception of that language and can incur lasting changes in brain structure (e.g., Au et al., Reference Au, Knightly, Jun and Oh2002; Barbeau et al., Reference Barbeau, Pierce, Deschamps, Kousaie, Gilbert, Chen, Baum and Klein2025; Bice & Kroll, Reference Bice and Kroll2019; Oh et al., Reference Oh, Au and Jun2010; Pierce et al., Reference Pierce, Chen, Delcenserie, Genesee and Klein2015). Even indirect exposure to Arabic language (and/or culture; as we discussed above, these two variables are difficult to disentangle) may have contributed to our participants’ increased ability to recognize emotions in Arabic prosody. If so, the language-proficiency advantage hypothesis supported by our findings may also feature an exposure component, where indirect forms of exposure may be sufficient for high performance in vocal emotion recognition. Future research could more systematically address the effect of indirect language exposure on emotion recognition, for example, by exposing participants to an unknown language or testing listeners that live in multi-cultural environments and examining changes in emotion recognition ability.
If indirect exposure to Arabic is indeed the driving cause of increased accuracy for Arabic emotion recognition, it is interesting to note that participants did not seem to be aware of this consciously. Again, only six participants reported any amount of Arabic exposure. Additionally, while participants tended to rate emotion recognition for both English and Dutch as “somewhat easy” or “extremely easy,” they tended to rate both Arabic and Hindi as “somewhat difficult” or “extremely difficult.” This would indicate that while Dutch-English bilinguals are highly accurate at recognizing emotions in Arabic sentences, subjectively this still felt more difficult than recognizing emotion in English or Dutch. Future research should continue to investigate the relationship between emotion recognition accuracy and other self-reported measures such as perceived confidence or perceived confusions with other emotions.
4.4. Limitations and future directions
Aspects of our experimental design and our findings leave some unanswered questions and potential directions for future research. To ensure that differences across the presented languages were not driven by baseline differences in stimuli quality, we incorporated norming values and random effects for items and participants in our generalized linear mixed-effects model. Thus, all reported comparisons attempt to control for potential baseline differences in stimuli quality. Nonetheless, we note that norming ratings still explained a certain portion of the observed variance in accuracy, and validity may be limited due to the fact that norming ratings are derived from separate validation studies (i.e., our validation for Dutch stimuli, versus English, Hindi and Arabic validation from Pell et al., Reference Pell, Paulmann, Dara, Alasseri and Kotz2009b). Additionally, our experiment included only one speaker of each language. While stimuli were normed by native speakers of each language and although this format has been used in previous research (i.e., Pell et al., Reference Pell, Monetta, Paulmann and Kotz2009a; Zhang & Pell, Reference Zhang and Pell2022), it is nonetheless conceivable that potential differences in performance could be influenced by speaker-specific rather than language-specific prosodic features (for examples of studies on emotional prosody that use multiple speakers per language see Banse & Scherer, Reference Banse and Scherer1996; Goudbeek & Broersma, Reference Goudbeek and Broersma2010; Goudbeek & Scherer, Reference Goudbeek and Scherer2010; Laukka et al., Reference Laukka, Juslin and Bresin2005; Liang et al., Reference Liang, Goudbeek, Konopka, Choi and Broersma2025).
Our study may also be limited by the specific languages chosen for comparison to the participants’ L1 and L2. Hindi and Arabic were chosen as the two “unknown” languages to facilitate comparisons with previous research who have used these stimuli with monolinguals (i.e., Pell et al., Reference Pell, Monetta, Paulmann and Kotz2009a; Zhang & Pell, Reference Zhang and Pell2022). However, Hindi and Arabic differ from Dutch and English in more ways than simply being unfamiliar languages – they belong to different language families and vary in linguistic distance from the participants’ known languages (see also Bradlow et al., Reference Bradlow, Clopper, Smiljanic and Walter2010; Scherer et al., Reference Scherer, Banse and Wallbott2001). Future research could more directly investigate these effects by selecting “unknown” languages that share the same language family or have comparable linguistic distance to the known languages.
Lastly, previous work has highlighted alternative perspectives on basic emotion theory and the universality claims in emotion recognition (see Lorette & Dewaele, Reference Lorette and Dewaele2024, for a discussion). Alternative perspectives argue against the “basic” emotion and universality accounts, instead emphasizing the cultural relativity of emotion perception (i.e., Barrett et al., Reference Barrett, Mesquita and Gendron2011) and providing evidence that emotion recognition is highly context-dependent (i.e., Barrett, Reference Barrett2011; Gendron et al., Reference Gendron, Roberson, Van Der Vyver and Barrett2014; Nelson & Russell, Reference Nelson and Russell2013). Along these lines, this alternative approach would argue that methodological factors – particularly the use of forced-choice experimental tasks – introduce a particular context that can artificially inflate emotion recognition accuracy, especially in cross-cultural contexts (i.e., Gendron et al., Reference Gendron, Roberson, Van Der Vyver and Barrett2014). Compared to methodologies such as open-ended response formats, forced-choice tasks like ours may prime emotion-concept knowledge, leading to inflated accuracy (Gendron et al., Reference Gendron, Roberson, Van Der Vyver and Barrett2014). This particular methodological context (forced-choice task) could be an alternative explanation for the high accuracies we observed, particularly the unexpectedly high recognition observed for Arabic stimuli. Although these perspectives suggest that some universality effects may be influenced by task structure, our findings do align with empirical studies that robustly observe universality effects (e.g., Paulmann & Uskul, Reference Paulmann and Uskul2014; Pell et al., Reference Pell, Monetta, Paulmann and Kotz2009a; Scherer et al., Reference Scherer, Banse and Wallbott2001; Thompson & Balkwill, Reference Thompson and Balkwill2006; Zhang & Pell, Reference Zhang and Pell2022), demonstrating that prosodic features can be reliably recognized in this particular experimental context. Future research could systematically compare potential task-related factors by including response options that allow participants to indicate uncertainty, such as a “cannot tell” choice, as implemented in previous studies (Lorette & Dewaele, Reference Lorette and Dewaele2019, Reference Lorette and Dewaele2020, Reference Lorette and Dewaele2024), or by using free-response formats (Gendron et al., Reference Gendron, Roberson, Van Der Vyver and Barrett2014).
5. Conclusions
Our results suggest that the in-group advantage in vocal emotion recognition does extend to a highly proficient L2 – even for participants living and tested in their L1 cultural environment who have limited experience living in their L2’s native cultural environment. Furthermore, correlations with language proficiency support that L2 emotion recognition can be learned and that this learning corresponds with increases in general language proficiency. These results challenge and extend earlier findings that listeners consistently show strong advantages in emotion recognition for their cultural in-group or native language (i.e., Elfenbein & Ambady, Reference Elfenbein and Ambady2002; Paulmann & Uskul, Reference Paulmann and Uskul2014; Pell et al., Reference Pell, Monetta, Paulmann and Kotz2009a; Zhang & Pell, Reference Zhang and Pell2022). Furthermore, our findings challenge theories such as the theory of cultural learning in emotional behavior and the dialect theory (Elfenbein, Reference Elfenbein2013; Laukka et al., Reference Laukka, Elfenbein, Thingujam, Rockstuhl, Iraki, Chui and Althoff2016), which suggest that in-group advantages in emotion recognition arise from culturally specific aspects of emotional communication within a cultural community. Instead, our results point to the role of language proficiency, suggesting that high levels of emotion recognition can be achieved with high levels of language proficiency, even without being part of a cultural in-group. Overall, these findings augment the current literature on emotion recognition and add to the small number of existing studies investigating the role of language proficiency and emotion recognition in bilinguals’ two languages. As we argued above, future work is needed to further define the specific contributions of cultural exposure and language proficiency for L2 and unfamiliar language emotion recognition.
Data availability statement
The data of this study, supplementary analyses and R code used for data analysis are openly available in OSF at https://osf.io/6ep4w/?view_only=426ebcdc87274f0c8ff85004cdae3c49.
Acknowledgments
This work is supported by NSF grants OISE-1545900, BCS-2041264 and DGE NRT 2125865 to Janet G. van Hell, a University Graduate Fellowship from the Pennsylvania State University to Adriana Miller and an NSF Graduate Research Fellowship Program Grant No. DGE1255832 to Cristal Giorio. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. We are grateful to Dr. Marc Pell for sharing the English, Hindi and Arabic stimuli sets for this experiment. We are grateful to Ana Cecilia Jaen Pabon and Mavel Pineda Velasquez for their assistance with data collection, and Margret Beuningen for her administrative assistance during data collection.
Competing Interests
The authors declare none.
Author contribution
Author roles were classified using the Contributor Role Taxonomy (CRediT; https://credit.niso.org/) as follows: Conceptualization: A.R.M., S.B., J.G.H; Formal analysis: A.R.M.; Funding acquisition: J.G.H; Investigation: A.R.M., G.S.; Writing – original draft: A.R.M., C.G.; Writing – review & editing: A.R.M., C.G., G.S., S.B., J.G.H.
Open access
