Introduction
The interaction between language and emotion has become a topic of great interest in the last decade (see Hinojosa et al., Reference Hinojosa, Moreno and Ferré2020, for a review). With language, we can conceptualize, express, and elicit an emotion. Many words have an affective (emotional) content (e.g., “love,” “hate,” “friend,” “murderer”). This emotional content is a consistent feature of language, and it can refer to internal affective states, processes, and relationships. Considering that the substantial effort devoted in the last years to characterize the effects of emotional content on word processing is not surprising (e.g., Kuperman et al., Reference Kuperman, Estes, Brysbaert and Warriner2014; Rodríguez-Ferreiro & Davies, Reference Rodríguez-Ferreiro and Davies2019; Vinson et al., Reference Vinson, Ponari and Vigliocco2014). This study focuses on a much less explored issue, that is, the contribution of affective content to the organization of words in the speakers’ lexicon.
According to dimensional models of emotion, the affective properties of words can be characterized in terms of two basic dimensions: valence and arousal (Bradley & Lang, Reference Bradley and Lang1999; Russell, Reference Russell2003; Russell & Barrett, Reference Russell and Barrett1999). Valence refers to the hedonic value describing whether a word is positive or negative, and it is often assessed on an unpleasant to pleasant scale (e.g., “hate” is an unpleasant or negative word, while “love” is a pleasant or positive word). On the other hand, arousal refers to the level of activation (from low arousal to high arousal) a word conveys (i.e., the extent to which its meaning refers to something calming [e.g., “peace”] or activating [e.g., “party”]; Russell, Reference Russell1980; Altarriba & Bauer, Reference Altarriba and Bauer2004; Citron et al., Reference Citron, Weekes and Ferstl2014). Therefore, by definition, affective words (i.e., words that have an emotional content) have a polarized score on the valence scale (i.e., they are perceived as highly pleasant or unpleasant). Regarding the level of arousal, there is variability among affective words, although most of them have high arousal values. On the other hand, neutral words (i.e., non-affective words, those that are neither positive nor negative) have a valence value around the middle point of the scale and a low level of arousal (e.g., “chair,” “pen”; Stadthagen-Gonzalez et al., Reference Stadthagen-Gonzalez, Imbault, Pérez Sánchez and Brysbaert2017).
Neurocognitive and behavioral research has shown that valence and arousal can affect word processing. Most studies have examined whether the processing of affective words differs from that of neutral words (from which they commonly differ in both valence and arousal). These studies have found that emotional content facilitates word processing (Kousta et al., Reference Kousta, Vinson and Vigliocco2009; Schacht & Sommer, Reference Schacht and Sommer2009; Citron, Reference Citron2012; Yao et al., Reference Yao, Yu, Wang, Zhu, Guo and Wang2016). Nonetheless, this advantage is more frequently associated with positive valence (Hofmann et al., Reference Hofmann, Kuchinke, Tamm, Võ and Jacobs2009). In contrast, the effect of negative valence is unclear: Some studies have reported an advantage in processing for negative words (Kousta et al., Reference Kousta, Vinson and Vigliocco2009), while others have reported a disadvantage (Estes & Adelman, Reference Estes and Adelman2008; Larsen et al., Reference Larsen, Mercer, Balota and Strube2008) or no effects at all (Scott et al., Reference Scott, O’Donnell and Sereno2014). Other studies have found that arousal alone can influence word processing, and some others have reported an interaction between valence and arousal particularly in negative valenced words (Kuperman et al., Reference Kuperman, Estes, Brysbaert and Warriner2014; Delaney-Busch et al., Reference Delaney-Busch, Wilkie and Kuperberg2016; see also Hinojosa et al., Reference Hinojosa, Moreno and Ferré2020, for a review).
Apart from having distinct valence and arousal values, affective words may also differ in their relationship with emotional content. Concretely, there is a relevant distinction between emotion words (EM words henceforth) and emotion-laden words (EL words henceforth). EM words directly refer to a specific emotion, that is, they denote an emotion (e.g., “love,” “hate”), while EL words do not refer to an emotion but can elicit it (e.g., “party,” “knife”; Altarriba, Reference Altarriba2006; Pavlenko, Reference Pavlenko2008; Zhang et al., Reference Zhang, Wu, Meng and Yuan2017). In other words, the affective content of EM words comes from their direct reference to an emotion, while the affective content of EL words is probably a product of the association of the word to an affective state/event. Most of the previous research in affective word processing has not taken into account this distinction, intermixing EM and EL words in their experimental materials (Kousta et al., Reference Kousta, Vigliocco, Vinson, Andrews and Del Campo2011; Palazova et al., Reference Palazova, Mantwill, Sommer and Schacht2011; Yap & Seow, Reference Yap and Seow2014; Chen et al., Reference Chen, Lin, Chen, Lu and Guo2015). This may have contributed to the inconsistency of findings in the field.
Nonetheless, the distinction between EM and EL words is also relevant, considering that the few studies that have compared these two types of words have, indeed, found differences in their processing. For instance, some behavioral studies have reported a greater emotional activation, shorter response times, and larger priming effects for EM words when compared to EL words (Altarriba & Basnight-Brown, Reference Altarriba and Basnight-Brown2011; Kazanas & Altarriba, Reference Kazanas and Altarriba2015). Furthermore, research involving event-related potential recording has also reported differences between EM and EL words. Indeed, Wang et al. (Reference Wang, Shangguan and Lu2019) found that positive EM words elicited a larger P2 amplitude than neutral words, but no effects were found in the comparison between EL and neutral words. These results indicate that emotion effects are observed earlier for EM words than for EL words. Zhang et al. (Reference Zhang, Teo and Wu2019) also found, during conflict processing, a larger amplitude of the N200 component for both positive and negative EM words, indicating a greater early emotional activation in comparison to positive and negative EL words.
Another important distinction between EM and EL words has to do with emotional prototypicality. Emotional prototypicality is the extent to which a word refers to an emotion. Therefore, it is a unique feature of EM words. In contrast, as EL words do not describe emotions, they do not have emotional prototypicality. This unique feature has been associated with a facilitative effect during the recognition of EM words, that is, more prototypical EM words are recognized faster than less prototypical EM words (Haro et al., Reference Haro, Calvillo, Poch, Hinojosa and Ferré2022).
Therefore, differences in processing between EM and EL words might be related to differences between these two types of words in their relationship with emotions. As mentioned earlier, this relationship would be more direct for EM words than for EL words. In order to address this issue, it might be helpful to examine the organization of EM and EL words in memory. As far as we know, this research question has not been addressed before. The main aim of this study was to examine the organization of EM and EL words in memory by using a free association task. In this task, participants are asked to respond as quickly as possible to a cue word with the first word that comes to his/her mind (e.g., Coronges et al., Reference Coronges, Stacy and Valente2007; Ludueña et al., Reference Ludueña, Behzad and Gros2014; Vivas et al., Reference Vivas, Manoiloff, García, Lizarralde and Vivas2019). The set of associates to any cue word is obtained by asking large samples of participants to perform the task. Each associated word has an associative strength value, computed as the proportion of participants who have provided that specific word in response to a cue word (Nelson et al., Reference Nelson, McEvoy and Dennis2000; De Deyne et al., Reference De Deyne, Navarro, Perfors, Brysbaert and Storms2019). Associative strength indicates the connection between two words in the lexicon (De Deyne et al., Reference De Deyne, Navarro and Storms2013; Van Rensbergen et al., Reference Van Rensbergen, Storms and De Deyne2015). Consequently, free association databases provide the list of associated words elicited by a set of cue words, ordered in terms of their associative strength, with the first associated word being the one produced more frequently by the speakers.
Although word association datasets exist in various languages and some studies have described the characteristics of the cues/associates, only a few of them have considered the affective properties of words. For example, Altarriba et al. (Reference Altarriba, Bauer and Benvenuto1999) compared three types of cue words: abstract, concrete, and emotion words, finding differences between them in the number of associates produced and in their associative strength. Specifically, they observed that emotion words generated more associates, followed by abstract words and then concrete words. In addition, the first associate of concrete words showed a significantly greater associative strength than that of abstract and emotion words, without differences between the last two types of words.
In addition to the associative strength and the number of associates produced, the analysis of the characteristics of the words obtained from normative free association studies may contribute to the knowledge about the organizational structure of affective (and neutral) words in memory. Concretely, words can be characterized in terms of lexical, semantic, and affective properties, such as word frequency, concreteness, age of acquisition, valence, arousal, or part of speech, among others. There is evidence that words that share some of these properties are more likely to be connected in the mental lexicon. This was demonstrated in a study by Van Rensbergen et al. (Reference Van Rensbergen, Storms and De Deyne2015), who, using a large free association dataset in Dutch, examined the extent to which each cue word and its associates display similar properties, a phenomenon called assortativity (i.e., the extent to which the value of the cue word in a particular variable predicts the value of the associated words in the same variable; Vitevitch et al., Reference Vitevitch, Chan and Goldstein2014). Using linear regression analyses, they found a cue-associates correspondence for valence, arousal, concreteness, and dominance but not for word frequency, contextual diversity, and age of acquisition. These results indicated that some variables display assortativity and, therefore, are relevant in the organization of words in semantic memory. Similarly, in a study involving different languages (English, Dutch, and Spanish), Buades-Sitjar et al. (Reference Buades-Sitjar, Planchuelo and Duñabeitia2021) analyzed the predictive capacity of three properties of cue words (valence, arousal, and concreteness) on the characteristics of their associates. The results showed that the value of the cue word in each variable was a strong predictor of the value of the associated word in the same variable. Interestingly, these three variables had been previously identified in the study by Van Rensbergen et al. (Reference Van Rensbergen, Storms and De Deyne2015) as relevant properties in the organization of words in the lexicon. The studies of Van Rensbergen et al. (Reference Van Rensbergen, Storms and De Deyne2015) and Buades-Sitjar et al. (Reference Buades-Sitjar, Planchuelo and Duñabeitia2021) investigated the relationship between the characteristics of the cues and their associates without distinguishing between different types of cue words (e.g., affective vs. neutral words). We made such a distinction in the present study.
The present study
This study aimed to examine the semantic organization of EM, EL, and neutral (NT) words using a free association task. Our purpose was to analyze the characteristics of the words produced as associates to EM, EL, and NT cue words. We addressed this issue with a double approach. On the one hand, we analyzed the types of associated words produced for each type of cue word by classifying the associates into EM, EL, and NT words. On the other hand, we examined whether a set of affective (valence and arousal) and lexico-sematic variables (concreteness, frequency, and age of acquisition) display assortativity (i.e., a correspondence between the characteristics of the cues and the associates). This last issue was examined first by considering all the cues in general (like Buades-Sitjar et al., Reference Buades-Sitjar, Planchuelo and Duñabeitia2021 and Van Rensbergen et al., Reference Van Rensbergen, Storms and De Deyne2015) and then by distinguishing between the three types of cues (EM, EL, and NT).
With these objectives firmly in mind, we selected 840 Spanish cue words (including EM, EL, and NT words) and identified their first three associates based on their associative strength. Most of the associates were selected from an online database of free association norms in Spanish, which contains data for 6,739 cue words (Diez et al., Reference Diez, Alonso, Rodríguez and Fernandez2018). However, some of the selected cue words were not in the norms of Diez et al. (Reference Diez, Alonso, Rodríguez and Fernandez2018). Therefore, we carried out a word association task to collect their associates. Later we classified the associates into EM, EL, and NT words. Furthermore, we obtained the values of the cue words and their first associate in several lexico-semantic variables (valence, arousal, concreteness, age of acquisition, and frequency) to explore if they displayed assortativity.
The following research questions guided the study:
RQ1: Are words that share affective characteristics more likely to be connected in the mental lexicon and, if this is the case, is the EM–EL distinction relevant in terms of the organization of words in memory?
Hypothesis for RQ1: We expected EM words to be more consistently produced as associates to EM cues than to the other cues. Similarly, EL words would be elicited as associates to a greater extent by EL cues than by the other cues. Neutral words, in turn, were expected to be more consistently produced as associates to NT cues than to the other types of cues.
RQ2: Are EM associates elicited by EM cues more clear emotion words than EM associates elicited by the other types of cues?
Hypothesis for RQ2: We expected EM associates produced in response to EM cues to have a higher emotional prototypicality than EM associates produced in response to other (non-EM) cues.
RQ3: Do EL words acquire their affective properties through their relation to emotional states or events?
Hypothesis for RQ3: We expected EM words, which denote emotional states, to be produced as associates to a greater extent to EL cue words than to NT cue words.
RQ4: Do affective (valence and arousal) and lexico-sematic variables (concreteness, frequency, and age of acquisition) display assortativity?
Hypothesis for RQ4: Considering the results of Buades-Sitjar et al. (Reference Buades-Sitjar, Planchuelo and Duñabeitia2021) and Van Rensbergen et al., Reference Van Rensbergen, Storms and De Deyne2015), we expected valence, arousal, and concreteness to display assortativity (i.e., the values of the cues in these variables would predict the values of the associates in the same variables better than the values of the cues in any other variable). In contrast, frequency and age of acquisition would not display assortativity.
RQ5: Is assortativity for affective variables (i.e., valence and arousal) higher for EM and EL words than for neutral words?
Hypothesis for RQ5: Although we did not have a clear prediction regarding this research question, due to the exploratory nature of this analysis, it might be that assortativity for valence and arousal is greater in affective (EM and EL) words than in neutral words.
Methods
Materials
The stimuli consisted of 840 Spanish cue words. We assigned each cue word to one condition (EM, EL, and NT, see below). The EM and EL conditions had a total of 210 cue words each, while the NT condition consisted of 420 cue words. The EM cue words were obtained from the study of Pérez-Sánchez et al. (Reference Pérez-Sánchez, Stadthagen-Gonzalez, Guasch, Hinojosa, Fraga, Marín and Ferré2021). These authors collected emotional prototypicality ratings for 1,286 words. Using a 1–5 scale, the authors asked Spanish-speaking individuals to rate how strongly each word describes an emotion (1 = this word does not refer to an emotion; 5 = this word clearly refers to an emotion). We selected 210 EM words with a prototypicality score of 3 or higher (i.e., words that most Spanish speakers considered highly associated with emotions).
Further, we looked for the valence and arousal ratings of these EM cue words. Most of these ratings were retrieved from the emoFinder search engine (Fraga et al., Reference Fraga, Guasch, Haro, Padrón and Ferré2018), which contains different databases (Redondo et al., Reference Redondo, Fraga, Comesaña and Perea2005; Redondo et al., Reference Redondo, Fraga, Padrón and Comesaña2007; Ferré et al., Reference Ferré, Guasch, Moldovan and Sánchez-Casas2012; Guasch et al., Reference Guasch, Ferré and Fraga2016; Hinojosa et al., Reference Hinojosa, Martínez-García, Villalba-García, Fernández-Folgueiras, Sánchez-Carmona, Pozo and Montoro2016a; Stadthagen-Gonzalez et al., Reference Stadthagen-Gonzalez, Imbault, Pérez Sánchez and Brysbaert2017). Twenty-three EM cue words did not have valence and arousal values in emoFinder. For this reason, we collected these ratings through questionnaires. Like in Stadthagen-Gonzalez et al. (Reference Stadthagen-Gonzalez, Imbault, Pérez Sánchez and Brysbaert2017), valence and arousal ratings were obtained using a 9-point rating scale. For valence, 1 indicated that the word was highly negative/unpleasant, and 9 indicated that the word was highly positive/pleasant, while for arousal, 1 indicated that the word was very calming, and 9 indicated that the word was very exciting. We used the valence values to classify the EM cues as positive, negative, or neutral. Following Stadthagen-Gonzalez et al. (Reference Stadthagen-Gonzalez, Imbault, Pérez Sánchez and Brysbaert2017), words with a valence rating <4 were classified as negative, while those with a valence rating >6 were classified as positive, and words with valence values ranging from 4 to 6 were considered as neutral. Among the 210 EM selected cue words, there were 140 negative words and 70 positive words. Moreover, we used the Stadthagen-Gonzalez et al. (Reference Stadthagen-Gonzalez, Imbault, Pérez Sánchez and Brysbaert2017) database to search for the remaining cue words (EL and NT words) and their values in the valence and arousal dimensions. We obtained 210 EL words (distributed in the same manner as the EM words; 140 negative words and 70 positive words) and 420 neutral words.
Additionally, we collected data for several psycholinguistic properties of the cue words using emoFinder (Fraga et al., Reference Fraga, Guasch, Haro, Padrón and Ferré2018) and EsPal (Duchon et al., Reference Duchon, Perea, Sebastián-Gallés, Martí and Carreiras2013). Specifically, we obtained values for concreteness (Ferré et al., Reference Ferré, Guasch, Moldovan and Sánchez-Casas2012; Duchon et al., Reference Duchon, Perea, Sebastián-Gallés, Martí and Carreiras2013; Guasch et al., Reference Guasch, Ferré and Fraga2016; Hinojosa et al., Reference Hinojosa, Martínez-García, Villalba-García, Fernández-Folgueiras, Sánchez-Carmona, Pozo and Montoro2016a, Pérez-Sánchez et al., Reference Pérez-Sánchez, Stadthagen-Gonzalez, Guasch, Hinojosa, Fraga, Marín and Ferré2021), age of acquisition (Alonso et al., Reference Alonso, Fernandez and Díez2015; Hinojosa et al., Reference Hinojosa, Rincón-Pérez, Romero-Ferreiro, Martínez-García, Villalba-García, Montoro and Pozo2016b; Pérez-Sánchez et al., Reference Pérez-Sánchez, Stadthagen-Gonzalez, Guasch, Hinojosa, Fraga, Marín and Ferré2021), and word frequency measured as Zipf (Duchon et al., Reference Duchon, Perea, Sebastián-Gallés, Martí and Carreiras2013). Comparisons between the affective (EM and EL) and neutral cue words showed differences in valence, t(451) = 9.35, p < .001, and arousal, t(579) = 21.03, p < .001, while the EM and EL cue words were matched in both variables. The affective and psycholinguistic properties of the cue words are represented in Table 1.
Characteristics of EM, EL, and NT cue words
Note: All values are means and SD (±). Valence scale = 1–9; arousal scale = 1–9; concreteness scale = 1–7; age of acquisition scale = 1–11 (numbers indicate the age of acquisition: 1 = under 2 years old; 2–10 = 2–10 years old; 11 = 11 years old or older).
Procedure
The associates for most cue words were obtained from the free association norms of Diez et al. (Reference Diez, Alonso, Rodríguez and Fernandez2018). However, 67 EM cue words were not in the norms (note that the EM cue words came from the emotion prototypicality study of Pérez-Sánchez et al., Reference Pérez-Sánchez, Stadthagen-Gonzalez, Guasch, Hinojosa, Fraga, Marín and Ferré2021). Therefore, we collected the associates for these cue words through questionnaires. We also included a set of filler cue words. The reason was to use a procedure as similar as possible to that described by Diez et al., (Reference Diez, Alonso, Rodríguez and Fernandez2018). In that study, participants produced associates to cue words that were not distinguished by their affective properties (i.e., there could be EM, EL, and NT words among these cues). Moreover, a free association task involving only EM cues could be perceived as strange by participants, potentially leading to response biases.
Consequently, we constructed a series of questionnaires. The proportion of each type of cue was the same in all questionnaires: 50% EM cue words, 25% EL cue words, and 25% NT cue words. The questionnaires were responded by a total of 142 participants (mean = 41.73 per questionnaire; min = 30, max = 46): 117 females (82.39%) and 25 males (17.61%), whose mean age was 22.7 years (SD = 6.87). Each participant answered between 1 and 4 questionnaires. The questionnaires were completed online, and participants were instructed to read each cue word and answer with the first word that came to mind. There were 11 cues on each page. Participants were asked to answer each cue, although they could indicate that they did not know the word if that was the case. The cues were randomized for each participant, and they did not have a time limit by which to submit their responses.
Results
Data preprocessing
After collecting the associates, the data were normalized in several steps. Firstly, we removed capital letters (e.g., “Amor” -> “amor”) and corrected any special characters entered by the participants (e.g., “canciòn” -> “canción”). Plural responses to the same cue were collapsed with non-plural responses (e.g., “sentimientos” -> “sentimiento”), and any typographical error was corrected when the provided word was clear enough to be correctly interpreted (e.g., “asquerodo” -> “asqueroso”). Then we computed the associative strength of each associated word, that is, the proportion of participants who produced that word in response to a particular cue word (out of the total number of participants who responded to that cue word).
We then classified the associates into EM, EL, and NT words. To this end, we focused on the first three associated words. The reason was that the associates after the third position tend to have a low associative strength, indicating that a small number of participants have produced them. To identify the EM-associated words, we relied on prototypicality values taken from Pérez-Sánchez et al. (Reference Pérez-Sánchez, Stadthagen-Gonzalez, Guasch, Hinojosa, Fraga, Marín and Ferré2021). The associated words were considered as EM when their emotional prototypicality rating was greater or equal to three. Furthermore, to classify the rest of the associates as either EL or NT words, we relied on their valence ratings (using the same criteria employed to classify the cue words; see the Materials section). These ratings were taken from Stadthagen-Gonzalez et al. (Reference Stadthagen-Gonzalez, Imbault, Pérez Sánchez and Brysbaert2017). However, 195 associated words were not in that dataset. Hence, we constructed a series of questionnaires, following the same procedure as Stadthagen-Gonzalez et al. (Reference Stadthagen-Gonzalez, Imbault, Pérez Sánchez and Brysbaert2017). We created four questionnaires, which were responded by a total of 100 participants: 87 females (87%) and 13 males (13%), whose mean age was 21.68 (SD = 7.17).
Finally, to examine assortativity, we searched for the normative values of the variables to be examined in the analyses. Apart from valence and arousal, we searched for values of concreteness (Ferré et al., Reference Ferré, Guasch, Moldovan and Sánchez-Casas2012; Guasch et al., Reference Guasch, Ferré and Fraga2016; Hinojosa et al., Reference Hinojosa, Martínez-García, Villalba-García, Fernández-Folgueiras, Sánchez-Carmona, Pozo and Montoro2016a; Pérez-Sánchez et al., Reference Pérez-Sánchez, Stadthagen-Gonzalez, Guasch, Hinojosa, Fraga, Marín and Ferré2021), word frequency (as Zipf; Duchon et al., Reference Duchon, Perea, Sebastián-Gallés, Martí and Carreiras2013), and age of acquisition (Alonso et al., Reference Alonso, Fernandez and Díez2015; Hinojosa et al., Reference Hinojosa, Rincón-Pérez, Romero-Ferreiro, Martínez-García, Villalba-García, Montoro and Pozo2016b; Pérez-Sánchez et al., Reference Pérez-Sánchez, Stadthagen-Gonzalez, Guasch, Hinojosa, Fraga, Marín and Ferré2021) of the first associated word. There were no normative values for 299 words in concreteness and for 14 words in age of acquisition. Therefore, we collected ratings on these variables using questionnaires. To do so, we followed the same procedure as in the datasets of reference (Ferré et al., Reference Ferré, Guasch, Moldovan and Sánchez-Casas2012, for concreteness; Alonso et al., Reference Alonso, Fernandez and Díez2015, for age of acquisition). We elaborated 2 questionnaires with a total of 247 words each and they were responded by 97 participants: 88 females (91%) and 9 males (9%), whose mean age was 19.04 (SD = 2.55).
The file containing the cue words, the associated words, and the characteristics of both types of words is openly available in an Open Science Framework repository at https://osf.io/c8azn/.
The results are divided into two parts: 1) type of associated words produced and 2) assortativity.
Type of associated words produced
Data analysis
These analyses were related to RQ1, RQ2, and RQ3 and focused on the type of associated words elicited by each type of cue word. We computed the number of EM, EL, and NT associates produced by each type of cue word and their associative strength, considering the first three associates. We also computed the mean emotional prototypicality value of the EM associates produced in response to each type of cue word. Since we had 210 EM cues, 210 EL cues, and 420 NT cues, we selected a random subgroup of 210 NT words to have the same number of items per condition in the comparison between EM, EL, and NT cues. We ran a series of one-way ANOVAs for independent measures with the type of cue word as a factor (EM, EL, NT). Post-hoc analyses were done using the Bonferroni test (in case of normal distribution) and Tamhane correction (in case of non-normal distribution). The dependent variables were the number of EM, EL, and neutral words produced as associates (RQ1 and RQ3). Furthermore, we ran an independent t-test to compare the prototypicality of the EM words elicited as associates to EM cues and EL cues (RQ2). We did not include the NT cues in the analyses because they produced a very small number of associated EM words.
Results
The number of associated words of each type (EM, EL, and NT) produced in response to the distinct types of cue words is displayed in Table 2. Results showed a significant effect in the number of EM associates produced when comparing the three cue conditions (EM, EL, NT), F(2,627) = 503, p < .001, ηp2 = 0.62. Post hoc analyses (T2 Tamhane) revealed differences between the three groups (all ps < .05), indicating that EM cues produced more EM associates than EL and NT cues. At the same time, EL cues produced significantly more EM associates than NT cues.
Number of EM, EL, and NT (mean and SD) associated words produced by each type of cue word
The analysis of the number of EL-associated words produced revealed that the number of EL associates produced differed significantly between EM, EL, and NT cue words, F(2,627) = 83.61, p < .001, ηp2 = 0.21. Bonferroni post-hoc tests showed significant differences between the three groups (all ps < .05). Concretely, EL cue words elicited a significantly greater amount of EL associates than EM and NT cue words, while NT cue words produced more EL associates than EM cue words.
The analysis of the number of neutral associated words produced also revealed differences between EM, EL, and NT cue words, F(2,627) = 197.32, p < .001, ηp2 = 0.39. These differences were significant between all the groups (T2 Tamhane test, all ps < .05), indicating that NT cue words elicited more neutral associates than EM and EL cue words and that EL cue words produced more neutral associates than EM cue words.
Finally, the analysis of the prototypicality of the EM-associated words elicited in response to EM and EL cue words showed a significant difference, being prototypicality higher for EM words associated with EM cues (mean = 4.14, SD = 0.51) than for those associated with EL cues (mean = 3.86, SD = 0.62), t(231) = 3.30, p = 0.001, d = 0.52.
Assortativity
Data analysis
These analyses were related to RQ4 and RQ5 and examined whether valence, arousal, concreteness, word frequency (conceptualized as Zipf), and age of acquisition display assortativity in our dataset, focusing on the cue words and their first associate. As explained in the introduction, assortativity refers to the correspondence between the cue and the associate in relation to a particular variable (e.g., to what extent the valence score of the cue word predicts the valence score of the associated word; Vitevitch et al., Reference Vitevitch, Chan and Goldstein2014). One way to examine this is to compare the predictive capacity of several variables of the cue on the score of the associated word in a given variable (e.g., valence). When there is assortativity, the score of the cue word in that variable (e.g., valence) better predicts the score of the associated word in that variable (i.e., valence) than the scores of the cue word in other variables (e.g., arousal, concreteness, age of acquisition, and Zipf).
We ran five linear regression models, one for each variable of the first associated word, to examine the cue-response correspondence for the variables of interest. First, we did this considering all cue words together and then with each type of cue word separately.
Following Van Rensbergen et al. (Reference Van Rensbergen, Storms and De Deyne2015), we ran the analyses using the lmg metric in the R relaimpo package (Grömping, Reference Grömping2007). The metric lmg uses the R 2 , which is the percentage of variation in the dependent variable explained by the variation of the independent variable. The R 2 is a measure commonly used for regression models. However, when the R 2 is used in these models, the order in which the variables are entered may affect the outcome because each order can yield different results. The lmg metric solves this problem by averaging across all potential orders.
Results
The relative contribution of each cue variable to the variance explained by the model for a particular variable of the associated word (dependent variable, DV, in the first column of the table), considering the 840 cue words and their first associate, is represented in Table 3. The proportion of variance explained by the model is also shown (last column). The significant predictors (i.e., those whose predictive capacity was significantly higher than that of all other predictors) are in bold in the table and the ones in italic are significantly higher than at least one of the other predictors. We tested significance by examining the overlap of bootstrapped confidence intervals included in the lmg metric within the R relaimpo package (Grömping, Reference Grömping2007).
Proportion of variance of the first associated word in several variables that is explained by each variable (predictor) of the cue word (all cue words included)
Note: The “% Variance” column refers to the percentage of variance explained by the model. Values in bold are significantly higher from all other predictors and values in italic are significantly higher from at least one of the other predictors.
Regarding the valence of the first associate (first row in Table 3), the valence of the cue was the most significant predictor, followed by arousal, which differed significantly from concreteness, AoA, and Zipf. Similarly, the arousal of the cue was the most significant predictor of the arousal of the first associate (second row), followed by the valence of the cue, which was significantly different from concreteness, age of acquisition, and Zipf. Concerning the concreteness of the first associated word (third row), the concreteness of the cue word was the most significant predictor. When we compared the other predictors, we observed that age of acquisition was significantly different from valence and Zipf but not from arousal. Therefore, the second most significant predictor of the concreteness of the associate was the age of acquisition of the cue. Regarding the age of acquisition of the associate (fourth row), the only significant predictor was the age of acquisition of the cue word. Finally, all the predictors contributed equally to the Zipf of the first associate (fifth row).
The results of the analyses conducted with each type of cue word separately (EM, EL, and NT) are represented in Tables 4, 5, and 6, respectively. The analysis restricted to EM cues revealed assortativity only for valence and arousal (see Table 4). Hence, only the valence and arousal ratings of the EM cues are significant predictors of the valence and arousal ratings of the first associated word.
Proportion of variance of the first associated word in several variables that is explained by each variable (predictor) of the cue word (considering only EM cues)
Note: The “% Variance” column refers to the percentage of variance explained by the model. Values in bold are significantly higher than those of all other predictors.
Proportion of variance of the first associated word in several variables that is explained by each variable (predictor) of the cue word (considering only EL cues)
Note: The “% Variance” column refers to the percentage of variance explained by the model. Values in bold are significantly higher than those related to all other predictors and values in italic are significantly higher than at least one of the other predictors.
Proportion of variance of the first associated word in several variables that is explained by each variable (predictor) of the cue word (considering only NT cues)
Note: The “% Variance” column refers to the percentage of variance explained by the model. Values in bold are significantly higher than all other predictors and the values in italic are significantly higher than at least one predictor.
The analysis of EL cues showed assortativity for valence, arousal, and concreteness (see Table 5). The results also showed that arousal of the cue word was a good predictor of the valence of the associated word and different from concreteness and Zipf. Regarding the arousal of the associated word, valence was a significantly better predictor than Zipf. Furthermore, the age of acquisition of the cue word predicted the age of acquisition of the first associated word significantly better than valence, arousal, and Zipf, but no differences were found between age of acquisition and concreteness.
Lastly, the analysis of the NT cues indicated that valence and age of acquisition displayed assortativity, that is, those variables of the cues were the best predictors of the same variables of the first associated word (see Table 6). Additionally, the concreteness of NT cues was a better predictor of the concreteness of the first associated word than valence, arousal, and Zipf, but their predictive capacity was not significantly different from that of age of acquisition. Age of acquisition, in turn, was a significant better predictor of concreteness than valence, arousal, and Zipf.
Discussion
The aim of the present work was to investigate the contribution of affective content to the associative structure of words in the lexicon. To that end, we examined the characteristics of the words produced as associates to EM, EL, and NT cue words. We also examined the correspondence between several affective and lexico-semantic properties of the cue words and those of the associated words.
Our first goal was to investigate whether words that share affective characteristics are more likely to be connected in the mental lexicon, and, if this is the case, whether the EM–EL distinction is relevant in terms of the organization of words in memory. We predicted that EM, EL, and NT words would be elicited as associates to a greater extent by cue words of the same type (i.e., EM, EL, and NT cue words, respectively) than by the other types of cues. Our results supported this prediction. Indeed, the associates most frequently produced belonged to the same category of the cue that elicited them: EM cues produced more EM associates than EL and NT cues; EL cues produced more EL associates than EM and NT cues; and NT cues produced more NT associates than EM and EL cues. We also predicted that EM-associated words produced in response to EM cue words would have a higher emotional prototypicality than those elicited by EL cue words. The results supported this prediction too. Just as an example, the EM cue desesperanza (hopelessness) produced the EM word tristeza (sadness) as associate, with a prototypicality value of 4.91, while the EL cue juego (game) produced the EM word diversión (fun) as associate, with a prototypicality value of 3.05. Therefore, even though EL cue words elicited some EM words as associates, those were not considered as representative of an emotion concept as the EM-associated words elicited by EM cues.
These results evidence the contribution of affective content to the associative structure of words. They are in line with the studies of Buades-Sitjar et al. (Reference Buades-Sitjar, Planchuelo and Duñabeitia2021) and Van Rensbergen et al. (Reference Van Rensbergen, Storms and De Deyne2015), who using a different approach showed the high relation between the affective properties of the cue words and those of their associates. Of note, earlier studies had concluded that participants usually generate word associates through linguistic processes, such as completion (e.g., the cue holy elicits water as associated word) and sound similarity (e.g., the cue lumpy elicits bumpy as associated word, Santos et al., Reference Santos, Chaigneau, Simmons and Barsalou2011). Our findings, together with those of Buades-Sitjar et al. (Reference Buades-Sitjar, Planchuelo and Duñabeitia2021) and Van Rensbergen et al. (Reference Van Rensbergen, Storms and De Deyne2015), suggest that other processes come into play. Therefore, they are relevant for our understanding of semantic memory. Network-based models propose that concepts are connected to one another based on their semantic relatedness, in most cases operationalized in terms of association (see Kumar, Reference Kumar2021, for an overview). We have shown that affective content plays a decisive role in the establishment of these connections, a fact that should be incorporated by those models, which have not traditionally considered the affective properties of words. In this way, these results would give support to models of semantic memory, which consider that sensory-motor and affective information is part of semantic representations (i.e., grounded theories, see Meteyard et al., Reference Meteyard, Rodríguez-Cuadrado, Bahrami and Vigliocco2012, for an overview). In addition to that, we have demonstrated that not only affective content but also the type of affective word (i.e., EM vs. EL) matters. Differences in processing between these two types of words have been reported in the literature (e.g., Altarriba & Basnight-Brown, Reference Altarriba and Basnight-Brown2011; Kazanas & Altarriba, Reference Kazanas and Altarriba2015; Wang et al., Reference Wang, Shangguan and Lu2019; Zhang et al., Reference Zhang, Teo and Wu2019; see also Wu & Zhang, Reference Wu and Zhang2020, for a review). However, the issue of their representation and their organization in the lexicon has not been addressed before. This study is the first step in this direction.
Our results also have methodological consequences. A logical prediction from network-based models is that if two words are connected, the activation of the node (the representation) corresponding to the first word would spread to the node corresponding to the other word. This is the basis of the semantic priming paradigm, which has been used extensively to investigate semantic organization and processing. In this paradigm, the processing of a target word (e.g., in a lexical decision task) is facilitated by the previous presentation of a semantically related word (for reviews, see Lucas, Reference Lucas2000; Hutchison, Reference Hutchison2003). Researchers commonly select the prime-target pairs based on their semantic relatedness, but they do not consider affective content. A different line of research focuses on affective priming, that is, the facilitation in the processing of a target word by the previous presentation of an affectively congruent word (see Klauer & Musch, 2003, for an overview). In this case, researchers select the prime-target pairs based on their affective (in)congruency (e.g., positive–positive pairs vs. positive–negative pairs), but the degree of associative relatedness between the two words is not always considered (see, however, Hu & Liu, Reference Hu and Liu2019, for a study that has tried to dissociate both types of relations). Our results suggest that this may have contributed to the inconsistencies observed in the field, just as ignoring the EM–EL distinction. Considering that EM words tend to produce EM words as associates and that EL words tend to produce EL words as associates, the presence of null findings (i.e., lack of affective priming) may be partially explained by the presence of heterogeneous pairs (i.e., pairs containing an EM prime word and an EL target word and the other way around) in the experimental set. In fact, a recent study has provided evidence in this direction, showing that EL target words are facilitated by EL prime words but not by EM prime words (Wu et al., Reference Wu, Zhang and Yuan2021). Further research comparing homogeneous pairs (EM–EM and EL–EL) and heterogeneous pairs (EM–EL and EL–EM) is needed to reach firm conclusions.
Another aim of our study was to investigate whether EL words acquire their affective properties through their relation to emotional states or events. If that is the case, EM words, which denote emotional states, should be produced as associates to a greater extent to EL cue words than to NT cue words, which are not affectively loaded. These were exactly the results found in our study. Just as an example, the EM word miedo (fear) was the first associate of both the EM cue word fobia (phobia) and the EL cue word secuestro (abduction). One consequence of the affective properties of EL words stemming from their connection to emotional events is that the specific emotion to which each EL word is connected may show individual and cultural variations. For instance, the word “party” may be associated with happiness for an extroverted person but not for an introverted person. In contrast, the emotional content of EM words is expected to be more stable, because it is part of the core meaning of the word, not acquired through association. This may partly explain the larger facilitative effects in processing (i.e., in comparison to neutral words) for EM words with respect to EL words (see Wu & Zhang, Reference Wu and Zhang2020, for a review).
Another point that deserves to be mentioned in relation to the above is the asymmetry in the pattern of association observed: EL cue words produced more EM associates than NT cue words, but EM cue words did not produce more EL associates than NT cue words (in fact, the pattern is the opposite). This suggests that the connections between the two types of affective words are not entirely bi-directional. That is, the presentation of an EL word (e.g., secuestro, abduction) easily leads to the activation of the associated emotion (e.g., miedo, fear), while the presentation of an EM word (e.g., miedo, fear) does not lead so directly to the activation of a situation/event provoking that emotion (e.g., secuestro, abduction) rather to a word denoting another emotion (temor, dread). This asymmetry could also have contributed to the mixed findings in affective priming research because EM primes may not facilitate always EL targets even if they are affectively congruent.
Furthermore, as indicated above, NT cues elicited a higher number of EL associates than EM cues. A possible reason is that EM words constitute a more interconnected category than EL and neutral words. Although this has not been tested empirically, some data point in this direction: EM cue words seem to be the most consistent type of cues in eliciting associated words of the same category. Although this pattern was observed in the three types of cue words (i.e., there were more associated EL words in response to EL cue words than to the other cues, and the same for NT-associated words), it was more consistent for EM cues. Indeed, there were more EL-associated words in response to NT cue words than to EM cue words, and there were more NT-associated words in response to EL cue words than to EM cue words.
Another possible reason is related to the concreteness dimension. Indeed, some examples of EL words produced as associates to NT cue words include the words dinero (money, produced as associate to empresario [businessman]) and hambre (hunger, produced as associate to saciedad [satiety]). The word dinero (money) has a positive valence value of 6.75, while the word hambre (hunger) has a negative valence value of 2.30. Although dinero and hambre have an opposite valence (i.e., a highly pleasant word and a highly unpleasant word), they are both considered highly concrete words with a value of 5.36 and 5.25, respectively, on the 1–7 concreteness scale. Neutral words, by definition, have a medium score on the valence scale but tend to have high concreteness values too. On the contrary, EM words show low rating values on the concreteness scale (they tend to be more abstract). Considering that concreteness has been identified in previous studies as a relevant variable in the organization of the lexicon (Van Rensbergen et al., Reference Van Rensbergen, Storms and De Deyne2015; Buades-Sitjar et al., Reference Buades-Sitjar, Planchuelo and Duñabeitia2021), it may be that EL associates are produced in response to NT cues to a greater extent than in response to EM cues due to the higher correspondence in concreteness between EL words and NT words than between EL words and EM words.
In relation to the above, a final objective of our work was to examine if a series of variables (valence, arousal, concreteness, age of acquisition, and frequency) display assortativity. That is, if the value of the first associated word in each variable is better predicted by the value of the cue word in the same variable than in other variables. Considering previous findings, we expected a high cue-response correspondence between the values of valence, arousal, and concreteness but not of frequency and age of acquisition. In the analyses that included all the cue words, we found assortativity for valence, arousal, concreteness, and age of acquisition. In contrast, word frequency did not display assortativity. These results are similar to those reported by Buades-Sitjar et al. (Reference Buades-Sitjar, Planchuelo and Duñabeitia2021), who obtained a high correspondence between the cue words and the associated words in terms of valence, arousal, and concreteness. Our findings are also in line with those of Van Rensbergen et al. (Reference Van Rensbergen, Storms and De Deyne2015), who found assortativity for the same variables. These findings suggest that some variables have a decisive role in the organization of words in the lexicon. In our study, valence and arousal showed the greatest predictive capacity. Indeed, as can be seen in Table 3, the valence model explained 41% of the total variance. Within this model, 83.41% of the variance was explained by the valence of the cue. Similarly, the arousal model explained 33.94% of the total variance. Within this model, 78.88% of the variance was explained by the arousal of the cue. Apart from that, we found that the cue values in concreteness and age of acquisition are good predictors of the values of the first associate in those variables. Specifically, within the concreteness model, 79.66% of the variance was explained by the concreteness of the cue. Similarly, within the age of acquisition model, 68.80% of the variance was explained by the age of acquisition of the cue. Although these are high values, it should be noted that, overall, both models explained only a small percentage of the total variance (15.04% in the case of concreteness and 13.92% in the case of age of acquisition) compared to the total variance explained by valence and arousal models (41% and 33.94%, respectively). Therefore, although AoA displayed assortativity here, in contrast to the study of Van Rensbergen et al. (Reference Van Rensbergen, Storms and De Deyne2015), the explanatory power of that variable, considering the total variance explained by the model, is very small.
The last aim of our study was to examine if assortativity for affective variables is higher for EM and EL words than for neutral words. Although we did not have a specific prediction due to the exploratory nature of these analyses, we might expect affective variables (valence and arousal) to be more relevant than lexico-semantic variables in the cue-response correspondence of affective words (EM and EL). The opposite pattern might be expected for neutral words. The analyses of the EM cues showed that valence and arousal exhibited a very strong assortativity, while no effects were observed for the other (lexico-semantic) variables (see Table 4). In contrast, both affective (valence and arousal) and lexico-semantic variables (concreteness) exhibited assortativity in the analyses focused on EL cue words (see Table 5). Similarly, the analysis of the NT cues revealed that AoA and valence displayed assortativity and that the concreteness and AoA of the cue were the best predictors of the concreteness of the first associated word (see Table 6). Taking into account these results, three aspects need to be noted. Firstly, affective content seems to be the most relevant variable in the organization of EM words (i.e., only valence and arousal displayed assortativity in the analyses focused on EM cue words). Secondly, apart from valence and arousal, concreteness seems to play an important role in the cue-response correspondence of EL words. Lastly, valence is the only variable that shows assortativity in the three types of cue words. Thus, affective content plays a very relevant role in the organization of the lexicon for both affective and neutral words and appears to be more important than lexico-semantic variables for both EM and EL words. A methodological implication of these results has to do with the procedure used to characterize words in terms of affective and semantic variables. The traditional approach has been asking large groups of participants to provide subjective ratings for large sets of words (e.g., Warriner et al., Reference Warriner, Kuperman and Brysbaert2013; Guasch et al., Reference Guasch, Ferré and Fraga2016), which is a very time-consuming method. In a pioneering study, Van Rensbergen et al. (Reference Van Rensbergen, De Deyne and Storms2016) extrapolated the emotional values of a large set of words from the values of their associated words, finding a very high correlation between the ratings estimated by this method and those obtained from human participants. Our findings, regarding assortativity, support this approach and go one step further, suggesting that it may be a particularly suitable strategy when EM and EL words are involved. The reason is that in these cases the correspondence between the affective properties of the cue words and the associates is higher.
To conclude, this study has two key findings. The first one is that affective content is very relevant in the organization of the lexicon. The second one is that not only affective content matters, but also the type of affective word (EM–EL). These findings have theoretical implications, suggesting that models of semantic memory should incorporate affective content. They also have methodological implications, suggesting that affective variables need to be considered when designing semantic priming experiments and that the associative relatedness, as well the EM–EL distinction, need to be considered when designing affective priming experiments. Another methodological implication is that the values of words regarding affective variables (i.e., valence and arousal) may be predicted from the values of their associates, especially when they are EM and EL words. These results are also of interest for applied studies focused on the processing and organization of affective words in specific populations, like old adults, or in pathologies like dementia, schizophrenia, or autism, among others. Studies in the field have not traditionally distinguished between EM and EL words. Further research should be conducted to establish whether the deficit in emotional word processing observed in some of these populations (e.g., Rossell et al., Reference Rossell, Shapleske and David2000; Wong et al., Reference Wong, Yap, Obana, Asplund and Teh2022) is general or restricted to one type of affective word (i.e., EM or EL). More work is needed as well to examine whether EM words are a special class of words and whether they are more interconnected in the lexicon than other words. This study is a first step in this line.
Acknowledgments
This study was supported by the Ministerio de Ciencia e Innovación (PID2019-107206GB-I00 /AEI/10.13039/501100011033) of Spain and the Universitat Rovira i Virgili (2019PFR-URV-B2-32, 2022PFR-URV-47).
Replication package
The file containing the cue words, the associated words, and the characteristics of both types of words are openly available in an Open Science Framework (OSF) repository at https://osf.io/c8azn/.
