This paper reports the results of a pilot experimental study aimed at evaluating a set of natural semantic metalanguage (NSM) explications for English interjections. NSM is a well-documented technique for semantic and cultural analysis (Goddard, Reference Goddard2011; Goddard & Wierzbicka, Reference Goddard and Wierzbicka2014; Wierzbicka, Reference Wierzbicka1996). It is based on 65 semantic universals (Table 1) and their universal syntactic properties, which have been identified on the basis of cross-linguistic empirical studies (Gladkova, Reference Gladkova2010; Goddard, Reference Goddard2008; Goddard & Wierzbicka, Reference Goddard and Wierzbicka2002; Peeters, Reference Peeters2006; Vanhatalo, Tissari, & Idström, Reference Vanhatalo, Tissari and Idström2014). These semantic universals, or “semantic primes,” are meanings that are posited to be semantically simple; that is, they cannot be defined further (without circularity). Along with primes, NSM also relies on a limited number of semantic molecules, that is, concepts that are necessary for semantic analysis and whose meanings can be decomposed in terms of primes or primes and other semantic molecules (Goddard, Reference Goddard2010a).
Semantic primes: English exponents
Note: Primes exist as the meanings of lexical units (not at the level of lexemes). Exponents of primes may be words, bound morphemes, or phrasemes. They can be formally (i.e., morphologically) complex. They can have combinatorial variants or allolexes (indicated with ~). Each prime has well-specified syntactic (combinatorial) properties. Based on Goddard and Wierzbicka (Reference Goddard and Wierzbicka2014).
Primes enter into combinations according to a well-specified grammar, allowing, for example, combinations such as “someone wants to do something,” “it happened in one moment,” “if I do this, people can think something bad about me,” among many others. Thus, primes (or primes and semantic molecules) essentially constitute a minilanguage that lies at the core of every language. This minilanguage has sufficient expressive power as a metalanguage to adequately represent the meanings of more complex concepts and cultural attitudes (e.g., Gladkova, Reference Gladkova2013; Gladkova & Romero Trillo, Reference Gladkova and Romero Trillo2014; Goddard & Wierzbicka, Reference Goddard and Wierzbicka2014). The meanings of concepts are represented in the form of reductive paraphrases, known as “semantic explications.”
Goddard (Reference Goddard, Heine and Narrog2010b, p. 464) formulates two major requirements to developing a good NSM explication:
The validity of NSM explications can be tested on the basis of two main conditions. The first is substitutability in a broad sense: explications have to make intuitive sense to native speakers when substituted into their context of use, and to generate the appropriate entailments and implications. The second condition is well-formedness: they have to be framed entirely in semantic primes or molecules, and to conform to the syntax of the natural semantic metalanguage. In addition, explications have to conform to a coherence condition, i.e., they have to make sense as a whole, with appropriate chains of anaphora, co-reference, causal links, and so forth.
Because semantic primes should be intuitively intelligible to native speakers of a language, the results of semantic analysis couched in semantic universals should be open to verification by ordinary language speakers. Goddard (Reference Goddard2011, p. 65) writes, “The ultimate test of a good explication is that native speakers will agree that the explication expresses the same meaning as the original expression in context. This is another reason to formulate explications in straightforward, non-technical terms: they must be intelligible to ordinary language users.”Footnote 1
In the NSM literature, numerous semantic explications have been proposed on the basis of detailed analysis of linguistic data (often drawing on corpus data), and informed by the researchers' linguistic intuition and introspection. The desirability of directly verifying explications with ordinary language speakers has sometimes been acknowledged in the NSM literature (e.g., Goddard, Reference Goddard, Heine and Narrog2010b, p. 461, Reference Goddard2014a). For example, Goddard and Wierzbicka (Reference Goddard and Wierzbicka2014, p. 247) write, “Because explications are intelligible to native speakers, they are amenable to experiments that directly expose speakers to the analysis itself using recognition, matching, and rating tasks. Well-designed psycholinguistic experiments could help test the validity of explications and clarify the role of lexical semantic structures in thinking.” They further conclude: “The NSM paradigm can stimulate cross-disciplinary research and open the way for the development of new experimental and quasi-experimental methodologies” (Goddard & Wierzbicka, Reference Goddard and Wierzbicka2014, p. 249). Yet an experimental approach with NSM has not previously been implemented in a systematic fashion (apart from informal consultations with native speakers).
The present study fills this gap in the literature by proposing a novel online survey technique for testing NSM explications with language speakers. As input, it uses recently proposed explications of selected English interjections as published in Goddard (Reference Goddard2014a): wow, gosh, gee, yikes (“surprise” group) and yuck, ugh (“disgust” group). For example, Goddard (Reference Goddard2014a, p. 58) explicates the meaning expressed by yuck as follows:
The first two components of the explication link an immediate thought with an immediate feeling (a “very bad” feeling in this case). A pair of “aversive” components completes the explication.
As a type of lexical item, interjections suit the purpose of a pilot study well on account of their communicative nature and accessibility to lay people, and because their explications are relatively short, which makes them a good starting point for developing the survey technique. Interjections have been on the radar of NSM researchers for over 20 years (Ameka, Reference Ameka1992a, Reference Ameka1992b; Goddard, Reference Goddard2011, Reference Goddard2014a, Reference Goddard, Baider and Cislaru2014b; Wierzbicka, Reference Wierzbicka1992, Reference Wierzbicka2003/1991; Wilkins, Reference Wierzbicka1992).Footnote 3
Existing studies of interjections have mainly relied on analysis of corpus material, other naturally occurring data, and on linguists' own observations (Ameka, Reference Ameka1992a; Norrick, Reference Norrick2009; Wierzbicka, Reference Wierzbicka1992; Wilkins, 1992). Experimental studies of interjections have been very limited and have mainly involved an association technique. Among these, Fischer (Reference Fischer2000) elicited native speakers' perception of and attitudes toward the meaning of selected interjections and discourse markers using an open-ended questionnaire. Fischer (Reference Fischer2000, p. 43) concluded, “The method of asking speakers about their knowledge on discourse particles yields interesting results regarding their opinions on the evaluation of these items. However, [at least] linguistically untrained speakers do not provide conclusive data regarding discourse particle meanings and uses on the basis of questions like the one used in the questionnaire study.”
Fox Tree (Reference Fox Tree2007) conducted an experimental study of discourse markers (a category pragmatically close to interjections). It consisted of open-ended and multiple-choice questionnaires about laypeople's uses of um, uh, you know, and like, folk notions of meanings, self-assessment of use, history of discussing use, and attitudes toward the words. The “folk” meanings identified by the respondents were shown to be relatively close to their scientific definitions and explanations, thus suggesting that experimental studies can be a legitimate source for studying the meanings of discourse markers. Schröder (Reference Schröder2003) conducted an experimental study of “affect bursts” (a category overlapping with interjections) in German using a listening test and demonstrated that they possess a relatively high level of recognition (81%) by lay speakers. In a series of studies on synonymy (a total of 489 native subjects), Vanhatalo (Reference Vanhatalo2005) demonstrated that questionnaire-based studies can bring to light lexical information that could not be gained in other ways. This lexical information includes meaning differences between words that are conventionally categorized as literal synonyms. Thus, experimental studies suggest a promising, though limited, possibility of using lay people's opinions in studying the meaning of interjections (see Bott, Featherston, Radó, & Stolterfoht, Reference Bott, Featherston, Radó, Stolterfoht, Maienborn, von Heusinger and Porter2011).
The aims of the present study are threefold: to develop an online survey technique for evaluating NSM explications based on responses by lay language speakers, to apply this technique to collect experimental data evaluating a set of proposed NSM explications of interjections, and to assess the effect of background variables (age, gender, [non]native speaker status, and background knowledge about NSM) on the respondents' evaluations. The rest of the paper describes the experimental study (participants and the survey), presents the results, and discusses the results and possible ways of improving the technique.
METHOD
Participants
Participants were recruited in 2012 from current and former undergraduate and postgraduate linguistics students of the University of New England and Griffith University in Australia, and they were encouraged to invite further participants. One hundred one participants completed the survey. They were fairly diverse in age, from 18 to 77 years old, with a mean age of 41. They were 80% female, reflecting the proportion of women studying linguistics in the chosen universities. At the time of the survey, 59% were students and 61% had studied or were studying linguistics. Most of the participants (85%) were native speakers of English, and 50.5% spoke more than one language. All participants with a first language other than English (15%) rated their level of English as fluent or native. Almost all participants (95%) had lived in an English-speaking country for at least 10 years. Regarding prior knowledge about the NSM approach, 16% reported that they knew NSM fairly well, 29% had some basic knowledge, 7% had read something about it but could not use it themselves, 11% had heard or read about it but did not know much about it, and 38% did not know anything about it. As a reward for participation, potential participants were offered a $20 gift voucher from a local supermarket chain upon completing the survey. Nine participants chose not to be rewarded with the vouchers, while the rest received vouchers by mail. The study was anonymous, and the addresses for mailing the vouchers were kept separately from the survey data.
Survey design and implementation
At the beginning of the survey, respondents had to give a formal consent to participate in the survey as required by the ethics committees. The survey had three parts:
-
1. Part A: background information about age, gender, languages spoken, being a student, and familiarity with the NSM (11 questions, reported in Participants section);
-
2. Part B: rating of individual putative semantic components for the interjections wow, gee, gosh, yikes, yuck, and ugh; and
-
3. Part C: rating of the full previously published explications for these six interjections (Goddard, Reference Goddard2014a).
At the end of the survey the participants were asked to comment on the survey and to leave the address for sending the voucher. These questions were not compulsory to answer.
In Parts B and C, a Likert scale from 1 to 7 was used.Footnote 4 The range from 1 to 7 was chosen as statistically reliable (Cicchetti, Showalter, & Tyrer, Reference Cicchetti, Showalter and Tyrer1985; Rasmussen, Reference Rasmussen1989; Wakita, Ueshima, & Noguchi, Reference Wakita, Ueshima and Noguchi2012). Only scores 1 and 7 were associated with verbal descriptions, “not likely to be part of the message” and “very likely to be part of the message,” respectively, in Part B; and “doesn't capture the meaning well” and “captures the meaning very well,” respectively, in Part C. Figure 1 presents an example of a typical screen in Part B of the survey.
An example of a screen in Part B of the survey.
In Part B the respondents were asked to evaluate individual putative components of the six interjections. Among the interjections, wow, gee, gosh, and yikes are from a surprise group, and yuck and ugh are from a disgust group. All the interjections in a given group were ranked against the same list of putative components. The interjections in the surprise group were ranked against 35 components (see Table 2), and those in the disgust group were ranked against 21 components (see Table 3). Each respondent answered 182 questions in this part of the survey.
Component evaluations for the disgust group
Note: GE, Goddard's (Reference Goddard2014a) explication components.
a Components that passed the t test for “4.” Component 17 is the filler.
Component evaluations for the surprise group
Note: GE, Goddard's (Reference Goddard2014a) explication components.
a Components that passed the t test for “4.” Component 34 is the filler.
To maintain consistency in the analysis, components were given an identifying number. The numbering of the components in the two groups of interjections did not coincide, and these numbers were not known to the participants.
The putative components comprised all components from the explications of the group (from Goddard, Reference Goddard2014a; these are called “original components”), a number of possible variant components, and a “filler” component. The components in the disgust group included 7 original components and 13 variant components (semantically and grammatically related to the original). For example, the original component “I think like this: this is something very bad” had the variant components “I think like this: this is something bad,” “I think like this: this is something very good,” and “I think like this: this is something good.” Obviously, on account of time constraints on the survey, a line had to be drawn on how many variant components could be presented. Therefore, the researchers had to choose components that seemed most relevant and important in terms of the discussion of meaning of the words. Some components were also taken from Ameka (Reference Ameka1992a). For each word, a filler component was included from an explication of a word from a different semantic domain (e.g., “I want this something not to be one thing anymore” from the explication of the verb to cut in Goddard & Wierzbicka, Reference Goddard and Wierzbicka2009, p. 64). This was done to test how filler components would compare to putative components in ratings. We also thought that given the somewhat monotonous character of the questions, such components could sharpen the respondents' attention.
The survey was technically implemented by the Griffith University Research Survey Centre using a web survey software Sandpit Environment V1.9x, and it was activated from October 10 to December 31, 2012. One question was presented per page, and each question had to be answered in order to proceed to the next one (with the exception of the final comment question and the mailing address question, which were optional). The participants were not able to return to previously answered questions, but they were allowed to logout and return to continue the questionnaire later (unlimited entries with the same individual access code). These conditions were chosen in order to reduce time for the participants to finish the survey and, therefore, to make it more attractive. The questions for each interjection in Part B were randomized separately for each participant by the survey software. Each participant received an individual access code by email from the survey manager.
Response profiles
To get an initial sense of the response quality, we inspected the data manually for possible fake responses by searching if any participant had given the same responses to all questions, or implausible answers to the filler questions. None of the participants needed to be excluded for these reasons. It was noted in the analysis phase that a technical bug in the software, related to the possibility of completing the questionnaire in multiple sessions, had resulted in few missing values (n = 98, .005% of all values) in the answers. For this reason, the number of responses in each component may vary slightly (range = 99–101 responses). Because the individual responses were not dependent on each other, we chose to use all answers regardless of the missing values.
In the next section, we discuss the results for each interjection, present the results of the influence of background variables using a multivariate analysis, and then report the content of free comments to the survey.
RESULTS
Evaluations of components and explications
In this section we report the results of evaluation of explications proposed in Goddard (Reference Goddard2014a) and of the putative components. This allows us to assess the validity of the proposed explications and to make suggestions about possible changes to them on the basis of experimental data. We present the results for each interjection in turn, beginning with the overall evaluation of the explication (Part C of the survey), followed by the results about individual putative components (Part B of the survey).
In interpreting the evaluations of the individual components, we regard components with average rankings above 4 (i.e., the midpoint on the 1–7 Likert scale) as possessing positive likelihood of being part of the meaning of the words. To ensure that only statistically relevant rankings above 4 were considered, we performed a t test on raw ratings of each component and selected those that significantly deviate above 4 in the ratings. (Note that the t test assesses whether the mean of raw ratings for a particular component is statistically different from 4.) Within the 4–7 range, we regard components with average scores between 4 and 5 as having a good possibility of being part of the meaning, those in the 5–6 range as having a very good possibility, and those in the 6–7 range as having a high possibility of being part of the meaning.
Disgust group: Yuck
The explication of yuck (Goddard, Reference Goddard2014a, p. 58) used in the survey is the following:
The rating for the full explication of yuck (Part C of the survey) suggests a strong support for the proposed explication with a mean score of 6.2 (the highest among all explications) and a standard error of the mean of 0.11 (see Figure 2). The interquartile range (IQR) of responses, that is, the 50% range, lies between the scores 6 and 7 (see Figure 3), indicating a very high level of approval of the explication. Ninety-five percent of responses are concentrated in the range of 3 and 7 (see Figure 3), again indicating very good support for the explication.
Distribution histograms of ratings of Goddard's (Reference Goddard2014a) explications of interjections (mean ± standard error of the mean).
Interquartile range (IQR) and 95% range of ratings of Goddard's (Reference Goddard2014a) explications of interjections. The boxes depict the IQR, that is, the range of 50% of responses. In addition, the graph shows the 95% range of responses (whiskers).
Turning now to ratings for putative individual components in Part B of the survey, all the individual components of the proposed explication fall into the range above 4 and pass the t test (see Table 2). Overall, yuck receives rather clear ratings with positively evaluated components having averages in the ranges of 5–6 and 6–7.
The initial negative evaluative component of the explication (Component 2) is among those components having very good likelihood of being part of the meaning (see Table 2). Its similar in meaning components are 3 and 21. Component 3 receives a slightly higher ranking than Component 2, but the difference is negligible. The higher ranking of Components 2 and 3 above 21 suggests a preference for this evaluative component being atemporal, rather than being associated with the moment of speech (now), thus confirming this aspect of Goddard's (Reference Goddard2014a) explication.
A somewhat similar picture emerges with the second, emotive component of the explication (Component 6), which is also found in the range of components possessing very good likelihood of being part of the meaning. However, a similar Component 7 receives a slightly higher ranking. The preference for the feeling being described as “bad” rather than “very bad” could possibly be because, in folk understanding, feeling “very bad” may be reserved for greatest misfortune such as disease or death. Compared with these eventualities, the repulsion and aversion associated with yuck could be regarded as less negative. Following the theme of emotional attitude associated with yuck, Component 16 (which appears in the explication of ugh) is also ranked as having very good likelihood of being part of the meaning.
The “revulsion” component (Component 11) receives the most favorable ranking of all, and possesses high possibility of being part of the meaning. Similar Components 12 and 13 also fall into the range of components having high likelihood of being part of the meaning, but it is unlikely that all these variants need to be included in the explication. It is possible to suggest that “mouth” is the best candidate here given its association with ingestion (plus, Component 11 does receive a higher ranking than Components 12 and 13). That is, imagining something unpleasant being near one's mouth might induce a greater degree of repulsion due to the possibility of the object going inside the body. Component 15 also falls into the range of components with high likelihood of being part of the meaning.
The last line of the explication (Component 14) is also confirmed as a highly likely candidate. Although it is close in meaning to Components 11–13, the presence of the prime touch makes its meaning distinct and supports the value of this prime in the explication.
Other components that emerge with a good chance of being part of meaning are 18 and 20. There is, however, a meaningful connection between Component 20 and Components 11–15.
To summarize, the responses present overall support for the proposed explication of yuck. The results suggest that the evaluative components should be expressed via “bad” rather than “very bad.” In addition, components “I don't want it to be like this” and “I say this because I want you to know how I feel” need to be considered.
Ugh
The survey used the following explication of ugh (Goddard, Reference Goddard2014a, p. 58):
The full explication of ugh receives a reasonably high ranking with a mean of 5.3 and standard error of the mean of 0.17 (see Figure 2). The IQR of ratings falls between the scores 5 and 7. Ninety-five percent of answers fall between 1 and 7 (see Figure 3). Thus, support for the proposed explication is rather strong, though not as strong as it is for yuck. Ratings for all the individual components of the proposed explication fall into the range of 5–6, and overall, the positively evaluated components are in the 5–6 range, suggesting that there is good likelihood of their being part of the meaning (see Table 2).
The component of negative evaluation (Component 21) is confirmed, but similar Components 2 and 3 receive higher rankings, thus calling into question the association with the moment of speech (now). In addition, the higher ranking of Component 3 suggests a possibility of a milder degree of negative evaluation (possibly for similar reasons as discussed for yuck). The component of negative feeling (Component 6) is confirmed, but a similar Component 7 receives higher ranking, again suggesting a possibility of a milder degree of negative feeling.
The final component of the proposed explication (Component 16), linking the feeling with the physical reaction of retching, is also confirmed. Component 15 is confirmed and is the top-ranked component for this word. Components 11–14, expressing the desire for the things causing revulsion not to be near one's smell and taste sense organs, are also ranked quite highly. Note that Goddard (Reference Goddard2014a) proposes Component 11 as part of meaning for yuck, but not for ugh. As is the case with yuck, the other possible components that emerge from the data on ugh are 18 and 20.
To summarize, the results provide support for the proposed explication of ugh, but suggest the following directions for revision: exclusion of the association with now, milder evaluation (“bad” rather than “very bad”), including emotional attitude, the desire for the thing causing revulsion not to be near one's smell and taste sense organs, as well as the intention to let the interlocutor know one's state and the desire for the thing not to be like this.
Surprise group: Wow
The explication of wow (Goddard, Reference Goddard2014a, p. 57) used in the survey is the following:
The full explication of wow receives overall high rankings with the mean score of 5.7 and standard error of the mean of 0.13 (see Figure 2). It is the second-best explication score among all the words after yuck and the highest in the surprise group. The IQR ratings for wow are also quite high and fall in the range between 5 and 7. Ninety-five percent of the responses are in the 2 to 7 range (see Figure 3). Thus, there is reasonably good support for the proposed explication of wow. All the individual components of Goddard's (Reference Goddard2014a) explication fall into the 5–6 range, suggesting they have a very good possibility of being part of the meaning (see Table 3).
The respondents' answers strongly suggest an association of the meaning of wow with new knowledge. Component 13 is ranked higher than Component 14, the actual component of the proposed explication, although not significantly. There is preference for know rather than think in such components (Components 13 and 14 are ranked higher than Component 15), thus supporting the presence of know in the component in the explication.
There is a strong indication that wow is associated with experiencing something good (or even very good) happening, as Components 4, 8, and 9 receive reasonably high rankings. This data suggest that happen should be included in the explication. Component 1, which does not specify a type of knowledge, receives lower ranking than Components 4, 8, and 9, thus supporting the idea that there is some new knowledge associated with wow, but indicating that this knowledge involves something good happening.
The component evaluation suggests that a feeling, and even a strong positive feeling, is associated with wow. Goddard's initial Component 26 has a high possibility of being part of the meaning. A similar-sounding Component 29 receives a slightly higher score. Another similar Component 25 is ranked slightly lower. The component describing the impression as if seeing something very big (Component 33) is also confirmed as having a good possibility of being part of the meaning.
A positive evaluation component (Component 17) is also confirmed and falls into the range of components having a very good possibility of being part of the meaning. However, as discussed earlier, greater preference is given to variant components with the prime happen.
The following components are from the 4 to 5 range and have a weaker, although still good, possibility of being part of the meaning. They include components similar to the ones from the 5 to 6 range suggesting knowledge of something or something good happening (Components 2, 3, and 7) and positive thinking (Component 16). Other possible suggestions from this range include Components 21 and 32.
To summarize, the experimental data provide overall support for the proposed explication of wow, confirming elements of meaning such as knowing something, emphasizing that it is new knowledge, an associated positive feeling and a feeling similar to that experienced when seeing something very big. The analysis suggests a strong possibility that the semantic prime happen is part of the meaning as well. The data also suggest the possibility of components related to a desire to communicate one's state to another person and a desire to know more about what is happening.
Gee
The explication of gee (Goddard, Reference Goddard2014a, p. 57) used in the survey is the following:
The full explication of gee receives good rankings with the mean score of 5.1 and standard error of the mean of 0.15 (see Figure 2). The IQR ratings for gee are also good and fall in the range between 5 and 6. Ninety-five percent of the responses are in the range of 1–7 (see Figure 3). Thus, there is reasonably good support for the proposed explication of gee.
The majority of individual components from Goddard's (Reference Goddard2014a) explication are confirmed, but overall their rankings are distributed within a lower range than with other words (mainly 4–5 range with the highest score of 5.15; see Table 3). This suggests that either participants do not have strong agreement about relevant components or the proposed components still do not capture the expressed meaning very well.
The final component of the explication, that is, emotional reaction associated with gee (Component 29), falls into the range of components having a very good chance of being part of the meaning. No other components expressing either positive or negative emotional state pass the t test and fall into the range of possible components (because they are all ranked below 4). This strongly supports the proposed neutral wording of the emotional component for gee. However, another emotional component (Component 33) also falls into the range of components having a good possibility of being part of the meaning.
Component 1 is also among the ones having a very good likelihood of being part of the meaning. Even though gee receives a relatively small number of components passing the t test, there are several components referring to some sort of knowledge among the components that do pass it. Among them are Components 2, 7, 13, 14, and 15. The choice of these components strongly indicates that reference to some sort of knowledge is part of the meaning of gee. Additional selected components suggest the possibility that such knowledge is knowledge of something new and knowledge of something happening.
Component 24 from Goddard's (Reference Goddard2014a) explication (“I want to think about it for a short time”) receives ranking above 4, but it does not pass the t test for 4, which indicates that its deviation above 4 is not statistically significant. In other words, the participants neither agree nor disagree whether this component is part of the meaning of gee. An alternate, similar component that was ranked higher was Component 32. Another component that emerges from the data as having a good possibility of being part of the meaning of gee is Component 20.
To summarize, the proposed explication of gee receives positive ranking, and the elements of meaning of a neutral emotional attitude and some new knowledge are confirmed by experimental data. The proposed element of meaning associated with wanting to think about the stimulus for a short time does not receive strong support. Other areas of meaning that emerge from the data are: elements of new knowledge, knowledge about something happening, a feeling associated with seeing something very big, the desire to communicate one's state to another person, and the “puzzlement” of not knowing what to think about the stimulus.
Gosh
The survey used the following explication of gosh (Goddard, Reference Goddard2014a, p. 57):
The full explication of gosh receives good rankings with the mean score of 5.3 and standard error of the mean of 0.13 (see Figure 2). The IQR ratings for gosh fall in the range between 5 and 6. Ninety-five percent of the responses are in the range of 2–7 (see Figure 3). Thus, there is reasonably good support for the proposed explication. All the individual components of Goddard's (Reference Goddard2014a) explication are confirmed and fall into the range of components having good or very good likelihood of being part of the meaning (see Table 3).
The final, emotional Component 29 possesses a very good likelihood of being part of the meaning. All other putative components of either positive or negative emotional attitude do not pass the t test, therefore possessing little likelihood of being part of the meaning. Another emotional component that falls in the range of components as having good likelihood of being part of the meaning is Component 33.
The component of some knowledge (Component 1) also falls into the range of components with very good likelihood of being part of the meaning. However, in the range of positively evaluated components passing the t test, there are several components referring to some kind of knowledge. These components are 2, 10, 11, 13, and 14. These components confirm the importance of a component associated with knowledge being part of the meaning of gosh. At the same time, they suggest the possibility of the following dimensions of meaning: new knowledge and knowledge of something happening, in particular something bad or very bad happening.
The component associated with “puzzlement” (Component 20) falls in the range of components possessing good likelihood of being part of the meaning. A putative component of mental evaluation (Component 15) receives higher scores, thus reinforcing the likelihood of new knowledge and new understanding being associated with the meaning of gosh. Another possible component that emerges from the data is Component 32.
To summarize, the proposed explication of gosh receives support from the participants. Component analysis supports the value of an emotive component and suggests that a component capturing feeling as if seeing something very big is part of the meaning. The study supports the component of associated knowledge, but it brings forward associated elements of meaning such as new knowledge, knowledge of something happening, and even knowledge of something bad or very bad happening. The component of “puzzlement” is confirmed as a possible one, but the data also suggest the possibility of a new way of thinking about the matter being associated with gosh. The component expressing the desire to communicate one's state to another person is also shown to be a possible one.
Yikes
The survey used the following explication of yikes (Goddard, Reference Goddard2014a, p. 57):
The full explication of yikes receives the mean score of 4.6 and standard error of the mean of 0.18 (see Figure 2). This is the lowest score among all words and also the highest standard error of the mean. The IQR ratings fall in the range of 3–6. Ninety-five percent of the responses are in the range of 1–7 (see Figure 3). Thus, the explication of yikes is the one that meets with least approval from the participants.
The majority of the individual components passing the t test fall into the range of 4–5 (13 components), while only 3 components are in the 5–6 range (see Table 3). Three components out of four in Goddard's (Reference Goddard2014a) explication pass the t test and fall into the range of components with good likelihood of being part of the meaning. The component “I don't feel something very bad” receives ranking below 4.
Component 5 has good likelihood of being part of the meaning. There are also several similar-sounding components that are ranked as possible: Components 1, 2, 6, 7, and 10. Therefore, the data suggest that there is greater preference for the evaluation of the happening event being neutral rather than negative (i.e., Components 2 and 7 are ranked above Components 5, 6, and 10). Yet at the same time, the data do not deny the possibility of the negative evaluation of the event. The data also suggest some preference for the event being perceived as something happening in the past, rather than in the present.
Component 13 falls in the range as having good likelihood of being part of the meaning, but at the same time Component 14 is ranked higher. These components are very similar in meaning, and the choice between them should be made in favor of the one suiting the context of the neighboring components in the explication. The data also suggest relevance of a cognitive process involved as Component 15 receives a ranking of mean of 4.64 and a standard error of the mean of 1.31.
The negative emotive Component 27 possesses good likelihood of being part of the meaning. At the same time, the component of neutral feeling, Component 29, is ranked higher and the component of very bad feeling, Component 28, is ranked lower, while still passing the t test. It should be noted that another emotive component of the original explication Component 30 receives very low ranking, thus being rejected as a possible part of the meaning. This component is “paired” with the previous component “I feel something bad because of this” in the explication and should be read along with this component. However, in the survey, each component was presented separately, and it might have produced more accurate results if paired components had been presented together in the survey. It is possible that the survey results (greater preference for a “neutral” feeling, but still good results for a negative feeling [bad or very bad]) are consistent with the original wording of the explication (when the two components are taken together). Other possible components emerging from the data are a desire to communicate one's state to another person (Component 32), negative thinking about the event (Components 18 and 19), and the desire for things not to be like this (Component 35).
To conclude, the explication of yikes is the one that elicited least support of all the proposed explications, as ranked only 4.6 on the scale of 1 to 7. The proposed components overall received support from the participants, but several other dimensions of meaning emerged from the data. In particular, there is a possibility of the event being evaluated neutrally and also it being a past event. There is also a possibility that the associated feeling should be neutral. Other emerging components are negative thinking about the event, the desire for the things not to be like this, and the desire to communicate one's state to another person.
The effect of background variables on explication scores
Does background knowledge of NSM matter?
As reported in Table 4, we used multivariate analysis to test whether background factors, such as having knowledge of NSM, being a native speaker of English, being a student, studying linguistics, age, and gender, influence the ratings that respondents give to Goddard's (Reference Goddard2014a) NSM explications.Footnote 5 Theory predicts that such explications should be comprehensible to experts and nonexperts alike. Therefore, NSM knowledge is unlikely to have any impact on the explication scores. We tested this hypothesis by performing an analysis using the following simultaneous regression model:
\begin{eqnarray*}
y_{i} &=& \beta_{\rm 0} + {\rm \beta }_{\rm 1}{\rm NSM}_{i} + {\rm\beta }_{\rm 2}{\rm Native}_{i} + {\rm \beta }_{\rm 3}{\rm Age}_{i} + {\rm \beta }_{\rm 4}{\rm Gender}_{i} + {\rm \beta }_{\rm 5}{\rm Student}_{i} \\
&&+\, \beta _{\rm 6}{\rm LingStu}_{i} + {\rm \varepsilon,}\end{eqnarray*}
where yi denotes the explication scores on the six words (wow, gee, gosh, yikes, ugh, and yuck) from individual i; and NSM i , Native i , Age i , Gender i , Student i , and LingStu i are the NSM knowledge, native speaker, age, gender, student, and linguistics student variables, respectively. The variable NSM knowledge runs from 0 to 5 with a higher score indicating that the respondent has better NSM knowledge. The variable age reports the age of the respondent. Note that there are 101 individuals/respondents in our sample, that is, i = 1(1)101. If NSM knowledge were a key driver of the explication scores, we would expect the β1 coefficient to be positive and statistically significant. However, our data suggest otherwise. The coefficients on the NSM Knowledge variable in Columns 1–6 are statistically insignificant. In other words, they are insignificantly different from zero, suggesting that knowledge of NSM has no effect on the explication scores for all six interjections (wow, gee, gosh, yikes, ugh, and yuck). We also find that for all six interjections, the coefficients on the native speaker and linguistics student variables are statistically insignificant. Furthermore, the explication scores appear to be gender neutral, because the gender variable has no impact on the explication scores. All this evidence taken together appears to be consistent with the NSM theoretical prediction that the explications are comprehensible independently of specialist NSM or linguistic knowledge, and independently of gender. The estimated coefficient on the age variable is negative and significant in Column 4, indicating that older respondents systematically allocate a lower score to the yikes explication. This perhaps suggests that older individuals rarely use the interjection yikes. The estimate coefficient on the student variable is positive and significant in Column 5, indicating that student respondents systematically rate the ugh explication higher. This perhaps suggests that students use the interjection ugh more frequently.
The effect of background variables on explication scores
Note: The dependent variables are the ratings on the natural semantic metalanguage (NSM) explications developed by Goddard (Reference Goddard and Wierzbicka2014a). The variable NSM Knowledge runs from 0 to 5 with a higher score indicating that the respondent has better NSM Knowledge. The variable Native Speaker = 1 if the respondent is a native speaker of English and 0 otherwise. The variables Gender = 1 if a respondent is a male and 0 otherwise, Student = 1 if a respondent is a student and 0 otherwise, and Linguistics Student = 1 if a respondent is a linguistics student and 0 otherwise. The variable Age reports the age of the respondent. The reported numbers are Beta weights. Standard errors are robust to heteroskedasticity and within group correlations. All regressions are carried out with an intercept.
***p < .01.
Results from the free comments to the survey
Before we proceed to the discussion, we find it important to report the results of the free comments to the survey. Thirty-nine out of 101 people left free comments. The results follow.
Six respondents commented on their limited use and/or knowledge of the following interjections: ugh (two respondents), gee (three respondents), gosh, and yikes (four respondents each). One respondent noted that he/she was not sure about pronunciation of ugh.Footnote 6 Nine respondents noted that some interjections may vary in meaning depending on context, suggesting that in some cases they could not make a clear decision when evaluating semantic components.
Twelve respondents left positive comments about the survey, calling it interesting, enjoyable, intriguing, comprehensive, well designed, adequate, and leaving a positive impression. Several people noted that this survey was a new experience for them (in a positive way), giving them a chance to reflect on the meaning of words in great detail.
Nine respondents reported some difficulties in answering the survey. Five described statements or sentences (presumably, components) as a little unclear, a little confusing, confusing, awkward, not easily understood, and at times not making sense. One person named several components (mainly the fillers) as meaningless and not approachable at all.
Four respondents left free commentary about their understanding of the meaning of the words in question, sometimes in a comparative perspective. This could suggest that they felt a need to state it in their own words, rather than in the given metalanguage, or that they felt that they were not given enough of a chance to rate their own feedback (which, once entered, was locked in, with no possible return to earlier parts of the survey).
DISCUSSION
The study demonstrated an overall acceptance of Goddard's (Reference Goddard2014a) proposed explications of interjections. The proposed explications received positive rankings on the 1–7 scale (grand mean score = 5.38, range = 4.6–6.2). However, the difference in the scores for different words could be associated with the clarity or otherwise of the explications, and it could also be associated with the level of familiarity of the participants with the words in question. As free comments to the survey suggest, the participants' knowledge of the words used in the survey varied. In particular, limited knowledge and/or use was noted for interjections ugh, gee, gosh, and yikes. Similarly, explications for these words receive lower scores than explications for yuck and wow (see Figure 2). Therefore, in future studies of this kind, it might be useful to include a question assessing familiarity and use of the words by the participants and to correlate this information with the evaluations.
We regard the collected data as important input in understanding the meaning of the words in question. However, further reference to large-scale corpora (such as Collins WordBanks Online or the Corpus of Contemporary American English) are needed because any emerging components based on this study need to be tested by substitution into naturally occurring contexts in the corpora. This task lies outside the scope of this paper.
Statistical analysis reported earlier demonstrated that neither background NSM knowledge nor other background variables, such as age, gender, being a native speaker, and studying linguistics, affect the results. It will be important in future studies to confirm these correlations on larger population samples and on different types of linguistic input.
Because it is a pilot study, this survey allows us to identify directions for future development of surveys of this kind. In particular, we highlight the importance of the following issues for future studies: how to present explications for testing and how to interpret the results. In the present study the explications were tested in two forms: as a whole and as separate components. Testing explications as a whole yields valuable results and can be continued. It might also be valuable to introduce matching tasks in which, for example, a full explication is presented with one (or more) alternatives and respondents are asked to select the best match to a given word.
Presenting putative components for evaluation poses several challenges in terms of the design of the study and the interpretation of the results. In terms of number of putative components tested, we are overall satisfied with the direction undertaken in the study. The tested components should be “broad” enough to allow researchers to explore a “theme” or “direction” in an explication. At the same time, the study needs to be viable and realistic for respondents to complete in a reasonable time. In terms of the design, it may be important to explore ways of presenting “paired” components together in the survey (as should have been done for yikes). Future work could also explore contrast tasks, where, for example, respondents are asked to choose between similar components such as “I feel something bad” and “I feel something very bad.”
Regarding the filler components, which were taken from explications belonging to other semantic domains, we now believe that they are not necessary for a survey of this kind. Some respondents commented that such components were confusing; moreover, their rankings did not receive the lowest scores. Therefore, their inclusion did not contribute to understanding the usefulness of other relevant components.
One drawback of the current form of the study, which offers individual putative components for evaluation, is that it neglects the “textual” nature of NSM explications. While it is possible to statistically separate components in interpreting the results, simply putting all the top-rated components together would not necessarily produce an “ideal” explication because there would not necessarily be coherence and logic in their order. Resolving this aspect in improving the explication could only be achieved by the researchers inferring and establishing a textual connection between the components. This means that there are limits to what can be ascertained from a survey design like this one. It can give indications of what kinds of components are (or are not) likely to be part of the meaning, but the precise wording will often have to be decided with logical (coherence) considerations in mind.
Following the free comments to the survey, the following points emerge. In future studies, it might be useful to include questions asking participants to explain in “plain” English their understanding of the meaning of certain words and then to compare the results with evaluations of NSM components and explications. In addition, in future studies there could be a scope for distinguishing different uses and meanings of the words. Audio files with pronunciation of the words and examples of contextual uses could be included to achieve a more directed response from the participants. The participants feedback also suggests that it might be beneficial to make it possible to return to previously answered questions and adjust the answers.
The present study used interjections as an input. It is reasonable to suggest that different kinds of word (e.g., adjectives, nouns, and verbs) may require adapted versions of the “component-by-component” rating task used in this kind of experiment. This is so because the semantic content of words of other kinds can be much more complex than with interjections and because there may be greater interdependency between components. In addition, words from other divisions of the lexicon are often polysemous, and it may be necessary to develop new ways of dealing with this. However, the groundwork has been done with this study, and it has proposed an approach that can be used in future experimental research in semantics in general and NSM in particular.
Conclusion
This study developed and implemented a technique for testing NSM explications and applied it to a set of proposed explications of interjections (Goddard, Reference Goddard2014a). The study demonstrated overall support for the proposed explications and provided directions for their improvement.
The results of our study support Fox Tree's (Reference Fox Tree2007) observations that lay speakers' folk understanding of meaning overlaps with its scientific interpretation and that experimental studies of meaning can be an important technique for studying meaning. It is a promising tool for NSM semantics in particular, because it makes it possible to collect experimental data that directly bears on the validity of proposed explications. The study also supports the universal character of the NSM metalanguage in that neither NSM knowledge nor other background variables (age, gender, being a native speaker, or studying linguistics) proved to be important in influencing the results. More metamethodological studies are needed, in order to refine the instrument and to understand it better.
ACKNOWLEDGMENTS
The study was supported by the University of New England Research Seed Grant (2012). We thank Sambit Bhattacharyya and Sampsa Vanhatalo for their helpful comments on an earlier draft of the paper.
