Assessing Natural Language: Measuring Emotion-Words within a Sentence or Without a Sentence?

Recently, researchers in the quantitative, questionnaire tradition have been increasingly interested in the impact of emotion language and the measurement of this. Some of these researchers measure emotion-words within a meaning-giving sentence, others measure them in isolation. Based on semantic theory, we argue that emotion-words presented in isolation should mean less to a participant than emotion-words combined with other words forming a sentence as in natural language. Reflecting this, we demonstrated in our study that emotion-words are rated with great intensity when an emotion-word is placed within a sentence than when presented in isolation. As a result, we suggest that questionnaire research is better off measuring emotion-words within a meaning-giving sentence.


Introduction
In recent years, quantitative scholars have developed measurement techniques that are aimed to tap into the language used to symbolize subjective experiences of emotions and through this, quantify and generalize emotion-expressions. Some (e.g., Smith, Webster, Parrott, & Eyre, 2002) therefore design their questionnaires with a general heading à la "What would [someone or you] be feeling and thinking" and then present various key-words (like "embarrassed") for the participants to rate. Others argue that questionnaire statements should be measured within a meaning-giving context (e.g., a sentence) so the key-words (like "embarrassed") form a part of a meaning-giving appraisal emotion chain (e.g., Gausel, Leach, Vignoles, & Brown, 2012) as in natural language. But which is the better approach?
Based on semantic theory and appraisal theory of emotion we argue that a word has an unclear meaning until combined with other words in a sentence. Therefore, in this study we aim to demonstrate how self-critical emotion-words can be measured with greater precision and intensity when placed within a sentence than if presented in isolation (i.e., without being part of a sentence). The explanation is that an emotion-word presented in isolation should means less to a participant than an emotion-word combined with other words forming a sentence.

Does an Isolated Word Have a Meaning?
Obviously, words have a meaning. But the meaning of words is found in how they are used and not in how they are pre-defined (Wittgenstein, 1958, for a discussion, see Davidson, 1984;Quine, 1960). Hence, as words have an unclear meaning until they are placed within a sentence that allows for a meaningful interpretation of it (Davidson, 1984), single, isolated words often vary considerably in their meaning. This is why a dictionary typically offers several definitions depending on how a word is used by the relevant linguistic community, and even then, we must guess the correct meaning based on the context in which the specific word is used. The ambiguous word "small" is a good example. Oxford Advanced Learners' Dictionary (Horny, 1989) uses 75 lines to explain the meaning of the English words "small" and "small-ness", but only 3.3% explain its emotional meanings --even though people often use the word "small" to describe how they feel about something. natural language, like English, they must typically contain what may be identified as a nominal and a verbal phrase (Greene, 1972). In predicate logic, variables must, as a minimum, be assigned one predicate and bound by quantification; but to say simple things such as "I feel embarrassed", for instance, we need two predicates, one connective and all of it bound by a quantifier. Hence, it would be extremely difficult, and often impossible, to guess correctly what adults said if they restricted their language use to single-word utterances or if we looked up every word they used in a dictionary. Moreover, as language is a social institution, we learn to talk about, to differentiate among and to think about emotional experiences through the insight available to someone who has already learned the practice (Quine, 1960;Wittgenstein, 1958). This, however, depends on the linguistic community's ability to discern correct from incorrect utterances about emotions. For physical objects (say a "chair") this can be rather precise as we talk about them depending on different perspectives (Quine, 1960). Emotions, on the other hand, are not about physical objects. They are about subjective impressions. It is therefore difficult for the surrounding linguistic community to discern correct from incorrect use of emotion-words as they have to base their interpretation on behavioral observation as indirect evidence of the emotion (Frijda, 1986). Emotion experiences (especially the self-critical ones) can therefore be very difficult to identify (Beck, Emery, & Berg, 1986;Gausel & Leach, 2011), to articulate (Izard, 2009) and to communicate (Lewis, 1971).

Understanding Emotion-Words through Appraisals
How can we then understand what is meant with emotion-words?
When emotions are felt, we often express our emotions with words that are meant to symbol our emotional experience. But it is difficult, if not impossible to understand what is meant with the words if they are not relating an affective reaction to observable events. Aristotle (1984) therefore suggested that in order to define emotions one had to describe the situation to which they were a reaction. This is very similar to appraisal theory of emotions, where it is argued that an emotion can best be understood based on how one appraise or interpret the situation in which one finds oneself (Lazarus, 1991; see also Schachter & Singer, 1962). Hence, the determination of emotions is made possible by accounting for something meaningful that has happened or is happening. Lazarus (1991) therefore suggested that in order for emotions to be defined, there must be some cognitive goal-relevance associated with it. Without this one cannot know what is felt, as an emotion "cannot be adequately defined without reference to an individual's (…) appraisal" (p. 10).
As already mentioned, the word "small" can mean many things. It can be used to describe actual size of objects for example of a chair. Or it can be used to illustrate a trivial matter, or to say that something is of modest impact. But it can also be used to illustrate an emotional experience as in "I feel small" (e.g., Lewis, 1971;Smith et al., 2002). Obviously, the word "small" can be hard to define if it does not have a meaningful, cognitive appraisal attached to it (Lazarus, 1991) that can explain what is meant. Thus, if one knows what is cognitively appraised, then one also have a chance to understand what is meant with the words that people use to express their emotional experiences (Beck et al., 1986;Gausel & Leach, 2011).
In other words, a single emotion-word (measured in isolation) would not convey meaning unless it is associated with cognitive appraisals that can define it. It is only then that the participants can be fairly sure that they (and we, the researchers) understand what the emotion-words symbol.

The Current Study: Method
As we have argued that words do not carry a sufficient meaning until it is placed in a semantic context with other meaning-giving words, we expected that emotion-words would be better understood -and therefore responded to with greater intensity -if measured within a meaning-giving sentence than if measured in isolation (i.e., without being part of a sentence). Hence, we revisit a study by Gausel and colleagues (2012) where we investigate four items that were presented to participants within a sentence and without a sentence.

Participants
One hundred and ninety four Norwegian community participants (80 men and 114 women; mean age: 26.7, age range: 18-68 years) voluntarily agreed to partake in an anonymous study focusing on emotional responses to historical in-group misdeeds.

Procedure
In the first section of the questionnaire (page 1) we asked the participant to fill in their demographics. This was followed by a factual description about how a gypsy minority in Norway (termed "Taters") had been severely abused by the larger Norwegian community and the Norwegian state (for a closer description, see Gausel et al., www.ccsen 2012). Dir failure. W practically responses thoroughly

Measu
The four e either with they have specific m a general s I think ab questionna

Discussion
There are several ways to tap into the symbolic emotion-words used in a language if deploying a questionnaire. Some prefer to list isolated key-words for participants to rate (e.g., Smith et al., 2002). Others choose to place the key-words within a meaning-giving sentence and then let participants rate these sentences (e.g., Gausel et al., 2012). Even though there can be good reasons for both designs, we had argued that the latter would be preferable. Our conviction is mainly based on the knowledge we have from the semantic field that a word in itself have an unclear meaning until it is placed within a sentence that allows correct interpretation (Davidson, 1984).
As a single word can mean many things depending on how it is used, a participant that is asked to rate an isolated word (i.e., without a meaning-giving sentence) may be insecure about its meaning, and as a consequence, rate the word with lower intensity. In contrast, if the word is placed within a meaning-giving sentence, then the participant may be more secure about its meaning, and as a consequence, rate the word with higher intensity. Consistent with this hypothesis, the isolated emotion-words almost doubled in their mean-levels when placed within a meaningful sentence that allowed for a precise interpretation.
In fact, as around 95% (give or take some) of the definition of the word "small" is defined by something else than emotion (in both English and Norwegian language), the wide usage of this word renders the possibility that it can be interpreted widely as well. This is underlined by the large standard deviation (SD = 1.40, related to the mean it constitute 75.3%) for the isolated word "small" (M = 1.86). Had the word "small" always had this wide meaning then one should expect that this large disagreement would reflect itself within a sentence as well. But in comparison, when the word "small" (M = 3.30) is within a sentence, it has a standard deviation of 1.86, that, when related to the mean constitute 56.4% -a significant difference of 18.9%, z (194) = 3.93, p < .001. Hence, the word "small" is less disagreed among when placed within a meaning-giving sentence.
Taken together, this result support semantic philosophers' view that single words carries with it little meaning unless it is placed within a meaning-giving sentence (e.g., Donaldson, 1984;Quine, 1960;Tomassi, 1999;Wittgenstein, 1958) and it supports the sentence-based design of statements that some quantitative researchers promote (e.g., Gausel et al., 2012).

Possible Limitations
That said, we must acknowledge at least two limitations. First, the low intensity ratings of isolated emotion-words may be explained in a different manner. As unpleasant (and especially self-critical) emotion-words might have serious impact on participant's self-esteem , they are often avoided (e.g., Beck et al., 1986;Gausel, 2012;Lewis, 1971). Avoiding the impact of such emotion-words seems easier when they are placed outside of a meaning-giving sentence. In such cases, the participant has the possibility to deem the isolated emotion-words as less meaningful and therefore rate them lower. This tendency to avoid unpleasant emotions words by rating them lower might explain the low mean levels on unpleasant emotion words generally found in questionnaire research (Leach, Zeineddine, & Čehajić-Clancy, 2013).
Secondly, our study was conducted in the Norwegian language. As there are differences between languages, the words might also differ in their usage and definitions so some words might have a more specific meaning and other words a less specific meaning. Nevertheless, in the Norwegian language the word "liten" (i.e., "small")like in English -is a widely used word. According to the most influential Norwegian dictionary, Riksmålsordboken (Guttu, Skadberg, & Wettergren-Jensen, 1977), the word "liten" (i.e., "small") is explained with 18 lines where approximately 5.6% refer to an emotional experience. Although this indicates that this word has slightly more informative meaning concerning an emotional experience in Norwegian than in English (3.3%), it can be compared to its usage in the English language.

Practical Implications
In order to design a questionnaire that measure emotion-words, one must accept that the meaning of words derives from their relation with other words within a sentence (Davidson, 1984). A word simply cannot be isolated from the semantic context where it belongs and then be expected to keep its original and precise meaning. Hence, a questionnaire with a list of single emotion-words removed from its semantic context (i.e., a meaning-giving sentence) will likely produce systematic errors of a kind that undermines the genuine attempts by the researcher to understand what participants mean with their ratings. In fact, such an approach might produce a Type II error as the researcher cannot be sure that the instrument measures what it is believed to be measuring.
In addition to the above, as science is indeed an international activity, scientific results are read and produced across several languages. As each language has its own variation in the usage of emotion-words, one might risk that the interpretation of emotion-words will vary considerably across linguistically different communities. To place emotion-words within a meaning-giving sentence instead of measuring them in isolation, will help reduce this variation to one type of use, thus minimizing the risk of misinterpretation across languages as well.

Conclusion
Emotions are subjective experiences. Therefore, we need a semantic context that can provide an emotion-word with meaning so we can understand what people mean when they use emotion-words, and so they can understand what we mean with the emotion-words that we ask them to rate. As seen in this study, when unpleasant emotion-words are presented within a sentence, then our participants rated these words with significantly higher intensity than when presented to them in isolation. Consequently, for a more precise understanding, we researchers need the linguistic context rendered by a full sentence in order to be fairly sure that we measure what we think we measure. This is very important because the number of well-formed utterances within a language allows for far more variation and interpretation than what is offered by a lexicon definition of a word.