2.1 Overview of the emotional process

The emotional component is shown in more detail in Figure 1. In this figure, real boxes represent different processes within the model. The characters first perceive the environment. Upon receiving a perception (which can, for example, be the presence of another character or some resources, or even an action from another character) the character appraises the situations in a scene and triggers the specific intensities of emotional states depending on two major appraisal variables: desirability and likelihood. Since we focus on social characters, the values of two appraisals mainly depend on interactive actions done by characters. Two social learning algorithms are proposed, including the character’s awareness of the affiliations with other character as well as learning about the impact of other characters’ actions.

Figure 1

Emotion process

Generally personality traits play an important role in creating some sort of relationship with an individual or in cases where several virtual characters are placed in a social setting. In this emotional process, a set of nonlinear emotion progression functions are then used to vary the susceptibility of characters to a certain emotional state according to particular personality characteristics.

An emotion (or a mixture of emotions) once calculated is passed to the filtering process to produce the strongest emotion. The filtering process can activate emotions using certain emotional thresholds and output to the action selection process to generate an emotional behavior. At the same time, the emotions are decayed and fed back to the system for the next cycle. Furthermore, a set of insufficiently satisfied needs such as hunger, thirst, rest, safety, love, and affection might give rise to different motives with a certain strength. The character chooses an appropriate action depending on the motive itself and its strength besides emotional states. To avoid oscillations between conflicting behavioral strategies, a dynamic persistent factor is added to strengthen the strength of the currently active motive (described in the action selection context).

A description of those key components and how to integrate them to generate believable characters will be provided next.

2.2 Appraisal

The main function of the appraisal component is to generate a set of emotions with certain intensities based on the character’s appraising context environment. For example, being attacked increases the intensity of emotion anger, while a friendly greeting causes the character’s intensity of emotion joy to increase. As noted in Reilly (2006), a number of existing computational emotion models represent only the type and intensity of the emotion. The underlying cause that gives rise to such an emotion, however, provides an additional structure to the emotion, which in turn supports a range of interesting and important emotion-based behavioral effects. Our system uses the following more concrete emotional structure:

(1)

Here, E_i refers to a type of emotion i. R presents cause event, which will give rise to emotion i. Cause event can be looked as the character’s internal cognition on the external event that results in a number of internal goals of the character becoming more likely to succeed or fail. T is the character targeted by the emotion i, and V_pi represents the intensity of emotion i, which can be determined by functions I and P. The two functions will be illustrated next.

In the emotion structure, the categorize approach is used to separate emotional phenomena into a set of basic emotions. The OCC model (Ortony et al., Reference Ortony, Collins and Clore1988), for example, categorizes the range of emotions into 22 emotions. Later on, Elliott (1992) augmented the OCC model from 22 to 26 emotion types. Table 1 shows the relationships between the emotion type and related causing event used in our model.

Table 1

Relationship between cause event and emotion type

Initially, the quantitative intensities of emotions V_pi can be calculated through function I based on two appraisals:

1. (1) desirability of the event, which refers to the measure of whether the event facilitates or thwarts what the character wants, and

2. (2) likelihood of events to occur, which refers to the probability that a negative or positive event that is associated with each goal might happen.

In our paper, the two appraisals are taken from the social learning process, which will be illustrated in the social learning context. Thus, the intensities of these emotions are calculated using the following equations formulated by Reilly (1996):

(2)

The equations show the method by which intensities are calculated for various emotions given a likelihood value and an event desirability measure. Note that as expected events are less emotionally intense than surprising events, emotions such as joy and sadness use changes of likelihood instead of likelihood itself to affect their emotions intensity. For example, passing an exam is expected to succeed and then does succeed result in a less-intense emotional reaction than an unexpected success.

Other emotions like dread, anger, or love towards another, which do not depend directly on likelihood and desirability, are crucially measured according to the character’s attitudes, which are taken from the social learning process. For example, if the character learned a friendly attitude toward another character with intensity ν according to its past experience, in the future, it will experience the emotion of love towards another, with a degree of ν.

2.3 Personality

Personality represents the unique characteristics of an individual. In psychology research, there are many personality models that consist of a set of dimensions, where every dimension is a specific personality attribute. For example, the OCEAN model (McCrae & Costa, 1996) has five dimensions: Openness–Conscientious–Extraversion-Agreeableness–Neuroticism. Similarly, Rousseau and Hayes-Roth (1998) have developed a model of personality traits for social individuals that includes self-confidence, activity, and friendliness.

Our personality model considers three dimensions, which seem crucial for social interaction. Level of extraversion varies from sociable and talkative to unsocial and taciturn. Self-confidence varies from assertive and optimistic to timid and pessimistic. Friendliness varies from friendly and forgiving to rude. We assume numerical quantification of these dimensions, with a value between [−1.0, 1.0]. For example, a value of 1 in the friendliness dimension means that the character is very friendly, a value of −1 in the self-confidence dimension means that the character is very shy and timid.

Generally personality traits are used to set the thresholds for generating emotional states and controlling the intensities (Vinayagamoorthy et al., 2006). Information about a character’s personality can influence the probability of choosing actions explicitly or through algorithms that model uncertainty. Egges et al. (Reference Egges, Kshirsagar and Magnenat-Thalmann2004) define an m × n matrix called the personality–emotion influence matrix to explain how each personality factor influences each emotion, which was then linked to a dialog system represented by a 3D face. Chittaro and Serra (Reference Chittaro and Serra2004) use an OCEAN-based model of personality as input to a probabilistic automaton where behavior sequences are chosen from an animation library (and sometimes modified) based on personality.

In our personality process, a ‘sigmoidal nonlinearity’ function is used to model the difference in personality types as it affects a character’s emotional intensities, designed as follows:

(3)

Here V _0i and V_pi are the original values and new values of emotion i. The parameter k ₁ shifts the sigmoid left or right, and the parameter k ₂ controls the sigmoid steep or gentle. The two parameters repent how fast the output V_pi changes with the input V _0i. A smaller value of k ₁ and k ₂ can make the sigmoid steeper and more responsive. The parameter θ is a threshold. When it is far to the right, then a stronger input is required to activate an output.

Through adjusting the parameters k ₁, k ₂, and θ, the original emotional state is then used in conjunction with the character’s particular progression function to compute the degree of this emotion experienced by the character given their particular personality characteristics. Thus, three types of personalities, namely ‘general’, ‘confident’, and ‘timid’, have been defined to influence the fear emotion. By adjusting the parameters of the ‘sigmoidal nonlinearity’ function, we get three functions:

1. (1) a median function designed for a ‘general’ personality,

2. (2) a reduced function designed for a ‘confident’ personality, and

3. (3) an enhanced function designed for ‘timid’ personality.

Figure 2 shows the personality progression function used in this paper. Here the x-axis represents the original fear emotion value and the y-axis represents the new fear emotion value under the influence of the three types of personalities. We used trial and error to find the value of parameters (k ₁, k ₂, θ); those for confident are set to (0.6, 0.24, 0.2), for general to (0.4, 0.2, 0.15), and for timid to (0.2, 0.1, 0.1).

Figure 2

Fear emotion progression functions

Thus, by applying a set of character-specific progression functions, characters with particular personalities will behave differently given the same environmental or emotional stimulus.

2.4 Social learning

Learning can have a major influence on the creation of believable characters and various researchers have addressed its importance in the emotional process (Balkenius & Moren, Reference Balkenius and Moren2000; Gadanho, Reference Gadanho2001). Our research focuses on models of emotional learning or memory in situations where interaction arises (specifically, interacting with other characters). Two types of social learning are presented, including predicting actions of other characters that please or displease the character and the social relationships between characters.

2.4.1 Predict other character’s action sequence

In a multi-character interaction environment, learning the patterns of behavior of other characters and how likely particular actions of others are to occur are crucial for the process of generating emotions. More specifically, it acts directly on the appraisal variable likelihood, which in turn leads a character to adopt different emotional behaviors in social situations.

A probabilistic approach is used to learn the behavioral patterns of other characters. The actions available to characters are composed of two main types: looping and execution action. For example, the character walks towards food (looping action), and eats the food (execution action) when it reaches the food location. The two sequential actions lead to the character’s hunger need being satisfied. Therefore, we focus on two consecutive actions, that is looping and execution actions, by the other character. The calculation of another character’s behavioral patterns is given as follows:

Step 1: In the case of ‘social’ interaction, a statistical table is created to record another character’s sequential actions and counts, as well as the actions previously carried out by the character itself (see Figure 3).

Figure 3

Sequential actions

Here the dashed box denotes actions previously carried out by the character A itself, and the solid box denotes another character B’s sequential actions. When a pattern is first observed, an entry is created in the table that records sequential actions, with a count of 1. Then, every time this sequence is repeated, the count is incremented.

Step 2: Based on the previous statistical table, Equations (4) and (5) are used to help the character A predict the probability of another character B’s action sequence

(4)

Here the probability p(a_k) denotes the probability of an action a_k being carried out by character B under the conditions of character A’s current situation, c[a_k] denotes the counts of action a_k carried out by character B, denotes the counts of all actions carried out by B

(5)

Here the conditional probability p(a_k/a_m) denotes the probability of an action a_k taking place, given that action a_m has previously been carried out by B, and c[a_m, a_k] denotes the counts of sequential actions [a_m, a_k] carried out by B.

2.4.2 Relationship

In situations where social interaction exists, characters need to interact with each other over long periods of time, so that it is vital that we create characters with the ability to form social relationships. One important aspect of such a relationship is modeling the change of one character’s attitudes toward or dispositional (dis)-liking of another. Thus an individual variable called affiliation is presented in our model. Affiliation is a subset of a more complex dimensional representation of social–emotional relationships proposed by Argyle (Reference Argyle1990). The character’s affiliation value is affected directly by its interaction with other individuals and indirectly by its interaction histories with each of them. The following equation is used to learn interpersonal relation:

(6)

where F_m(t) is the new value of a character’s relationship, varying from 0.0 to 1.0. When F_m(t) is close to 0.0, a more unfriendly relationship is built. On the contrary, when F_m(t) is close to 1.0, a very friendly relationship is built. F_m begins with 0.5, representing an intermediate relationship. w(t) is the value attached to the current interactive action carried out by the character (−1.0 ⩽ w(t) ⩽ 1.0). A positive value denotes that the character’s current action is friendly (e.g. greet or play with the other character), otherwise a negative value is specified. η is a learning rate. We used trial and error to find a range between [0.02, 0.05]. c is the level of confidence associated with a relationship value between [0.0, 1.0]. It is initialized to 0.0 and then revised as follows:

(7)

Here c(t) is the new confidence value for the social memory, and α is a learning rate (0 ⩽ α ⩽ 0.02), which is chosen by the user to specify how many interactions it takes to form a relationship. Multiplying min(c, 1−c) keeps the confidence c between 0 and 1.0.

Finally, the attitude of one character to another is determined by the following equation:

(8)

The parameters here have the same meaning as previously specified. When F_m(t) > 0.5, F(t) represents a friendly attitude to another character; when F_m(t) < 0.5, F(t) represents a hostile attitude to another character. The strength of this attitude influences the level of the emotional appraisal variable desirability, which is increased or decreased directly, and finally leads characters to generate various social behaviors.

2.5 Emotion filtering and emotion decay

Each time emotions are created, a potential value for each emotion is determined by the appraisal process. However, emotions do not become active automatically. Each character has a set of emotional thresholds. The threshold represents the character’s resistance towards an emotion type. When an emotion intensity is below the threshold, it does not appear in the character’s emotional state. An emotion is taken into account only if the emotion potential surpasses the defined threshold. There may be several emotions active at one time. Inhibition can occur directly between these emotions. The filtering process chooses the emotion with the greatest intensity to determine the behavior of the character.

However, emotions do not disappear once their causes have disappeared, but rather dissipate over time, reflecting the dynamics of the emotional system itself. Therefore, the intensity of an emotion can be characterized as a decay function expressed by

(9)

Here V_pi is the intensity of emotion i when it was first generated. is the intensity of emotion i after some time T. When the value of is below the defined threshold, it is inhibited from the character’s emotional state. ϕ (the decay factor) represents how fast the intensity of an emotion type fades out. In the case of hope and fear, as noted by Reilly (2006), the shapes of the intensity curves are not the same as for other emotions. They can stay high as long as the trigger event remains. Accordingly, we give ϕ, a value close to zero, and then a bigger value when the trigger event is confirmed.

2.6 Action selection

As noted by Canamero (1997), action selection can be summarized as how to choose the appropriate actions at each point in time so as to work towards the satisfaction of the current goal (the most urgent need). Using Maslow (Reference Maslow1970)’s well-known theory of hierarchy of needs, we include two categories of needs:

1. (1) Physiological needs (such as hunger, thirst, fatigue, and safety) can be reduced by the consumption of matching resources or inhibited by a more urgent need. On the other hand, they can be enhanced by internal factors, such as amount of movement, intensity of movement, and the duration since the last need was satisfied. In addition, external stimuli can also increase the character’s related need. For instance, when the virtual character passes near a specific place where a need can be satisfied, the value of this need (and the associated goal-oriented behaviors) is increased proportionally to the inverse of the distance to this location.

2. (2) Needs for love and affection are adjusted through early interactive experiences with other individuals. They are triggered by other friendly characters. Note that an unfriendly individual can not only inhibit another character’s needs for love and affection but also increase its safety need.

Using these factors, we model the subjective evaluation of needs in terms of the following equation:

(10)

(11)

Here D_i(t) refers to the value of need i (between [0, 1.0]) at time t. strength_i refers to the intensity of movement (e.g. running has a higher value than walking). Again, we used trial and error to set the range between [0, 0.01]. growth_i(t) refers to duration. Both values can be specified by internal factors. effect_ki(t) refers to the strength (between [0.0, 1.0]) on an external stimulus k in adjusting need i at time t.

In our model, the strength of an external stimulus k depends solely on the time-varying distance d(t) between the stimulus and the character—the smaller the d(t), the stronger the stimulus. Let x(t) = 1/d(t) denote the strength of the stimulus. θ ₁ and θ ₂ are threshold values ranging between [0.1, 0.25] and [0.4, 0.5].

In this way intensities of needs exceeding the defined threshold can be generated. The action selection process chooses the one with the highest intensity to determine the behavior of the character. This method may sometimes lead to a dithering between behaviors: plans can be abandoned partway through execution to pursue other goals that have just become only slightly more pressing. To deal with this problem, we add a dynamic persistent_factor_i(t) to the relative strength of the currently active motive so as to avoid behavioral dithering (the rapid oscillation between different behaviors). The new persistent_factor_i(t) variable can then be updated by the following calculation:

(12)

Here D_i and likelihood_succeed can be calculated according to how the environmental context influences the current active goal. A low persistent_factor leads to more flexible behavior, while a high persistent_factor makes it difficult to change the active motive and helps to avoid oscillations between conflicting behavioral strategies. The range is a scaling rate between [5, 10], also set by trial and error.

Finally, the internal need with the highest level will trigger appropriate actions to get itself back into its comfort zone. In this process, emotions are added to modify or amplify needs and behaviors. More specifically, emotions play three roles:

1. (1) Modulating and evaluating a choice: Since emotions are related to goals, they can change goal priority in situations requiring an urgent response. If a character is being attacked, for instance, a dread or anger emotion will be generated, and the intensity of the safety need will be strengthened as well, inhibiting other goals so as to allow the character to escape from or attack its attacker.

2. (2) Enhance social interactions: Emotions and their expression are crucial in communication for virtual characters situated in complex social environments. Our proposed social learning component can also endow the characters with the ability to form social relationships autonomously and display them during the course of interaction.

3. (3) Make behaviors more realistic: Emotions generated at every time step can change the way in which behavior is executed so as to develop characters’ own personalities and increase their believability. Emotions contribute to the generation of richer, more varied, and flexible behaviors.

In order to understand the entire emotional process described in the previous section, the next section examines a small example.

4.1 A dynamic simulative environment

Based on our previous work (Pan et al., 2005), we have developed a virtual environment (shown in Figure 9) in C++ with a real-time 3D graphics library (based on Microsoft Direct3D) and articulated characters (using the Cal3D library). Roughly, the virtual world is composed of two kinds of entities: resources and virtual characters.

Figure 9

The virtual environment

Resources are entities that can trigger and satisfy the character’s bodily needs, such as hunger, thirst, fatigue, and so on. This work is aimed not only at task achievement (e.g., finding food) but also at doing so in a social context. So characters can steal/give resources from/to other characters, thus inhibiting or enhancing the character’s social affiliations. For example, being given (grabbed) food from/by another character B causes the hungry character’s attitude toward B to become more friendly (unfriendly). Their attitude toward other characters can then affect the style in which characters carry out social actions with respect to each other. Figure 10 shows some resources used in our system.

Figure 10

Some resources

Characters are entities that respond to and reconstruct the virtual world. They perceive the world by means of their sensors and act on the world by means of their motor controllers. The decision on what action to carry out results from the combination of a character’s physiological, cognitive, and social influences. In our experimental environment, we categorize the characters’ behaviors into three types: friendly(c), unfriendly(c), and explore resource, which are illustrated in Table 2.

Table 2

Behavior description

To convey the characters’ emotional states in the model, we use facial and body expressions. More specifically, friendly behaviors can be expressed as physical closeness, more direct orientation, more gaze, smiling, head nods, and lively gestures. On the contrary, crossed arms, lack of smiling, physical avoidance, and depressed gestures can be shown as unfriendly behavior (see Figure 11).

Figure 11

Building expressions into social characters

4.2 Validation

The virtual world contains three characters, ‘big head’, ‘small head’ and ‘boy’, as well as many resources. To evaluate whether or not our emotional model causes characters to exhibit human-like social behaviors, we performed a human user study involving 15 users randomly chosen in our lab to watch and interact with the characters in the following three runs:

1. (1) The three characters autonomously select their social behaviors based on the social learning method (‘learning method’). In addition, to evaluate how the personality component influences social behavior, in this run, we endow the character ‘boy’ with coward and friendly personality, ‘big head’ with confidence and friendly personality and ‘small head’ with general and unfriendly personality. During the run, the system records all the behaviors chosen by the ‘boy’ to interact with others.

2. (2) The three characters select their social behavior randomly at every meeting time (‘random method’). The ‘general’ personalities are used for all the characters.

3. (3) The three characters have been assigned clarity relationships and social behaviors, for example, ‘boy’ does always friendly behavior with ‘big head’ and unfriendly behavior with ‘small head’ at each interaction (‘scripted method’). The ‘general’ personalities are also used for all the characters.

In each run, the camera focuses on the character ‘boy’. Each character starts off with no social relationship. Each experiment runs roughly for 10 minutes. One can think of them as interactive versions of short stories or animated shorts. They are not especially large but are still complex enough to be interesting.

To help users to clarify the characters’ behaviors in the context of validation, on the one hand, we display some text on the graphic interface that described the character’s internal needs, internal emotions, and actions (shown on the left-top side in Figure 9). On the other hand, in the first run, the users can use the keyboard to direct virtual characters do something inconsistent with their goal-oriented behaviors. When this happened, the characters can adopt different affective behaviors to express their own feelings. For example, when the user controls the character to greet with an unfriendly character, it will express unwilling and unfriendly behavior such as crossed arms or turning away. User control actions include the following: (1) give resources to other characters; (2) greet or play with other characters; (3) steal resources from other characters; and (4) attack other characters. In the remaining runs, the users do not interact with the character. After the three runs, the users are asked to rate his or her opinion on a range of topics:

1. (1) Do you think the ‘boy’ is able to exhibit a clear relationship with the other two characters (y/n) during the three runs? Rate ‘boy’s’ friendly behavior, unfriendly behavior, and overall social behavior through facial and body expression. Answers in scale of zero to five (0—not clearly and 5—very clearly).

2. (2) Do you recognize each character’s personality according to its style of social behaviors (y/n) in the first run? If yes, describe their personalities.

3. (3) Do you feel the character’s behavior is consistent even when the users make the character carry out conflicting actions in the three runs. Rate your enjoyment in scale of zero to five (0—not interesting and 5—very interesting).

For the first question, all users rated the interpersonal relationship between characters in the first and third run. The detailed evaluation results are shown in Figure 12. From the users’ evaluation, we can see that the character ‘boy’ receives high scores in the first run on ‘overall behavior’ (rating was on average 4.0), ‘friendly behavior’ (rating was on average 4.0), and ‘unfriendly behavior’ (rating was on average 4.13). This demonstrates that users find the character boy’s social behavior convincing. The random character in the second run, considered a non-social character, cannot build clear relationships in all interactions. More users think the character disrupts their ‘suspension of disbelief’. Not surprisingly, the users rate the characters’ social behaviors in the third run with the highest scores, since the characters have pre-defined relation information. However, the first run still has nearly the same score as the third run.

Figure 12

The behavior of ‘boy’ evaluated by users

For the second question, Figure 13 shows behaviors of ‘boy’ through the interactions. Here, the x-coordinate represents tested behaviors including friendly and unfriendly behaviors. To identify the type of personality more clearly, the unfriendly behavior is subdivided into avoid and attack behavior. The y-coordinate represents the probability of each tested behavior adopted by the character ‘boy’. According to the users’ evaluation, all are able to distinguish the personalities among the characters clearly in the first run. They also recognize the character ‘boy’ is shy and fearful, while the character ‘big head’ is friendly and the character ‘small head’ is aggressive. The evaluations correspond to the system’s statistical results. These results demonstrate the impact of personality on characters’ social behaviors.

Figure 13

The social behavior of ‘boy’ to ‘big head’ and ‘small head’

For the third question, a low score is registered for the characters in the second run (the rating was on average 0.5). All the users find that the social characters in the first run express their own feelings by specific behavior under the control of users. The character’s expressions arouse the users’ interest with the highest score (rating was on average 4.2). Although the users think the characters’ behaviors are consistent in the third run, the characters always behave in the same way in a similar situation without the users’ control, which results in the character’s interest dropping to a low level (the rating was on average 2.67).

The results attained, although very limited, do however suggest that the use of emotional characters can lead to believable situations and do evoke users’ empathy, which results in the users becoming more immersed into the virtual world.