2.1.1 Simulation of escaping panic

Helbing et al. (Reference Helbing, Farkas and Vicsek2000) propose a model of crowd behaviour in the case of emergency escape. They aim to model panic situations where crowd pressure can be dangerous and to find solutions for the evacuation of dangerous places (e.g. smoke-filled rooms). The authors chose to represent the agents as particles in a force field: all physical and social interactions with other agents and the environment are represented as forces. The movement of particles representing the human beings are described by electromagnetic-like equations.

2.1.2 Simulation of pedestrian mobility after an earthquake

Van Truong et al. (Reference Van Truong, Beck, Dugdale and Adam2013) propose a model of urban pedestrian mobility after an earthquake in Lebanon, in order to show the importance of population preparedness to reduce casualties. In a data-based geographical space composed of streets, buildings and green spaces, agents representing human beings behave in order to evacuate to the green spaces. Their behaviour is driven by a set of rules influenced by agent features (gender, age, etc.) and by stimuli from the environment and from the other agents.

2.1.3 Simulation of survival in a group

Cecconi and Parisi (Reference Cecconi and Parisi1998) propose the use of simulation to evaluate various survival strategies of individuals in a social group. Surviving (and reproducing) or dying depends on the resources available to the agent. These resources can be physical resources (such as food, money), other agents (such as a reproduction partner) or personal skills (such as fighting or working capability). Agents can use two types of strategies: individual survival strategies and social survival strategies; a particularly interesting social strategy consists in giving part of what the agent collects to supply a common pool of resources. The authors were interested not only in the evolution of skills over generations but also in the ideal size of the social groups. Each agent is represented as a neural network. The evolution is modelled by a genetic algorithm aimed at improving the capability to find and produce food.

2.1.4 Crowd simulation for emergency response

Shendarkar et al. (Reference Shendarkar, Vasudevan, Lee and Son2008) model a crowd evacuation under terrorist bomb attacks in a public area. They investigate the influence of various important parameters (e.g. density or size of the crowd, number of policemen driving the evacuation, number of exits in the room) on the quality of the evacuation (e.g. evacuation time, number of casualties). Agents are implemented with an extended BDI architecture, which includes an emotional component and a real-time planner. The behaviour of the agents is captured from a human evolving in a Virtual Reality laboratory (participative modelling).

2.1.5 Revamping the simulation of the survival in a group

Sun and Naveh (Reference Sun and Naveh2007) propose to give genuine cognitive capabilities to the agents involved in Cecconi’s simulation (Cecconi & Parisi, Reference Cecconi and Parisi1998), in order to make it more cognitively realistic. Moreover, they introduce social institutions that impose obligations on the agents and sanctions in case of violation; for example, an agent can be punished if it refuses to contribute food to the central store when required. Through these additions to the original model, the authors aim at studying the interactions between individual cognition and social institutions. The cognitive processes of the agents are implemented by using the CLARION cognitive architecture, mainly based on Sun et al. (Reference Sun, Merrill and Peterson2001) work in cognitive sciences. It implements mechanisms for implicit vs. explicit learning, among numerous other cognitive capabilities.

2.1.6 Summary

These few examples show a huge variety in the agent architectures. Indeed, they illustrate respectively: a particle architecture; a rule-based architecture; a neural network architecture; a BDI architecture; and a cognitive architecture. We can make a dichotomy between these five architectures w.r.t. the concepts manipulated to implement the behaviours and their complexity level.

The first three architectures (particle, rule-based and neural network) do not involve any cognitive concepts in the reasoning and decision-making process. Even if agents represent human beings, their behaviours are for the most part a function of the external stimuli (e.g. the external forces in the first example, the perception of the environment in the second and third examples). These architectures are traditionally called reactive agents (Wooldridge, Reference Wooldridge2009). They are very well adapted to represent simple entities without complex reasoning capabilities (e.g. ants) or to model human beings when focussing on simple behaviour, but they cannot deal with complex reasoning mechanisms involved in more complex human behaviourFootnote ¹ .

In contrast, the last two architectures represent two kinds of intelligent agents (Wooldridge, Reference Wooldridge2009): BDI agents and agents with cognitive architectures. The expression ‘cognitive architecture’ used here refers to architectures based on cognitive theories, such as SOAR, ACT-R or CLARION (Anderson, Reference Anderson1983; Newell, Reference Newell1990; Sun, Reference Sun2002); they are often very complex and based on low-level physiological or neurological concepts. It should not be confused with the term ‘cognitive agents’ that is usually synonym to ‘intelligent agents’ in the MAS literature. In this wider sense, agents with a BDI architecture are also considered cognitive; but the BDI architecture provides a description of behaviour at a much higher level, in terms of mental attitudes. Intelligent agent architectures allow agents to be more autonomous and proactive, not only reacting to environmental stimuli, therefore matching humans more closely. They enable the modelling of more complex issues than would be possible with purely reactive agents (as shown by the progression between Sections 2.1.3 and 2.1.5), making them very relevant to our aim of integrating much more descriptive agents in social simulations. Finally, their strong theoretical grounding can provide many guidelines in the modelling of agents, but also many constraints.

2.2.1 Another survey of agent architectures

Balke and Gilbert (Reference Balke and Gilbert2014) recently published a survey of 14 decision-making architectures for agents, from very simple to very complex, aiming at providing guidelines regarding which one of them should be used for a given research question. One of the architectures they study is the BDI one, some others are various extensions of the standard BDI (with emotions, norms, etc.).

On the contrary in this paper, we focus only on the BDI architecture. We started from the observation that the agents used in simulations are too simple, and more descriptive agents would bring many benefits in realism and validity of results. In the next paragraph we explain why we focus on BDI rather than cognitive architectures such as SOAR or ACT-R.

Our goal is therefore different in that we specifically argue for the wider use of BDI agents in social simulations. We also go beyond Balke and Gilbert’s review by providing a technological guide showing interested designers not only why they should use BDI agents, but also how to use them. Indeed, even when designers are aware of the benefits that BDI agents could bring in their simulation, they are often limited by technical difficulties, in particular a lack of methodologies and tools.

2.2.2 BDI vs. cognitive architectures

Both BDI architectures and cognitive architectures allow modellers to deal with the cognitive processes of the agents and to provide the descriptivity we (after Edmonds & Moss, Reference Edmonds and Moss2005) claim is needed in some social simulations. However, these two kinds of architectures are very different.

First of all, they differ on their conceptual inspiration, that is, the origin of the underlying concepts. Cognitive architectures are based on cognitive sciences and aim at describing human cognitive processes as precisely as possible. This requires in-depth analysis of the functioning of the human brain from a physiological or neurological point of view (e.g. to determine the area of the brain that performs a given function). In contrast, the BDI architecture is based on a philosophical stream. Bratman’s (Reference Bratman1987) work is formalised mainly with modal logics (Cohen & Levesque, Reference Cohen and Levesque1990; Rao & Georgeff, Reference Rao and Georgeff1991). These logics describe the three main intentional states (beliefs, desires, intentions) that lead the agent from its world knowledge, to the choice of the action to perform, and to the execution of these actions. The BDI architecture therefore offers a more straightforward description making models easier to understand for modellers and end-users.

As a result, BDI and cognitive architectures also differ on the abstraction level of the concepts manipulated. BDI concepts are higher-level concepts (mental attitudes), that actually result from the low-level concepts and principles described in cognitive architectures (e.g. memory chunks in ACT-R). BDI architectures can thus be viewed as the explicit and symbolic part of cognitive architectures. BDI architectures focus on the conscious level of one’s mind, at a level where human beings can consciously manipulate the concepts and communicate them to others (Norling, Reference Norling2004). This allows the easy mapping of knowledge from experts to agent behaviour, and helps non-computer scientists to understand agent behaviour.

Finally, they differ on their degree of theoretical standardisation. Each cognitive architecture implementation builds upon its own theory: for example, CLARION (Sun et al., Reference Sun, Merrill and Peterson2001) is based on Sun’s work (Reference Sun2002); SOAR (Rosenbloom et al., Reference Rosenbloom, Laird and Newell1993) is based on Newell’s work (Newell, Reference Newell1990); and ACT-R (Anderson & Lebiere, Reference Anderson and Lebiere1998) is based on Anderson’s work (Anderson, Reference Anderson1983, Reference Anderson1993). They have no common focus: CLARION is interested in the dichotomy implicit vs. explicit, in particular for the learning process, whereas ACT-R focusses on the activation process. There is no consensus either on the memory component of the architecture, be it on the number of memories or their nature. On the contrary, in all of the substantial number of existing implementations of the BDI framework and formalisations of the underlying modal logic, the BDI trinity is a constant, allowing a huge community of researchers to work on its improvement and extensions.

Sun has argued that cognitive architectures can enhance simulations (Sun, Reference Sun2006), and proved his point by revamping an existing simulation with a cognitive architecture instead of the initial reactive agents (Sun, Reference Sun2007). We argue that BDI architectures can similarly enhance simulations, and also provide additional benefits as shown by Norling: ‘the close match between the core BDI concepts and the natural terminology that the experts being modelled used to describe their reasoning facilitated knowledge capture, representation and debugging’ (Reference Norling2004).

Therefore BDI architectures appear to be both descriptive enough to describe cognitive processes influencing behaviour, and intuitive enough to be easily understood by modellers and non-computer scientists in general. For these reasons, in this paper we are more particularly interested in BDI agents, and want to provide a guide as to when they can and should be used in social sciences simulations. A similar guide could be proposed about when to use cognitive architectures, but this is not the scope of this article.

2.3.1 BDI Theory

The BDI model is theoretically grounded in the philosophical work of Bratman (Reference Bratman1987) and Dennett (Reference de Silva, Sardina and Padgham1989) where the three basic components (beliefs, desires and intentions) are defined. The model attempts to capture the common understanding of how humans reason through: beliefs which represent the individual’s knowledge about the environment and about their own internal state; desires or more specifically goals (non-conflicting desires which the individual has decided they want to achieve); and intentions which are the set of plans or sequence of actions which the individual intends to follow in order to achieve their goals.

Cohen and Levesque (Reference Cohen and Levesque1990) distinguish BDI from purely reactive models by adding a level of commitment to the set of intentions to achieve a long-term goal. This notion of commitment is therefore a critical part of the BDI model. Rao and Georgeff (Reference Rao and Georgeff1991) show there must also be some form of rational process for deciding which intentions to select given a set of circumstances. While the specific way in which this is done varies among implementations, it is generally guided by folk psychology principles (Norling, Reference Norling2004).

So the core functionalities that BDI systems should implement are as follows: a representation of beliefs, desires and intentions; a rational and logical process for selecting intentions; and a flexible and adaptable commitment to the current set of intentions.

2.3.2 Implemented systems

There are a substantial number of implemented BDI systems (e.g. JACK (Busetta et al., Reference Busetta, Ronnquist, Hodgson and Lucas1999), Jason (Bordini et al., Reference Bordini, Hubner and Wooldridge2007), 3APL (Hindriks et al., Reference Hindriks, De Boer, Van der Hoek and Meyer1999), Gorite (Rönnquist, Reference Rönnquist2008), JADEX (Pokahr et al., Reference Pokahr, Braubach and Lamersdorf2005), etc.) supporting the creation of agents defined using these three constructs, and several systems extending these to provide additional human reasoning capabilities. Common to all these implementations is the concept of a plan library which is a collection of plans that the agent is capable of performing. Each of these plans consists of a body, a goal which the plan is capable of achieving, and a context in which the plan is applicable. The plan body can consist of both: actions which can affect the environment directly, and subgoals which will be expanded into other plans when needed. This nesting of goals within plans allows for a plan–goal hierarchy structure to be constructed, with branches from a goal node leading to plans capable of achieving this goal, and branches from a plan node leading to goals which must be achieved to complete this plan.

Implemented BDI systems also contain a BDI execution engine, which guides the agent reasoning process. There are some variations in the algorithms used by different engines, but a basic version is outlined below:

1. 1. Perception: find any new events which may have been triggered, either within the agent or externally from the environment.

2. 2. Update: update beliefs with new information provided by this event.

3. 3. Intention revision: if the change in beliefs means that an intention or goal is no longer valid (either because it has already been achieved or is no longer possible) then remove it from the set, which may result in some form of failure recovery by attempting to achieve it with a different plan.

4. 4. Plan filtering: if the intention was revised, determine the set of applicable plans for the current event, which are appropriate given the current context, and are defined as capable of achieving the goal associated with the event.

5. 5. Plan selection: if there is no current plan, select a new plan from this set (the selection method varies from selecting the first available plan to selecting the plan with the greatest expected utility) and add the components of its body to the intention list.

6. 6. Action: execute the next step of the current plan, which may involve executing an action or expanding a subgoal by triggering a new event.

2.3.3 Core features and benefits

The BDI architecture therefore offers the following desirable core features:

• ∙ Adaptability: plans are not fully specified and can consist of nested subgoals, which allows flexibility in the agent, making it able to adapt to a changing environment.

• ∙ Robustness: the plan hierarchy structure means that if a plan fails, potentially because of environment changes during its execution, the agent is able to recover by executing another applicable plan if one is available.

• ∙ Abstract programming: the programmer specifies beliefs, desires and intentions at a high level, making it possible to create complex systems while maintaining transparent, easily understandable code.

• ∙ Explainability: using a goal-based approach as opposed to a task-based approach means that an agent knows why it is doing a specific task. This allows agents to explain their behaviour to the modeller or the user in an intuitive manner.

2.3.4 BDI extensions

Beyond these core features, the BDI framework can be extended to provide additional desired features, and many works were dedicated to such extensions.

For instance, Singh et al. (Reference Singh, Sardina, Padgham and James2011) extend BDI with learning, in order to improve plan selection by dynamically adjusting a confidence measure. There has also been work for integrating real-time planners into the BDI architecture to support the creation of plans at runtime using First Principles (de Silva et al., Reference Dennett2009) among other techniques. Nair et al. (Reference Nair, Tambe and Marsella2005) use a BDI model of coordination, combined with POMDP to handle uncertainty, and prove that their agents perform better teamwork.

A large panel of social concepts have been added to the basic BDI framework. Dignum et al. (Reference Dignum, Morley, Sonenberg and Cavedon2000) introduce the concept of norms to ‘assist in standardising the behaviour of individuals, making it easier to cooperate and/or interact within that society’. In the same way human behaviour is very much influenced by social norms, agent behaviour can be influenced by norms too, allowing agents to have a better understanding of what other agents are likely to do. Gaudou (Reference Gaudou2008) focusses on the introduction of social attitudes (and in particular group belief), and (Lorini et al., Reference Lorini, Longin, Gaudou and Herzig2009) go one step further by modelling the group acceptance in an institutional context. Trust has also been introduced in several BDI works: Taibi (Reference Taibi2010) has implemented trust in a BDI interpreter to help agents to delegate tasks; while Koster et al. (Reference Koster, Schorlemmer and Sabater-Mir2012) allow the agent to reason about its trust model and to adapt it to a given context.

Adam et al. extend the BDI framework with emotions by formalising their triggering conditions (Adam et al., Reference Adam, Gaudou, Herzig and Longin2006, Reference Adam, Herzig and Longin2009), and propose an associated reasoning engine for BDI agents (Adam & Lorini, Reference Adam and Lorini2014). Other works had the same aim: Pereira et al. (Reference Pereira, Oliveira, Moreira and Sarmento2005) present an extension to the BDI architecture to include Artificial Emotions affecting an agent’s reasoning in a similar way to human emotions; and Jones et al. (Reference Jones, Saunier and Lourdeaux2009) also propose an extended BDI architecture integrating emotions.

2.3.5 BDI in the scope of this paper

In this section we have explained what is BDI: a paradigm for formalising agent behaviour in terms of mental attitudes (beliefs, desires and intentions). But we have also shown that BDI has multiple implementations, and multiple extensions (with emotions, with norms, etc.). In the end, there are almost as many BDI architectures as there are applications, most of them relying on an ad hoc implementation. In the sequel of this paper, we are interested in the methodological and conceptual aspects of the BDI paradigm, by opposition to the engineering aspects. We will call ‘BDI’ all simulations claiming to use this paradigm, and describing the behaviours of their agents in terms of mental attitudes, whether they use only the original beliefs, desires, intentions or also additional mental attitudes, and whatever their underlying implementation.

3.1.1 Representation and reasoning

The BDI framework is a model of how humans represent their perceived environment in terms of mental attitudes, and how they reason about it to deduce new mental attitudes from their existing ones. Works in modal logic have investigated these concepts in details, and provided formalisations of their links with each other and of various deduction rules. Modal logic can be seen as a strict and constraining formalism, but it has the advantage of being totally unambiguous.

The BDI model allows a subjective representation of the environment in terms of beliefs, that can be incomplete, erroneous (due to flawed perceptions), or differ between agents. For example, in the AMEL model of evacuation after an earthquake, each agent has their own map of the roads and position of obstacles (Van Truong et al., Reference Van Truong, Beck, Dugdale and Adam2013). To account for the uncertainty of perceptions in unknown or partially known environments, Farias et al. (Reference Farias, Dimuro and da Rocha Costa2010) have extended BDI agents with fuzzy perception.

Another major benefit is the possibility for the agent to represent and to reason about other agents’ mental attitudes (Theory of Mind, Goldman, Reference Goldman2012). For example, Bazzan et al. (Reference Bazzan, Wahle and Klügl1999) have developed a model of traffic in which (social BDI) drivers can reason about other drivers’ behaviour when making their decision. Similarly, Bosse et al. (Reference Bosse, Memon and Treur2007) have developed a BDI model allowing agents to reason about other agents and their reasoning process, and use it to represent a manager’s manipulation behaviour over his employees.

Such a capacity to represent and reason about the world in terms of mental attitudes can be used to develop agents that can explain their behaviour in understandable terms to human modellers and users: these are known as self-explaining (Harbers, Reference Harbers2011; Fähndrich et al., Reference Fähndrich, Ahrndt and Albayrak2013).

3.1.2 Self-explanation of behaviour

The issue of explaining a reasoning is as old as expert systems (Swartout et al., Reference Swartout, Paris and Moore1991), a field where a lot of work has been done to develop explanatory systems and methodologies. In particular, when an expert system is used in a learning environment (for instance, a military training environment (Gomboc et al., Reference Gomboc, Solomon, Core, Lane and Lent2005)), it is very important for the users to be able to get explanations about the reasoning made by the system, and about how and why it behaved as it did. More generally, Ye and Johnson (Reference Ye and Johnson1995) have shown that an expert system explaining its response was better accepted by end-users: indeed if the system can justify its choices, then it will increase the end-user’s trust. This issue is still current in MAS (Fähndrich et al., Reference Fähndrich, Ahrndt and Albayrak2013).

In agent-based simulations, the problem is quite similar: a simulator used to teach something is more effective if it is able to give explanations of its behaviour. More generally, Helman and Bahuguna (Reference Helman and Bahuguna1986) have shown that explanations of computer science simulations are useful for modellers to better understand the simulation. For example, we can assume that an agent-based simulation used as a decision-support tool by policy-makers (e.g. in urban planning, ecological risk assessment, etc.) would be better understood and accepted, and as a consequence more efficient, if some agents (and in particular those representing human beings or institutions) could be asked for explanations.

The introduction of BDI agents in the simulation is particularly interesting from this point of view. For example, Harbers et al. (Reference Harbers, van den Bosch and Meyer2010) have investigated the usefulness of BDI agents to provide explanations to the learner in a training simulation. Indeed an interesting capability of BDI agents is that they can explain their behaviour in terms of their goals (why they did what they did) and their beliefs (what were the reasons for their choice of action); and these explanations can become much more precise by adding complementary concepts (e.g. norms) to the basic BDI model. Moreover, the concepts involved in these self-explanations are very intuitive for human beings because they match their own natural language and their way of thinking (Norling, Reference Norling2009), making explanations provided by BDI agents more effective.

Therefore, BDI concepts facilitate the creation of explanations. Nevertheless, the counterpart of this gain in terms of explanation power is a much more complex reasoning process. This is not necessarily a drawback against the use of BDI agents, on condition that these agents are systematically endowed with a self-explaining capability. Indeed, as highlighted in Harbers et al. (Reference Harbers, van den Bosch and Meyer2010), without providing this capability to agents, the agents behaviour and therefore the whole simulation become almost impossible to understand, both by computer scientists implementing it and by modellers trying to use it. To put it differently, a side-effect of the introduction of BDI agents in simulations is the necessity to systematically endow agents with a clear self-explanation capability in order to make the simulation understandable, but the implementation of this self-explanation capability is facilitated by the BDI concepts underlying the reasoning of these agents.

3.1.3 Complex interactions via communication

In the agent-oriented programming community, in parallel to the development of the BDI architecture many efforts have been dedicated to the development and formalisation of Agent Communication Languages (ACL) (e.g. KQML (Finin et al., Reference Finin, Fritzson, McKay and Robin1994) or FIPA-ACL (FIPA, 2002a) to cite the most important ones) and interaction protocols (IPs) (e.g. the FIPA Contract Net Interaction Protocol (FIPA, 2002b)). These ACL and protocols enable structured interaction between agentsFootnote ² .

Such ACLs and IPs have two main advantages, that can be directly applied to simulation. First of all they allow agents to communicate in a realistic (human-like) way, with a large choice of speech primitives. The communication is closely linked to the mental states of the agents by the ACL semantics (e.g. the FIPA-ACL one (FIPA, 2002a)). Therefore, this allows one to easily model the effect of the communication on the agent behaviour: received messages modify the mental states of the agent, which in turn have an influence on its behaviour. For example, Fernandes & Nunes (Reference Fernandes and Nunes2008) propose a traffic simulator in the context of Intelligent Transportation Systems, with communication between vehicles. Thus, the simulator must take into account this communication and its influence on the vehicle driver’s behaviour. The authors follow the Prometheus modelling methodology (Padgham & Winikoff, Reference Padgham and Winikoff2002) to integrate BDI agents and communication via FIPA-ACL.

The second interest of using complex communication in the simulation is the use of the IPs. Some protocols have been formalised, in particular by the FIPA consortium, in order for agents to coordinate their tasks. This is particularly useful when the simulation manages a swarm of agents such as in rescue simulations. Some simulation platforms integrate these mechanisms for this purpose. For example Sakellariou et al. (Reference Sakellariou, Kefalas and Stamatopoulou2008) have developed a forest fire simulator where firemen agents cooperate and coordinate under the Contract Net Protocol. To this purpose, they have extended the NetLogo platform (Wilensky, Reference Wilensky1999) with a tool for easily integrating BDI agents exchanging FIPA-ACL messages in a FIPA protocol. In addition, Thabet et al. (Reference Thabet, Hanachi and Ghaedira2009) show on a grid scheduling example that a MAS involving BDI agents communicating with high-level FIPA-ACL can bring benefits on two levels: at the micro-level, agents are able to perceive the network, discover failures and reason about them to choose the most adapted behaviour; at the macro-level, agents use high-level IPs, providing them with a powerful and flexible way to communicate.

Complex communication mechanism with BDI agents are also used to model negotiation behaviours. Such models have been used, for instance, to simulate negotiation in mix human-agents teams in virtual worlds for training (Traum et al., Reference Traum, Rickel, Gratch and Marsella2003). The aim of this simulation is to improve the leadership skills of human beings by interacting with virtual teammates as believably as possible.

To conclude, when using BDI agents, the use of complex communication with ACL and IP is quite easy, in particular because both share the same philosophical ground and the same concepts. This provides, when it is needed in the model, powerful coordination, cooperation or negotiation mechanisms via communication.

3.1.4 Planning

Plans are sequences of actions that an agent can perform to achieve its goal(s). In some simulation scenarios a modeller may want agents to continuously rethink their behaviour, based on their situation and goal(s), in a manner that cannot be captured by simple if-then rules. This may involve, for example, modelling a complex decision process, supplying the agent with a comprehensive library of plans, or equipping an agent with fundamental planning capabilities.

Examining the application of multi-agent simulation even in just one domain (crisis response) we already find agents equipped with a broad range of planning capabilities. PLAN CFootnote ³ uses reactive agents with simple scripted plans to model medical and responder units in a mass casualty event simulations to explore the medical consequences of possible response strategies. DrillSim (Balasubramanian et al., Reference Balasubramanian, Massguer and Mehrotra2006) model fire warden and evacuee agents with a recurrent artificial neural network to select their course of action and an A* algorithm for low-level path planning; these agents are used to test the benefits of information collection and dissemination in a fire evacuation scenario. D-AESOP (Buford et al., Reference Buford, Jakobson, Lewis, Parameswaran and Ray2006) models medical relief operations in crisis situations, in an attempt to support more effective organisation, direction and utilisation of disaster response resources. It appears to be necessary to consider the influence of significantly more information (social, medical, geographical, psychological, political and technological) on agents’ behaviour. Therefore, the developers claim to need a rich agent model and choose the BDI paradigm where they replace event/goal plans with ‘situation’ plans that capture a broader view of the disaster situation.

As discussed in Section 2.3, the standard BDI paradigm uses a predefined library of plans tagged by their applicability context, and the goal(s) which they seek to achieve. There exist different extensions to the BDI paradigm for equipping agents with fundamental planning capabilities, so they can build their own plans during the simulation: classical planning (e.g. de Silva et al., Reference Dennett2009) lets agents synthesise courses of action depending on available domain knowledge; Hierarchical Planning (e.g. Sardina et al., Reference Sardina, de Silva and Padgham2006) supports the generation of high-level plans based on hypothetical reasoning about the relative value of plans.

There are some examples of such extensions being used in the context of multi-agent simulation, Shendarkar et al. (Reference Shendarkar, Vasudevan, Lee and Son2008) use agents with a real-time planning capability in a crowd simulation. Schattenberg present JAMES (Schattenberg & Uhrmacher, Reference Schattenberg and Uhrmacher2001), a Java-based agent modelling environment for simulation which supports the development of BDI agents with fundamental planning capabilities. The authors claim that ‘adding planning capabilities to an agent architecture offers the advantage of an explicit knowledge representation combined with a stable and robust decision procedure for the strategic behaviour of the agent’. Novák et al. (Reference Novák, Komenda, Cap, Voknnek and Pechoucek2013) work on autonomous military robots teams for surveillance missions: their first implementation using a reactive hierarchical planner was shown to be inflexible due to its extreme simplicity, and thus later replaced with BDI agents. However, these authors noted that BDI agents were too heavy-weight and should be limited to scenarios where complex deliberation was actually needed.

3.1.5 Emotional abilities

Cognitive psychological theories (Ortony et al., Reference Ortony, Clore and Collins1988; Lazarus, Reference Lazarus1991) describe emotions in terms of mental attitudes; for example, joy is the conjunction of a desire and a belief that it is satisfied. This straightforward connection with the BDI model played in favour of the use of such emotion theories in agents, as noted by Nair et al. ‘one of the appeals of cognitive appraisal models as the basis of computational model is the ease with which appraisal can be tied to the BDI framework often used in agent systems’ (Reference Nair, Tambe and Marsella2005). Conversely, this easy link between BDI and emotional concepts also constitutes an argument in favour of using BDI agents in simulations where emotional abilities are needed.

The standard BDI framework needs to be extended to integrate emotions (see Section 2.3.4), but it then helps to reason about them and understand their influence on the agents behaviour. Therefore, a number of BDI models of emotions have been developed (Adam, Reference Adam2007; Steunebrink et al., Reference Steunebrink, Dastani and Meyer2008; Adam et al., Reference Adam, Herzig and Longin2009), and a number of BDI agents have been endowed with emotions to make them more believable (e.g. embodied conversational agents (de Rosis & et al., Reference DeAngelis and Gross2003; Guiraud et al., Reference Guiraud, Longin, Lorini, Pesty and Rivière2011)) or efficient (e.g. virtual tutors (Gratch & Marsella, Reference Gratch and Marsella2005)). Such emotional agents also find applications in various types of simulations, for example, to model panic in an evacuation scenario (Minh et al., Reference Minh, Adam, Canal, Gaudou, Vinh and Taillandier2012) or to model the population possibly irrational behaviour in a bushfire (Adam et al., Reference Adam, Beck and Dugdale2015).

In addition to the appraisal process (the way stimuli are appraised to trigger an emotion), the coping process (the way agents deal with their emotions via various strategies) is also very important as it impacts the agents reasoning and behaviour, and as such has already been integrated in some BDI agents (Adam & Longin, Reference Adam and Longin2007, Reference Adam and Lorini2014). For example, Gratch and Marsella (Reference Gratch and Marsella2004) have developed a general BDI planning framework integrating appraisal and coping processes to trigger emotions in an agent, based on its interpretation of the world, and to model their influence on its behaviour. This framework called emotion and adaptation (EMA) has been applied to military training simulations: human officers interact in a virtual world with virtual humans in order to improve their skills in real-life stressful situations. Complex agent behaviours (in particular emotional behaviours) are needed to add realism and favour immersion, which in turn enhances training.

BDI agents have also been endowed with emotions in simulations aiming at exploring the relationships of emotions with various other concepts. Staller and Petta (Reference Staller and Petta2001) show that emotions have a bi-directional relationship with norms: emotions can be an incentive to respect norms (e.g. shame or guilt) or to enforce them (e.g. contempt); and norms influence the triggering and regulation of emotions. They have proposed an implementation in TABASCO and aim at reproducing the results of Conte and Castelfranchi (Reference Conte and Castelfranchi1995). Nair et al. (Reference Nair, Tambe and Marsella2005) have investigated how emotional abilities can improve teamwork in teams composed of either only agents or of both agents and humans; agents simulating human beings have a BDI architecture. Inspired by neurosciences, Bosse et al. (Reference Bosse, Hoogendoorn, Klein, Treur, Van Der Wal and Van Wissen2013) modelling proposes an evacuation simulation where the agent behaviour model is based on beliefs, intentions and emotions, and the relationships between these three concepts. Authors also take into account social interactions, and in particular emotional contagion. Jones et al. (Reference Jones, Saunier and Lourdeaux2009) have developed a BDI extension called PEP→BDI that enriches the BDI engine with personality, emotions, as well as physiological factors; they apply it to a simulation aimed at training security actors to handle major scale crisis, such as terrorist attacks.

However, not all simulations use BDI agents to model emotions, some use very simple mechanisms instead to deal with emotions. For example, in Minh et al. (Reference Minh, Adam, Canal, Gaudou, Vinh and Taillandier2012), emotion is represented as a single numerical value that increases with perceived danger and influences from others, and decreases with time. Authors claim it is enough to deal with the impact of emotions on evacuation in case of a crisis. But much more complex mechanisms can also be used. For instance, Silverman et al. (Reference Silverman, Badler, Pelechano and O’Brien2005) have integrated emotions in their multi-agent communication for evacuation simulation (MACES) crowd simulation system by using PFMserv, a cognitive architecture claimed to enable more realistic agent behaviour, by providing the agents with a working memory similar to human short-term memory, as well as a long-term memory, both used in their decision cycle.

Although other (more simple or more complex) models of emotions exist, the BDI model brings major benefits in social simulations, because it allows reasoning and interpreting emotions of self and others (reverse appraisal, Adam et al., Reference Adam, Gaudou, Longin and Lorini2011; Rivière et al., Reference Rivière, Adam and Pesty2012), and formalising their influence on behaviour via coping strategies.

3.1.6 Reasoning about norms

Norms are a set of deontic statements (obligation, permission or prohibition) specifying how agents are expected to behave; they are often represented in deontic logic (von Wright, Reference von Wright1963; Conte & Castelfranchi, Reference Conte and Castelfranchi1995). Social norms are norms shared by a group of agents; they are a very powerful tool to describe the rules in a group of agents. It is important to consider the fact that these norms can be violated; therefore a mechanism to deal with violations is needed, such as sanctions applied to guilty agents. Each agent has to take norms into account in his decision-making process, his desires should be balanced by norms. The basic version of BDI does not integrate norms, but some extensions do, for example, the BOID architecture (Broersen et al., Reference Broersen, Dastani, Hulstijn, Huang and van der Torre2001; Neumann, Reference Neumann2010) which focusses particularly on conflicts between the four modalities.

Norms are very important in several applications of MAS (such as business-to-business or e-business) but they are particularly essential in the modelling of socio-environmental systems (Ostrom, Reference Ostrom2007; Gaudou et al., Reference Gaudou, Sibertin-Blanc, Therond, Amblard, Auda, Arcangeli, Balestrat, Charron-Moirez, Gondet, Hong, Lardy, Louail, Mayor, Panzoli, Sauvage, Sanchez-Pérez, Taillandier, Nguyen, Vavasseur and Mazzega2013). Savarimuthu and Cranefield (Reference Savarimuthu and Cranefield2009) draw a quite complete overview of the use of norms in simulation, and categorise simulations w.r.t. their normative mechanisms. In simulations where the norms and their influence on the agents behaviour are of interest, it is important to have a fine-grained model of agents’ cognition that allows agents to explicitly reason about norms (see Section 3.4). For example, some simulations aim to cognitively explain criminal behaviour (Bosse et al., Reference Bosse, Gerritsen and Treur2009) and thus use BDI agents.

In Conte and Castelfranchi (Reference Conte and Castelfranchi1995), the function of social norms is to control aggression in society. The authors deliberately used very simple agents to study the function of norms at a macro-level. These agents can perform some simple routines (move, eat, attack) to survive in an environment with few food resources; they have a strength that is increased by eating and decreased by movement and attack. The authors compared two strategies to control aggressions in this artificial society: in the normative strategy the agents respect a ‘finder-keeper’ norm, while in the utilitarian strategy they attack for food any agent not stronger than themselves. The simulation showed that the normative strategy was much better to reduce the number of aggressions. Staller and Petta (Reference Staller and Petta2001) extend Conte and Castelfranchi (Reference Conte and Castelfranchi1995) but focus on studying the bi-directional relationship between emotions and social norms: social norms influence the triggering of emotions (e.g. feeling ashamed when violating a norm) and their regulation and expression (e.g. one should not laugh at a funeral); and emotions in turn influence normative behaviour (e.g. shame drives people to respect norms, while contempt drives them to apply sanctions to enforce norms). Due to their focus on the micro-level and its interactions with the macro-level, the authors cannot content themselves with the very simple agent model used in Conte and Castelfranchi (Reference Conte and Castelfranchi1995). Instead they use the TABASCO three-layer agent architecture, and a BDI architecture called JAM for the reasoning part of their agents.

BDI architectures seem very adapted to model two mechanisms: the internalisation of norms, and the influence of norms on the decision-making process. Indeed, the explicit representation of norms allows an agent to reason about them and make autonomous decisions regarding their internalisation (instead of automatically accepting all norms); based on its desires and beliefs, the agent can thus choose to violate norms (such as in Castelfranchi et al. (Reference Castelfranchi, Dignum, Jonker and Treur2000).

One of the main critics against BDI architecture regarding norms is that such architectures use a priori existing norms (implemented offline). The emergence and recognition of norms at runtime can be handled with other architectures such as EMIL (Campenn et al., Reference Campenn, Andrighetto, Cecconi and Conte2009); but this problem has only been marginally and very recently tackled in the BDI literature. Criado et al. (Reference Criado, Argente and Botti2010) extend a graded BDI architecture to allow autonomous (and agent-dependent, based on their preferences, contrarily to EMIL) norm acceptance.

To conclude, BDI architectures offers a powerful tool to reason on norms and to integrate them in the decision process, but are more adapted to simulations with initially established norms. Therefore, the emergence of informal social norms in groups of agents would be harder to tackle with a BDI architecture. However, this restriction is not so strong since BDI architectures are still applicable to the large number of simulations where laws and regulations are established by a superior institution.

3.1.7 Decision-making

First of all, we can notice that the decision-making process is often not key in BDI frameworks and architectures. Norling (Reference Norling2009) notices that BDI theory assumes that agents make utility-based rational decisions, optimising the expected utility of the selected action, and that in implementations (e.g. JACK) agents usually choose the first applicable plan regardless of its expected utility. But Norling claims that both strategies are not representative of human decision-making, and proposes to extend BDI with a different decision strategy: recognition-primed decision (RPD). This strategy is typically used by experts who spend most of the time finely analyzing the situation, before making an almost automatic decision based on this situation.

Some works show that the BDI paradigms eases the modelling of agents with a more realistic decision-making process. Lui et al. (Reference Lui, Connell and Vaughan2002) model military commanders in land operations scenarios with a BDI approach, which they claim to provide the agents with the ability to demonstrate human-like decision-making. Noroozian et al. (Reference Noroozian, Hindriks and Jonker2014) use BDI agents to model individual drivers with heterogeneous driving styles for a more realistic traffic model. Indeed, the BDI paradigm makes it easier to design agents with various decision criteria that can react differently to the same stimulus.

In addition, the BDI framework makes it easy to integrate and explore the influence of several cognitive factors on the decision-making process. Bosse et al. (Reference Bosse, Gerritsen and Treur2007b, Reference Bosse, Gerritsen and Treur2009) present a BDI modelling approach to criminal decision-making in a violent psychopath agent. Their model is based on criminology theories that define opportunities for a crime. Desires are generated from biological, cognitive and emotional factors; beliefs in opportunities are generated from the observation of the social environment; their combination leads to intentions and actions to satisfy these desires. The implementation is done with the authors’ LEADSTO language that allows them to combine quantitative (physiological hormones levels) and qualitative (mental attitudes) concepts in a unified model. In a socio-environmental application, Taillandier et al. (Reference Taillandier, Thrond and Gaudou2012) build a model to deal with farmers’ decision-making about cropping plans, farmers behaviours being described with a BDI architecture. To this purpose, they have used a multi-criteria decision-making process, considering the desires as the criteria that drive the decision. Finally, Swarup et al. (Reference Swarup, Eubank and Marathe2014) argue that a BDI model of agents can be fully relevant to understand and model seemingly irrational human decision-making, such as H1N1 vaccination refusal, and its influence on the dynamics of an epidemic.

3.1.8 Summary

To summarise, the BDI model is quite agnostic in terms of the decision-making approach, and many algorithms can be used. Nevertheless, modellers can benefit from the high-level BDI concepts to design a much finer decision-making process than the simple economic maximiser agent. The BDI framework also allows to model individualised, possibly irrational, decision-making, where each agent makes its decision based on its own motivations. This is essential to simulate realistic human societies for various goals in various fields of application and for various goals. Table 1 summarises the importance and benefits of using a BDI architecture for each of the characteristics above.

Table 1

Summary of benefits and drawbacks of BDI architecture agents characteristics (learning will be discussed in Section 3.5.4)

ACL=Agent Communication Languages.

3.3.1 Prediction

Simulations can be used to predict potential outputs resulting from known complex inputs and assumptions about mechanisms.

For example, Gunderson and Brown (Reference Gunderson and Brown2000) use data mining to create a simulation from a large quantity of statistical data available about crimes, and then use it to predict criminal behaviour. The authors are not interested here in the details of the agents cognition, but in the global crime pattern at the scale of a city, and therefore do not need BDI agents (nor have the micro-level data needed to design a BDI model, plus it would add unneeded complexity and scalability problems). In a different field, Baptista et al. (Reference Baptista, Martinho, Lima, Santos and Prendinger2014) model consumers with a realistic underlying BDI agent model, which they claim to be valuable for predicting market trends.

Simulations that can predict the behaviour of the population in response to various policies are also a powerful tool for decision-support, on condition that the underlying model is as faithful as possible in order to lay valid results. A BDI architecture of agents representing humans thus ensures the right level of realism. Indeed, Swarup et al. (Reference Swarup, Eubank and Marathe2014) in their prospective paper claim that the BDI model is very well adapted to model (possibly irrational) human decision-making, such as H1N1 vaccination refusal, when simulating and predicting the dynamics of an epidemic.

Such decision-support systems can be used to assist policy-makers in various fields. For example, Sokolova and Fernández-Caballero (Reference Sokolova and Fernández-Caballero2007) present a decision-support system that analyses pollution indicators and evaluates the impact of exposure on the health of the population, modelled as BDI agents; their system allows ecological policy-makers to explore what-if scenarios, giving them evidence to support decision-making. Shendarkar et al. (Reference Shendarkar, Vasudevan, Lee and Son2008) propose a crowd model with BDI agents to simulate evacuation in case of a terrorist attack, in order to determine where it is more useful to place policemen to improve evacuation.

To conclude, the BDI architecture is a good candidate in simulations aimed at predicting future behaviour, because this requires the behaviour of agents to be as realistic as possible, adaptable to a large number of situations. However, it can only be used if there is enough detailed data available: this will be discussed in Section 3.5.1.

3.3.2 Performance of a task

Axelrod (Reference Axelrod2005) claims that ‘to the extent that artificial intelligence techniques mimic the way humans deal with these tasks, the artificial intelligence method can be thought of as simulation of human perception, decision making, or social interaction’. We therefore sort in this category artificial intelligence research that attempts to simulate human processes into agents to improve their efficiency at these tasks.

For example, the Virtual Humans community aims at creating artificial humans endowed with a wide range of human-like abilities (including face-to-face multimodal interaction, emotions, personality, etc.) (Gratch et al., Reference Gratch, Rickel, Andre, Badler, Cassell and Petajan2002; Swartout et al., Reference Swartout, Gratch, Hill, Hovy, Marsella, Rickel and Traum2006). Agents endowed with good (human-like) communication skills are useful as interface agents or as virtual companions (Wilks, Reference Wilks2010).

When virtual humans aim at a more realistic or complex behaviour, they usually rely on a BDI architecture, possibly extended with additional required features. For example, BDI agents with human-like negotiation skills have been used as mentors in role-playing games to train humans to improve their own negotiation skills in stressful situations (Traum et al., Reference Traum, Rickel, Gratch and Marsella2003). Another BDI agent endowed with emotions could express realistic emotions or to detect the user’s ones in Ambient Intelligence scenarios (Adam et al., Reference Adam, Gaudou, Longin and Lorini2011). Emotions and associated coping strategies were also used as human-inspired heuristics in a BDI reasoner for efficient robot (Adam & Lorini, Reference Adam and Lorini2014).

BDI agents with human-like emotional abilities have also been shown to perform better teamwork. Nair et al. (Reference Nair, Tambe and Marsella2005) use a BDI model of coordination, combined with POMDP to handle uncertainty. In pure agents teams, their BDI emotional agents have a more realistic behaviour, since humans are known to use emotions for goal prioritisation and action selection. In mixed humans-agents teams, they are better team-mates: awareness of their human partners’ emotions improves communication with them, and expression of their emotions contributes to building a relationship with them.

Peinado et al. (Reference Peinado, Cavazza and Pizzi2008) present BDI as a ‘promising tool for modelling sophisticated characters in Interactive Storytelling’. They mention in particular that it allows to create richer characters with their own beliefs and motivations causing their behaviour

Karim and Heinze (Reference Karim and Heinze2005) use BDI agents to design an autonomous pilot for military unmanned aerial vehicles (UAV); they find that the BDI paradigm provides a more modular design, a high level of abstraction, and improves scalability by avoiding hard-coded coordination.

So BDI agents seem to provide a good paradigm to realistically model human behaviour in agents aimed either at behaving in a believable, human-like way, or at performing tasks more efficiently by reproducing the human way of performing them. Indeed BDI provides a high-level description of such behaviour, so it is the perfect level of abstraction to efficiently capture psychological theories of human behaviours. Such agents endowed with human-like abilities find a very wide variety of applications, from virtual companions or storytellers to military UAV pilots.

3.3.3 Training

These simulations aim to ‘train people by providing a reasonably accurate and dynamic interactive representation of a given environment’ (Axelrod, Reference Axelrod2005). Axelrod then notices that simulations having such a goal need accuracy and realism rather than simplicity. Indeed lots of training simulations do use complex BDI agents, often extended with additional abilities such as emotions or negotiation skills.

A lot of works have used BDI agents in serious games for training, because they provide a human-like decision-making process described in terms of folk psychology. Baptista et al. (Reference Baptista, Martinho, Lima, Santos and Prendinger2014) thus model consumers with a realistic underlying BDI agent model, and claim it can be useful for training management skills of business students. Another application is the use of virtual humans as virtual patients to train medical professionals. Rizzo et al. (Reference Rizzo, Kenny and Parsons2011) have used a virtual human with a complex BDI architecture to create different virtual patients that provide realistic representations of mental problems to be diagnosed or discussed. Such virtual patients are useful to improve doctors’ interview and diagnostic skills for a difficult subject through practice.

The same virtual human has also been used in a variety of other simulations for the military. For example, it has been used in a war scenario, the Mission Rehearsal Exercise, to train military task-negotiation (Traum et al., Reference Traum, Rickel, Gratch and Marsella2003) and decision-making (Gratch & Marsella, Reference Gratch and Marsella2004) skills. Traum then enriched it with conversation strategies (Traum et al., Reference Traum, Swartout, Marsella and Gratch2005) to create a virtual negotiator that can play the role of a mentor to train military negotiation skills in crisis situations; the trainee learns principles of negotiation by interacting with the agent using them to play the opposing party. Many other military applications can also been found. Mcilroy and Heinze (Reference Mcilroy and Heinze1996) developed the Smart Whole AiR Mission Model that splits the physics of combat and pilot reasoning into two separate entities which are modelled, validated and implemented in completely different ways; fighter pilots have a BDI architecture based on the distributed multi-agent reasoning system (dMARS) (D’Inverno et al., Reference D’Inverno, Luck, Georgeff, Kinny and Wooldridge2004) realisation of the Procedural Reasoning System. Lui et al. (Reference Lui, Connell and Vaughan2002) use Jack Teams, a BDI team modelling framework, to model military commanders in land operations scenarios.

BDI agents have also been used because of their adaptability to a changing environment. Oulhaci et al. (Reference Oulhaci, Tranvouez, Fournier and Espinasse2013) designed a serious game for training in the field of crisis management, where non-played characters (NPC) have a complex behaviour, either behaving as expected or intentionally erroneously. These NPC are developed as BDI agents so that they can adapt to events of the world and to social interactions with and actions of other players (in particular the learner). Cho et al. (Reference Cho, Iketani, Kikuchi, Nishimura, Hayashi and Hattori2008) used BDI agents to simulate crowds in virtual worlds, claiming that BDI increases their adaptability to the dynamic environment, and therefore their realism. They present a prototype game with NPC extinguishing a forest fire, implemented with the SourceEngine three-dimensional (3D) game engine.

To conclude, simulations for training require a realistic immersive environment to be efficient. In particular, the behaviour of autonomous agents populating the virtual world (NPC) must be human-like, making BDI agents a very good choice, for their descriptivity of the human-like decision-making process, and their adaptability to a dynamic environment.

3.3.4 Entertainment

This goal is quite similar to training but the virtual worlds can be imaginary ones. There has been work on integrating BDI agents into games in order to provide the players with a richer, more enjoyable experience.

A number of works are dedicated to the integration of autonomous agents (bots) in Unreal Tournament. Small (Reference Small2008) uses a simple rule-based agent called Steve, that can quickly decide its action (e.g. move to healing source) based on high-level conditions (e.g. weak energy) in a dynamic environment. To be able to compete with human players, Steve mimicks their learning process through an evolutionary algorithm: its strategies are evolved (and improved) over time. However, it was still unable to consistently beat human challengers. Hindriks et al. (Reference Hindriks, van Riemsdijk, Behrens, Korstanje, Kraaijenbrink, Pasman and de Rijk2010) wanted to find alternatives to such reactive bots. They argue that using BDI agents instead of scripts or finite-state machines (often used in real-time games) might result in more human-like and believable behaviour, thus making the game more realistic. They add that when one gets data from observing actual game players, it is easier to specify it in terms of beliefs, desires and intentions and import it into BDI agents, than to translate it into finite-state machines. They thus tried to interface the GOAL agent programming language with Unreal Tournament 2004. Such an interface raises two main issues: finding the right level of abstraction of reasoning (the BDI agent should not have to deal with low-level details, while keeping enough control), and avoiding cognitive overload to achieve real-time responsiveness (the agent should not receive too many percepts, but should have enough information, even if possibly incomplete and uncertain, to make decisions). One of their motivations was to test agent programming platforms, in order to eventually make them more effectively applicable. They indeed noticed that BDI agent technology is not scalable yet, and they observed bad performance when increasing the number of agents.

Another try at integrating BDI agents into Unreal Tournament is described in Urlings et al. (Reference Urlings, Sioutis, Tweedale, Ichalkaranje and Jain2006). The authors are actually interested in humans-agents teams in the area of unmanned vehicles. They propose a framework to model these and then use the Unreal Tournament game as a real-time simulation environment to demonstrate this framework. Their agents are implemented with a BDI architecture using JACK. BDI agents have also been used in other games: Norling (Reference Norling2003) describes a BDI Quake player; the main character of the Black and White game (Molyneux, Reference Molyneux2001) is also a BDI-based agent. In another attempt to make video games more immersive and realistic (and not only graphically so), Palazzo et al. (Reference Palazzo, Dolcini, Claudi, Biancucci, Sernani, Ippoliti, Salladini and Dragoni2013) provide the Spyke3D framework, using BDI agents to facilitate the creation of more human-like NPC with plausible logical behaviours.

The works described above all concern video games, but BDI can also be used to model characters in interactive stories and endow them with a richer causal behaviour based on their beliefs and motivations (Peinado et al., Reference Peinado, Cavazza and Pizzi2008).

So BDI agents are an excellent tool to create believable characters that make games or stories more realistic and immersive. Unfortunately, there are performance issues when the number of agents increases, which curbs their wider use in games so far, when designers usually prefer to improve graphics and rely and simple but fast scripted behaviours. We will discuss solutions to the scalability problem in Section 3.5.2.

3.3.5 Education

According to Axelrod, ‘the main use of simulation in education is to allow the users to learn relationships and principles for themselves’ (Reference Axelrod2005). So this goal is similar to training (and entertainment), but the virtual world does not need to be as accurate (Axelrod says they ‘need not be rich enough to suggest a complete real or imaginary world’), as long as it lets the student experiment and learn by trial and error. For example, in the SmartEnergy project, Kashif et al. (Reference Kashif, Le, Dugdale and Ploix2011) propose a serious game aimed at raising awareness about energy saving; the realism is not key here as the main requirement is to give the users an insight of the effects of their actions.

Nevertheless to improve the users’ learning, it is often useful to provide them with an artificial tutor inhabiting this virtual world, and able to explain to them why they did wrong or right. Therefore, Harbers et al. (Reference Harbers, van den Bosch and Meyer2010, Reference Harbers2011) use BDI agents that can provide such explanations in terms of intuitive concepts explicitly represented in the BDI architecture: their goals and beliefs (see Section 3.1.2). For example, Elliott et al. (Reference Elliott, Rickel and Lester1999) study the integration of emotions in a virtual tutor called Steve that inhabits a 3D world and teaches students to operate a high-pressure air compressor; Steve has also been used in a training simulation with an army peacekeeping scenario to prepare officers to deal with difficult situations (Rickel et al., Reference Rickel, Gratch, Hill, Marsella and Swartout2001).

To conclude, in education-oriented simulations realism of the virtual world and of its population is not key, so it is not necessary to use a BDI architecture for all agents. Nevertheless, the BDI architecture is really useful for some specific agents that need to be able to explain their behaviour or the user’s errors, such as some NPC or an artificial tutor, in order to improve learning.

3.3.6 Proof

Another possible purpose of simulations identified by Axelrod is to provide an existence proof. For instance, Conway’s Game of Life (Gardner, Reference Gardner1970) is a game based on a set of simple rules governing the birth and death of cells, which can be used to prove or disprove conjectures such as ‘a finite population cannot grow infinitely large’.

The literature provides few examples of agent-based models aimed at existence proofs. One famous example, however, is Schelling’s (Reference Schelling1971) agent-based segregation model. Very simple rules govern the agents’ movement to other neighbourhoods based on their level of happiness with their neighbours. Just a small desire to live near people with similar traits can quickly lead to large clusters of homogeneous neighbourhoods. This model is considered to provide an existence proof that segregation does not require the people involved to be highly intolerant, but only mildly so.

The line between exploration and proof is difficult to draw. Axtell et al. (Reference Axtell, Epstein, Dean, Gumerman, Swedlund, Harburger, Chakravarty, Hammond, Parker and Parker2002) provide a multi-agent model of the exodus of the Kayenta Anasazi (modelled as relatively simple reactive agents) from Long House Valley, as a means to explore its possible causes. Whilst it provides a strong suggestion that the inability of the land to sustain the population was not the only cause of the exodus (by showing that some subset of the population could have been sustained) it falls short of being considered an existence proof that socio-cultural factors had some influence. It is not clear whether this is because this relatively simple model is in fact too complex, or indeed too basic to support such reasoning.

Sun and Naveh (Reference Sun and Naveh2007) argue that socio-cognitive simulation is essential to prove functionalist hypothesis, that justify social phenomenons by the function they have in society, that is, that explain a cause by its effect. Such hypothesis are usually hard to verify, but social simulation allows to verify or prove them through experimentation. However, the rudimentary cognition assumed in agents used in these simulations does reduce their realism and hence their applicability, and also prevents from proving links between the micro- (cognitive) and macro- (social) levels of the simulation. The authors therefore argue for the use of a specific cognitive architecture called CLARION; we claim that BDI agents would achieve the same purpose (as discussed above in Section 2.2).

To conclude, depending on the hypotheses that the simulation aims at proving, the complexity required from the agents involved is not the same. Hypotheses about the survival of a cell population, or the geographical movements of a human population, can be proved with very simple agents; in contrast, more complex hypotheses about the links between individual cognition and social organisation do require more complex agent models, with BDI being one option.

3.3.7 Discovery

The last purpose of simulations identified in Axelrod’s classification is knowledge discovery. Axelrod claims that social scientists have successfully ‘discover[ed] important relationships and principles from very simple models’ and argues that ‘the simpler the model, the easier it may be to discover and understand the subtle effects of its hypothesised mechanisms’ (Reference Axelrod2005).

Sun has tried to discover relationships and interactions between the micro-level (cognition of agents) and macro-level (dynamics of society) (Sun, Reference Sun2004), for example, in the simulation of social groups depending on the food management strategy (Sun & Naveh, Reference Sun and Naveh2007). He obviously needed a fine-grained model of agents, and used his CLARION cognitive architecture. Heinze et al. (Reference Heinze, Goss, Josefsson, Bennett, Waugh, Lloyd, Murray and Oldfield2001) model airborne early warning and control crew with BDI agents in a simulation used for the exploration, evaluation and development of tactics and procedures, in order to advise the Australian Defence Force in the conduct of maritime surveillance and patrol.

ABMS have also been used recently as a tool for public policy analysis, as it provides a means to discover possible sources of policy resistance. Policy resistance is the tendency for policy interventions to be defeated by unpredicted effects. For example, making public fuel-efficient transportation more affordable to the general public may reduce carbon emissions around the city, but the money saved due to reduced cost of local road travel might be spent on leisure, leading to an increase in air travel, resulting in no overall reduction in carbon emissions. Such effects are generally not intuited because the system is constantly changing, adaptive and evolving, with tightly coupled actors and nonlinear effects (Sterman, Reference Sterman2006)—factors which are well addressed by agent-based modelling (ABM).

For example, the Electricity Market Complex Adaptive System multi-agent simulation has been used to explore the restructuring and deregulation of the electric power market in Illinois (Cirillo & et al., Reference Cirillo, Thimmapuram, Veselka, Koritarov, Conzelmann, Macal, Boyd, North, Overbye and Cheng2006), and provided information on this policy issue ‘that would otherwise have not been available using any other modelling approach’ (Macal & North, Reference Macal and North2005). There are also many examples of agent-based simulations being used to study climate policy (e.g. Moss et al., Reference Moss, Pahl-Wostl and Downing2001; Posada et al., Reference Posada, Hernaandex and Laopez-Paredes2008).

Whilst policy development is an emerging field of application for ABMS, the tight link between policy impact and human behaviour strongly suggests a need for realistic, rich models of the latter, as supported by the BDI paradigm. In the context of using agent simulation for exploring economic policy, Farmer and Foley (Reference Farmer and Foley2009) suggest that ‘the major challenge lies in specifying how the agents behave and, in particular, in choosing the rules they use to make decisions’.

3.5.1 Influence of the availability of data

To build an model as realistic and useful as possible, the data available about the reference system has a huge importance, and can even influence the choice of the agent architecture. Indeed, if the only data available describes the behaviour of social groups (e.g. family statistics) the modeller does not have enough precise information to give a very rich architecture to the agents representing the individual members of these groups (Gil-Quijano et al., Reference Gilbert and Troitzsch2007). For instance, Gunderson and Brown (Reference Gunderson and Brown2000) use data mining to deal with the large quantity of statistical data available about crimes; this lets them model the global crime pattern at the scale of a city, but not the details of the agents cognition. Therefore, when the goal of the simulation (as discussed above in Section 3.3) requires realism, in particular when the simulation is used for prediction, a great deal of attention has to be paid to the collection and processing of precise data, and thus the use of efficient and dedicated techniques and tools.

It is generally much easier to collect objective and measurable data (such as age, income, etc.), which is sufficient to build a simple reactive agent model, than to gather subjective information explaining behaviour in terms of (not observable) mental states. But since this kind of subjective information is needed to build rich agent models such as BDI agents, another approach is to use participatory modelling: put a human in a virtual world to capture his behaviour, and then reproduce it in the agents of the simulation (Shendarkar et al., Reference Shendarkar, Vasudevan, Lee and Son2006, Reference Shendarkar, Vasudevan, Lee and Son2008). For instance, Silverman collected attributes values governing BDI characteristics by placing human beings into a Virtual Reality environment, to build a BDI model of the crowd behaviour after a terrorist attack (Shendarkar et al., Reference Shendarkar, Vasudevan, Lee and Son2008). The MAELIA project (Gaudou et al., Reference Gaudou, Sibertin-Blanc, Therond, Amblard, Auda, Arcangeli, Balestrat, Charron-Moirez, Gondet, Hong, Lardy, Louail, Mayor, Panzoli, Sauvage, Sanchez-Pérez, Taillandier, Nguyen, Vavasseur and Mazzega2013) Maelia investigates policies for water resource management; as the water used in the studied basin is mainly influenced by farming practices, the modellers interviewed farmers about how they made their decisions regarding cropping.

So the BDI model needs precise data about individual behaviours, which is harder to collect. But the BDI model also offers advantages, in particular its descriptivity. Indeed, Edmonds and Moss (Reference Edmonds and Moss2005) claim that descriptive models (in particular agent-based) can help modellers to exploit more of the available data, in particular qualitative information such as anecdotal or commonsense evidence, because of the straightforward correspondence with concepts used in the model; it thus combines well with participatory modelling. Norling also notices that it is easier for experts being modelled to provide information in an intuitive language: ‘people have a tendency to explain their actions in terms of their intentions, which in turn are explained in terms of their goals and beliefs’ (Reference Norling2004). She thus argues for the use of a descriptive approach such as BDI agents because ‘the fact that model builder and subject being modelled are referring to the same concepts does simplify matters’.

Another approach when fine-grained data are not available is to rely on psychological theories of human behaviour to describe expected behaviour, and to use them to design the agent model. For instance, data about emotions is rarely available from field studies or statistics, so theories of emotions can help. In that case, BDI has the advantage of providing the right concepts for capturing these theories and formalise them in terms of mental attitudes. When there is neither data nor theory available, that is, when the decision-making process is unknown, BDI is not an appropriate choice, as also noted in Wolfe et al. (Reference Wolfe, Sierhuis and Jarvis2008).

To conclude, BDI agent models might at first seem harder to design because of the lack of fine-grained statistical data, and the unsuitability of such quantitative data to design a qualitative model. But at the same time the BDI formalism allows modellers to exploit different types of data, such as observations from participatory modelling, or psychological theories, because of the straightforward matching between BDI concepts and the way in which people instinctively explain their behaviour.

3.5.2 Computational scalability

Another important criticism against the use of BDI agents in simulation is the computational weight of this approach. As it is already time-consuming to simulate only one BDI agent, with many reasoning components, it is intuitively hard to imagine that such a tool could be used for large-scale simulations with thousands of agents. Some works have integrated BDI agents in real-time environments such as games (see Section 3.3.3) and they tend to confirm this intuition. For example, in Hindriks et al. (Reference Hindriks, van Riemsdijk, Behrens, Korstanje, Kraaijenbrink, Pasman and de Rijk2010), the authors develop bots using a BDI architecture in the game Unreal Tournament, and conclude that there are indeed problems of scalability and performance when the number of agents increases.

However, this technical drawback is bound to decrease and eventually disappear, as computational power of individual computers increases exponentially, and grid computing also allows the distribution of simulations on many computers to accelerate their execution even more. For instance, Zhao and Son (Reference Zhao and Son2007) have built a BDI agent in a distributed environment, to partially replace the decision-making process of human beings in an automated system; this simulation only includes one agent so far but could technically be extended to many more agents. Nevertheless, this appears to be a shift from one research issue to another, because the distribution of a simulator over a network is a very difficult problem too, that is far from being solved.

So scalability still limits the use of BDI agents in large-scale simulations for now, but the computational power of present personal computers already allows the use of BDI agents in smaller scale simulations. For larger scale simulations, when distribution is too complex or limited by a lack of hardware resources, the modeller can resort to an optimised platform, or optimise the implementation of their BDI engine. For instance, Cho et al. (Reference Cho, Iketani, Kikuchi, Nishimura, Hayashi and Hattori2008) implemented a crowd simulator with BDI agents, and evaluated its performance in terms of computing time. They showed that the processing time was roughly proportionate to the number of agents, and mostly dedicated to the belief update process applied every time a new belief is added. Their solution to minimise this computing time was therefore to optimise the reasoner by applying belief update rules only when their effect was significant for the current desire.

In addition, building simulations using BDI agents does not imply that all the agents should be cognitive. A wise choice of which agent should or should not be a BDI agent will have a huge impact on the simulation scalability. Wolfe et al. investigate this ‘To BDI, or not to BDI’ dilemma (Reference Wolfe, Sierhuis and Jarvis2008) in the field of aerial traffic management simulations. They mention that the ‘BDI paradigm is an excellent choice when modeling well-understood decision making’, but notice efficiency issues when using BDI agents in large-scale simulations. They suggest that one solution is to choose wisely which agents to model as BDI agents (pilots in their example), and which agents could have simpler architectures (planes, etc.). This mixed population of BDI and not-BDI agents increases the performances of their simulator. Novák et al. (Reference Novák, Komenda, Cap, Voknnek and Pechoucek2013) implemented BDI reasoning capabilities in autonomous military robots teams for surveillance missions, to improve flexibility compared with a simple reactive planner. However, they noted that these agents were heavier and should thus be limited to scenarios where complex deliberation was actually needed.

To conclude, BDI architectures are indeed heavier due to the more complex reasoning processes of the agents, so they have to be used wisely. Many authors have successfully applied them in their simulations, by optimising the computation inside each agent, and by limiting their use to agents and scenarios where they were actually justified, mixing BDI agents with simpler agents.

3.5.3 Limited (conceptual) expressivity

As highlighted in Section 2.3.4, the BDI architecture has the advantage of being easily extensible with more concepts than the three basic ones (beliefs, desires, intentions). Nevertheless, before being integrated in a BDI agent, the properties of any additional concept and its links with the existing concepts (at least beliefs, desires and intentions, but possibly others in an already extended BDI architecture) should have been deeply studied.

DSTO (Australian Defence) programmers have listed a number of features that need to be added to the BDI framework so that developers do not have to explicitly implement them in every model, and so that applications can be less constrained. Norling (Reference Norling2004) quotes, for example, emotion, fatigue, learning or memory. We could also extend this list with group attitudes and social commitments. In particular, in a society of agents, the BDI architecture can efficiently be used to represent an individual reasoning process, but only few formalisms have been developed for the reasoning process of social groups. Actually there is no issue metaphorically using a BDI architecture for an agent representing a group of agents when it is well-defined offline (before the beginning of the simulation). But there are only a few studies on how to attribute mental states to social groups of agents emerging online, during the simulation, when the attitudes of the group should interact with the attitudes of the agents composing it.

Only few theoretical studies have been led about the above concepts. There is thus a huge conceptual lack here. Nevertheless, we stay optimistic and assume that in the next years many researchers will try to tackle these issues. For example, recent works have begun to analyze the links between BDI and emotions (Adam, Reference Adam2007) or social attitudes (Gaudou, Reference Gaudou2008). Such work has to be continued to provide a clear, well-grounded and standardised extended BDI framework.

3.5.4 Learning

One of the major a priori criticisms of the BDI architecture of agents is its inadequacy to support learning mechanisms: ‘one weakness that the BDI agent model shares with other human operator models (agent-based or otherwise) is its inability to support agent learning’ (Urlings et al., Reference Urlings, Sioutis, Tweedale, Ichalkaranje and Jain2006). This is particularly due to the fact that most powerful learning algorithms are dedicated to quantitative learning (learning numerical values, e.g. in neural network, etc.) whereas symbolic learning is a much harder issue (Tables 3).

Table 3

Summary of criticisms against BDI and proposed solutions

Nevertheless, there exist some extensions of the BDI architecture aiming at integrating a learning capability into BDI agents. For example, in the video game Black and White (Molyneux, Reference Molyneux2001), the main character has a BDI architecture extended with a reinforcement learning capability to allow the player to teach their avatar. In another example, Norling (Reference Norling2004) extended BDI with the RPD strategy used by experts of a domain: the agent progressively learns (from its mistakes) to make very fine distinctions between situations, to allow a nearly automatic plan selection based on it. These examples show that in contrast to preconceived ideas, BDI agents can be useful in simulations even when agents need learning capabilities.