3.1 Individual agents

The focus of most multi-agent programming languages and development frameworks has been on programming individual agents (see e.g. Bordini et al., Reference Bordini, Dastani, Dix and El Fallah Seghrouchni2005, Reference Bordini, Dastani, Dix and El Fallah Seghrouchni2009). In these works, multi-agent programs are considered as comprising a set of individual agent programs that are executed concurrently.

An essential characteristic of individual agents is their autonomy. An agent is called autonomous if it has the ability to decide and perform actions at each moment of time in order to achieve its objectives (e.g. Woolridge, Reference Woolridge2002). Without getting into the exact nature of autonomy, we consider an agent as autonomous if it has a decision-making component that governs its decisions based on its informational (e.g. belief, probability distribution, knowledge), motivational (e.g. desire, utility, preference), and deliberational (e.g. intention, plan, commitment) attitudes. One can argue that any computational system that interacts with other systems (e.g. other agents or software components) can be seen as autonomous, at least from an external point of view. However, the development of an autonomous software agent can also be considered from an internal point of view. Such a view requires an explicit decision-making component that can be specified, designed, implemented in terms of informational, motivational, and deliberational attitudes. The decision-making component should allow a programmer to implement different issues related to an agent’s decisions such as decision strategies, resolving decision conflicts, and rationality of decisions. In this sense, programming languages that support the implementation of autonomous agents should provide programming constructs to support the implementation of decision concepts and mechanisms.

Different decision models can be used for developing the decision component of autonomous agents, for example, Partially Observable Markov Decision Process model (POMDP) (see e.g. Tasaki et al., Reference Tasaki, Yabu, Iwanari, Yokoo, Tambe, Marecki and Varakantham2008), Belief-Desire-Intention model (BDI) (see e.g. Cohen & Levesque, Reference Cohen and Levesque1990; Rao & Georgeff, Reference Rao and Georgeff1991), or a combination of both (see e.g. Nair & Tambe, Reference Nair and Tambe2005). POMDP is a quantitative framework that can be used to model a sequential decision process in terms of actions, states, transition probability, observation probability, and reward function. In order to determine an agent’s decisions, its corresponding POMDP (the agent’s decision model) should be solved. However, solving POMDP’s are in general computationally intractable such that approximate methods are often proposed to solve POMDP’s. Moreover, POMDPs are not suitable to model agents that have complex goals and need to interact with dynamically changing environments (see e.g. Rens et al., Reference Rens, Ferrein and van der Poel2009). These problems make POMDP less plausible to be integrated in programming languages and development frameworks for individual agents.

In contrast, the BDI model can be seen as a qualitative decision model that explains an agent’s rational decision in terms of the agent’s information about the current state of the world (Belief), the states the agent wants to achieve (Desire), and its commitments to already made choices (Intention). The BDI model has proven to be an efficient model for reactive planning and for the agents that have complex goals interacting with highly dynamic environments (see e.g. Bratman et al., Reference Bratman, Israel and Pollack1988). Some existing BDI-based agent programming languages provide constructs to implement an agent’s beliefs, goals, and conditional plans. The conditions that are assigned to plans are specified in terms of beliefs and goals such that a plan can be decided/selected if its belief and goal conditions are satisfied by the agent’s beliefs and goals, respectively, and moreover the plan is not in conflict with the existing plans. Plan conflict can be defined in various ways such as realising inconsistent states or being in conflict with respect to the use of resources. In some BDI-based programming languages (e.g. Dastani, Reference Dastani2008) a plan is selected if it is not already generated to achieve the same goal. The reasoning engine of the BDI agents often involves a process that continuously decides a plan to execute.

Another characteristic of an individual agent is its reactive behaviour (see e.g. Muller, Reference Miiller1996). The implementation of reactive agents requires an event handling mechanism that generates reactions to the received events. There are many types of events such as messages that are received from other agents, information that are originated from the environment, and the information about the internal working of agents (e.g. the failure of a plan or the updates of beliefs and goals). Programming languages and development frameworks that support the implementation of reactive agents provide constructs to implement conditional plans where the plan’s conditions are defined in terms of events. The reasoning engine of reactive agents continuously checks if plans can be selected based on the received events. It should be noted that there is an essential difference between events and goals. In principle, an event causes the generation of a plan and, as soon as the plan is generated, the event is deleted and considered as being processed. Goals are similar to events in the sense that they cause the generation of plans. However, and in contrast to events, goals are not dropped after they have caused the generation of plans. An achieve goal is, for example, dropped if the state denoted by it is achieved, that is, if the agent believes that the state denoted by the goal is achieved. The relation between beliefs and goals is an essential characteristic of BDI agents, which is formulated by means of rationality axioms in the BDI logics (see e.g. Cohen & Levesque, Reference Cohen and Levesque1990; Rao & Georgeff, Reference Rao and Georgeff1995). It should also be noted that autonomy and reactive behaviour are two different characteristics and that agents can be both autonomous and reactive, be autonomous without being reactive, or vice versa.

3.2 Multi-agent environment

Soon after the emergence of the first generation of the execution platforms for agent-oriented programming languages and development frameworks, the need for the implementation of shared environments with which software agents can interact became apparent. This need was manifested by applications in which software agents have to interact with non-agentified software components, for example, shared resources or services. An immediate solution was to model an environment as an external software component with which individual agents interact. Such a software component can be an implementation of resources and services, or an interface to other (software of natural) systems that are external to the developing multi-agent system. In most BDI-based multi-agent programming languages (e.g. Pokahr et al., Reference Pokahr, Braubach and Lamersdorf2005; Bordini et al., Reference Bordini, Huubner and Wooldridge2007; Dastani, Reference Dastani2008; Hindriks, Reference Hindriks2009) an environment is simply a software component that is implemented in the same programming language as that of the interpreter of the agent-oriented programming language (e.g. Java or C++). In other programming frameworks (e.g. Bellifemine et al., Reference Bellifemine, Bergenti, Caire and Poggi2005; Winikoff, Reference Winikoff2005) an environment is a software component implemented in Java. In both cases, the state of the software component is considered as the state of the environment, while the methods that allow the interaction with the software component are used to implement the effect of the actions that agents could perform in the environment. In fact, the repertoire of actions that an agent can perform in an environment is determined by the methods of the corresponding software component. In this view, a software agent decides which method to call (i.e. which action to select and perform) and the environment program determines the effect of the action.

One of the first overview papers in the field of multi-agent system environments emphasised that the concept of environment was originally used with different meanings causing confusion about the exact nature of this abstraction (see e.g. Weyns et al., Reference Weyns, Van Dyke Parunak, Michel, Holvoet and Ferber2005; Van Dyke Parunak & Weyns, Reference Van Dyke Parunak and Weyns2007). As argued in this overview paper, some researchers consider a multi-agent system environment as the run-time environment and as equivalent of infrastructures such as message transport system and other infrastructural tools, for example, brokers and management tools, while other researchers consider a multi-agent system environment as the embedding environment of the multi-agent system consisting of entities external to the multi-agent software system. Yet, other researchers consider a multi-agent environment as a component that encapsulates resources, services, and objects. Existing agent programming languages (e.g. Pokahr et al., Reference Pokahr, Braubach and Lamersdorf2005; Bordini et al., Reference Bordini, Huubner and Wooldridge2007; Dastani, Reference Dastani2008; Hindriks, Reference Hindriks2009) consider an environment as a first-class abstraction in multi-agent system that encapsulates resources and services and has its own state and processes.

It should be emphasised that from the programming point of view, it does not make sense to consider other notions of environment. For example, the notion of environment as an entity external to the multi-agent system refers to something to be assumed by the system developer and not an entity to be programmed. In addition, the notion of environment as an infrastructure denotes a set of tools and facilities that a multi-agent system developer can assume as given and not something that the system developer has to program for each application separately. Note also that the concept of multi-agent environment as encapsulating resources and services is different from agent-based simulation environment. An agent-based simulation environment can be used to model, execute, and analyse agent-based simulations. An agent-based simulation environment can be compared with an integrated development environment for multi-agent systems. In particular, agent-based simulation environments can be seen as special cases of integrated development environments where the focus is on tools for analysing simulation behaviours rather than general purpose tools for the development of multi-agent systems.

In the same overview paper, a multi-agent system environment is claimed to be used for various purposes, for example, to facilitate and coordinate agent’s interactions by means of exchanging information through it (blackboard architectures and tuple spaces)Footnote ¹, or to provide agents the sense and act abilities in order to observe and modify the environment’s state, respectively. For example, an environment can provide artifacts or services to allow agents to manage their coordination or to exchange information. An environment can also provide various sense and act modalities such as blocking and non-blocking sense operations, event broadcasting, event subscription mechanisms, and synchronous or asynchronous actions.

The implementation of environments therefore requires dedicated programming languages and development frameworks that allow direct and effective implementations of its related concepts and abstractions such as resources, services, and objects. Like the development in agent-oriented programming languages, one may expect typical architectures for multi-agent environments. Such architectures would suggest specific concepts, concerns, or components that often need to be implemented when developing a multi-agent system environment. In particular, a dedicated environment programming language or development framework should provide programming constructs to implement resources, services, artifacts, processes, several sense, and action types and mechanisms. The A&A model (see e.g. Omicini, Reference Omicini2007) has been proposed as a generic approach for modelling environments. In the A&A model, an application is composed of agents as well as the so-called artifacts. An implementation of the A&A model is available in form of the distributed architecture and middleware infrastructure Cartago (see e.g. Ricci et al., Reference Ricci, Viroli and Omicini2006). Such an environment architecture consists of a dynamic set of artifacts, each of which encapsulates resources, services, or objects designed by the environment developer.

3.3 Multi-agent organisation

The overall objectives of a multi-agent system depend on the behaviours of its participating individual agents that pursue their own objectives. The objectives of the participating individual agents may be different from, and even be in conflict with, the overall objectives of the multi-agent system. One way to guarantee the overall objectives of a multi-agent system is to control and coordinate the behaviour of its individual agents and their interactions. This can be done either endogenously by integrating the organisation mechanism within the agents themselves, or exogenously by designing the organisation mechanism outside the agents, or a combination of both. An endogenous organisation implies that agents are internally designed and implemented to follow, for example, specific interaction protocols, norms, or organisational rules. In an exogenous approach agents are coordinated by means of external components that control the agent’s actions according to some interaction protocols, norms, or organisational rules. Generally speaking, endogenous coordination mechanisms can be used for the development of closed multi-agent systems where the set of participating agents is predefined by the system designer. Exogenous coordination mechanisms can be used for open multi-agent systems where software agents can join and leave the system at run time. It should be noted that an exogenous organisation can also be effective for the development of closed multi-agent systems as such an approach supports the separation of concerns and encapsulation principles.

There have been various proposals for regulating and organising the behaviours of individual agents. Some of these proposals advocate the use of coordination artifacts that are specified in terms of low-level coordination concepts such as synchronisation (see e.g. Arbab et al., Reference Arbab, Astefanoaei, de Boer, Dastani, Meyer and Tinnermeier2009). Other approaches are motivated by organisational models, normative systems, or electronic institutions (e.g. Jones & Sergot, Reference Jones and Sergot1993; Esteva et al., Reference Esteva, de la Cruz and Sierra2002, Reference Esteva, Rodriguez-Aguilar, Rosell and Arcos2004; Grossi, Reference Grossi2007; Dastani et al., Reference Dastani, Grossi, Meyer and Tinnemeier2009b; Hübner et al., Reference Hübner, Boissier, Kitio and Ricci2010). In these approaches, the behaviours of individual agents are regulated by means of norms and organisational rules that are either used by individual agents to decide how to behave, or being enforced or regimented through monitoring and sanctioning mechanisms. In these approaches, the social and normative perspective is conceived as a way to make the development and maintenance of multi-agent systems easier to manage. A plethora of social concepts (e.g. roles, groups, social structures, organisations, institutions, norms) has been introduced in multi-agent system methodologies such as Gaia (Zambonelli et al., Reference Zambonelli, Jennings and Wooldridge2003), models such as OperA (Dignum, Reference Dignum2004), specification and modelling languages such as ${\cal S}{\minus}{\scr M}{\rm OISE}^{{\plus}} $ (Hübner et al., Reference Hübner, Sichman and Boissier2006) and ISLANDER (Esteva et al., Reference Esteva, de la Cruz and Sierra2002), and computational frameworks such as AMELI (Esteva et al., Reference Esteva, Rodriguez-Aguilar, Rosell and Arcos2004).

The implementation of organisations requires programming languages and development frameworks that provides programming constructs to implement social and organisational concepts and abstractions. In particular, the implementation of endogeneous mechanisms implies that the agent programming languages provide constructs to allow the representation and reasoning about norms, sanctions, and organisational rules. Such constructs should allow multi-agent programmers to implement agents that make their decisions not only based on their individual goals and beliefs, but also based on the existing interaction protocols, norms, sanctions, and other organisational rules. The idea is that individual agents can be implemented in terms of cognitive and social abstractions such that their behaviours are determined based on reasoning about such abstractions. The implementation of exogenous mechanisms requires abilities to monitor and control the behaviours of individual agents. The idea is to have an external organisation software component that is able to monitor and control the behaviour of individual agents. The question is what should be monitored and how the agent’s behaviours can be influenced. As the internals of individual agents cannot be assumed in general, their external behaviours (i.e. communication and interaction with the environment) are the only controllable entities. The organisation software component can thus observe agent’s external behaviour and determine what needs to be done. For example, if the organisation specification disallows certain agents to interact, then the organisation software should be able to block or respond to such interactions. This suggests that the agent’s actions (e.g. communication, environment actions including sense actions) should be processed and managed through the external organisation component, that is, the organisation component intermediates the interaction between agents as well as the interaction between agents and the environment.

4.1 Programming frameworks for individual agents

One of the earliest agent-oriented programming languages is AGENT-0 proposed by Shoham (Reference Shoham1993). In his seminal paper, Shoham proposes to implement agents in terms of mental components such as beliefs, commitments, capabilities, and actions. An agent program in AGENT-0 consists of an initial belief base, a set of capabilities, a set of commitment rules, together with a repertoire of private actions. Agents can perform different types of actions such as communication, private, conditional, and unconditional actions. Agents enter into new commitments by means of commitment rules. A commitment rule consists of conditions on an agent’s mental state and the incoming messages. The application of a commitment rule generates a commitment consisting of an action together with the agent identifier towards whom the commitment is made. In fact, the commitments define the actions that an agent have to perform. The execution of an agent is a continuous cyclic process. At each cycle, the received messages are processed, commitments are generated, and actions are performed. AGENT-0 is undoubtedly one of the first attempts to develop an agent programming language that supports the implementation of autonomous agents, that is, agents that decide actions based on their mental states. However, as indicated in the discussion section of this seminal paper, the state of an AGENT-0 agent lacks motivational attitudes such as utility, desire, goal, or preference such that an agent’s decisions are based only on events and messages rather than the agent’s motivational attitude.

As the introduction of AGENT-0 various agent-oriented programming languages have been proposed that extend AGENT-0 with a larger repertoire of agent concepts and abstractions. The aim of these programming languages is to support the implementation of multi-agent systems, although most of them do not support the implementation of abstractions concerning multi-agent environment and organisation. Some of these agent-oriented programming languages have an imperative programming style as they extend Java with agent concepts and abstractions, some languages have a declarative programming style as they extend logic programming languages, and yet other programming languages combine both imperative and declarative styles by integrating, for example, Java and Prolog. The programming languages that are based on Java have no explicit formal semantics. In the following, we give a brief overview of some of these programming languages.

4.2 Programming frameworks for multi-agent organisations

In the literature on multi-agent systems, there have been many proposals for specification languages and logics to specify and reason about normative multi-agent systems, virtual organisations, and electronic institutions (see e.g. Jones & Sergot, Reference Jones and Sergot1993; Prakken & Sergot, Reference Prakken and Sergot1996; Ågotnes et al., Reference Ågotnes, van der Hoek and Wooldridge2008; Boella & van der Torre, Reference Boella and van der Torre2008). How to develop, program, and execute such normative systems was one of the central themes that were discussed and promoted during the AgentLink technical fora on programming multi-agent systems (see Dastani & Gomez-Sanz, Reference Dastani and Gomez-Sanz2005 and Dastani & Gomez-Reference Dastani and Gomez-SanzSanz, Reference Dastani and Gomez-Sanz2006 for the general report of these technical fora). In this section, we discuss some proposals for specifying and implementing normative multi-agent systems.

One of the early modelling languages for specifying institutions in terms of institutional rules and norms is ISLANDER proposed by Esteva et al. (Reference Esteva, de la Cruz and Sierra2002). In order to interpret institution specifications and execute them, an execution platform, called AMELI, has been developed by Esteva et al. (Reference Esteva, Rodriguez-Aguilar, Rosell and Arcos2004). This platform implements an infrastructure that, on the one hand, facilitates agent participation within the institutional environment and supports their communication and, on the other, enforces the institutional rules and norms as specified in the institutional specification. The key aspect of ISLANDER/AMELI is that norms can never be violated by the agents. In other words, systems programmed via ISLANDER/AMELI make only use of regimentation in order to guarantee the norms to be actually followed. The norms in Esteva et al. (Reference Esteva, Rodriguez-Aguilar, Rosell and Arcos2004), Garcia-Camino et al. (Reference Garcia-Camino, Noriega and Rodriguez-Aguilar2005), Silva (Reference Silva2008) are related to actions that the agents should or should not perform. In these approaches, the issue of expressing more high-level norms concerning a state of the system that should be brought about is ignored. Such high-level norms can be used to represent what the agents should establish—in terms of a declarative description of a system state—rather than specifying how they should establish it.

Another approach concerning specification of normative multi-agent systems by means of social and organisational concepts is ${\scr M}{\rm OISE}^{{\plus}} $ , proposed by Hübner et al. (Reference Hübner, Sichman and Boissier2007). This modelling language can be used to specify multi-agent systems through three organisational dimensions: structural (e.g. specifying roles, groups, and links within organisations, subgroup relation, number of agents that can play a role), functional (e.g. goals, missions, and social schemes specifying structured sets of goals), and deontic (e.g. norms, obligations, and prohibitions within organisations). In a series of papers, different computational frameworks have been proposed to implement and execute ${\scr M}{\rm OISE}^{{\plus}} $ specifications. Examples of such frameworks are $${\cal S}{\minus}{\scr M}{\rm OISE}^{{\plus}} $$ as proposed by Hübner et al. (Reference Hübner, Sichman and Boissier2006) and its artifact-based version ORG4MAS as proposed by Hübner et al. (Reference Hübner, Boissier, Kitio and Ricci2010). These frameworks are concerned with norms that are about declarative descriptions of a state that should be achieved. Following the ${\scr M}{\rm OISE}^{{\plus}} $ specification language, ${\cal S}{\minus}{\scr M}{\rm OISE}^{{\plus}} $ is an organisational middleware that provides agents access to the communication layer and the current state of the specified organisation. Moreover, this middleware allows agents to change the organisation and its specification, as long as such changes do not violate organisational constraints. In the artifact version of this framework, ORG4MAS, various organisational artifacts are used to implement specific components of an organisation such as group and goal schema. In this framework, a special artifact, called reputation artifact, is introduced to manage the enforcement of the norms.

To summarize, in the work on electronic institutions ISLANDER/AMELI norms pertain to low-level procedures that directly refer to actions, whereas ${\scr M}{\rm OISE}^{{\plus}} /{\cal S}{\minus}{\scr M}{\rm OISE}^{{\plus}} $ are concerned with more high-level norms pertaining to declarative descriptions of the system. However, ${\cal S}{\minus}{\scr M}{\rm OISE}^{{\plus}} $ does not allow agents to violate organisational rules and norms by ensuring that they respect the organisational specification. This suggests that norms in ${\cal S}{\minus}{\scr M}{\rm OISE}^{{\plus}} $ are regimented rather than being enforced by means of sanctions. In the artifact version of this framework, ORG4MAS, the enforcement of norms is assumed to be managed indirectly through a reputation mechanism, but it remains unclear how such a reputation system realises sanctioning. Another important issue is that AMELI and ${\cal S}{\minus}{\scr M}{\rm OISE}^{{\plus}} $ lack a complete operational semantics that capture all aspects of normative systems, including the enforcement of norms. An explicit formal and operational treatment of norm enforcement is essential for a thorough understanding and analysis of computational frameworks of normative multi-agent systems. In addition, the computational frameworks related to ${\scr M}{\rm OISE}^{{\plus}} $ are not grounded in a logical system such that the soundness and properties of the programmed systems cannot be analysed through formal analyses and verification mechanisms. Finally, it should be noted that ISLANDER/AMELI and ${\scr M}{\rm OISE}^{{\plus}} /{\cal S}{\minus}{\scr M}{\rm OISE}^{{\plus}} $ provide a variety of social and organisational concepts.

powerJava, as proposed by Baldoni et al. (Reference Baldoni, Boella and Van Der Torre2005), and powerJade as proposed by Baldoni et al. (Reference Baldoni, Boella, Dorni, Grenna and Mugnaini2008), are developed to implement institutions in terms of roles. While powerJava extends Java with programming constructs to implement institutions, powerJade proposes similar extensions to the Jade framework. In these frameworks, an institution is considered as an exogenous coordination mechanism that manages the interactions between participating computational entities (objects in powerJava and agents in powerJade) by means of roles. A role is defined in the context of an institution (e.g. a student role is defined in the context of a school) and encapsulates capabilities, also called powers, that its players can use to interact with the institution and with other roles in the institution (e.g. a student can participate in an exam). For an object or an agent to play a role in an institution in order to gain specific abilities, they should satisfy specific requirements as well. In powerJava roles and organisations are implemented as Java classes. In particular, a role within an institution is implemented as an inner class of the class that implements the organisation. Moreover, the powers that a player of a role gains and the requirements that the player of the role should satisfy are implemented as methods of the class that implements the role. In powerJade, organisations, roles, and players are implemented as subclasses of the Jade agent class. The powers that the player of a role gains and the requirements that a player of a role should satisfy are implemented as Jade behaviours (associated to the role).

Finally, a recent programming language that is developed to support the implementation of multi-agent organisations is 2OPL (Organisation Oriented Programming), as proposed by Dastani et al. (Reference Dastani, Grossi, Meyer and Tinnemeier2009b) and Tinnemeier et al. (Reference Tinnemeier, Dastani, Meyer and van der Torre2009b). This is a rule-based programming language that facilitates the implementation of norm-based organisations. In this approach, an organisation is considered as a software entity that exogenously coordinates the interaction between agents and their shared environment. In particular, the organisation is a software entity that manages the interaction between the agents themselves and between agents and the shared environment. 2OPL provides programming constructs to specify (1) the initial state of an organisation, (2) the effects of agent’s actions in the shared environment, and (3) the applicable norms and sanctions. In 2OPL, norms can be either enforced by means of sanctions or regimented. In the first case, agents are allowed to violate norms after which sanctions are imposed. In the second case, norms are considered as constraints that cannot be violated. The enforcement of norms by sanctions is a way to guarantee higher autonomy for agents and higher flexibility for multi-agent systems. The interpreter of 2OPL is based on a cyclic control process. At each cycle, the observable actions of the individual agents (i.e. communication and environment actions) are monitored, the effects of the actions are determined, and norms and sanction are imposed if necessary. An advantage of 2OPL approach is its complete operational semantics such that normative organisation programs can be formally analysed by means of verification techniques (see e.g. Astefanoaei et al., Reference Astefanoaei, Dastani, Meyer and Boer2009). This organisation-oriented programming language is extended with programming constructs that support the implementation of concepts such as obligation, permission, prohibition, deadline, norm change, and conditional norm; see Tinnemeier et al. (2009Reference Tinnemeier, Dastani and Meyera, Reference Tinnemeier, Dastani, Meyer and van der Torre2009b, Reference Tinnemeier, Dastani and Meyer2010).

4.3 Programming frameworks for multi-agent environments

A framework for the development of multi-agent environments is Cartago (Common artifact infrastructure for agent open environment), which is proposed by Ricci et al. (Reference Ricci, Viroli and Omicini2006). This framework is based on the A&A model that proposes a working environment to be used by agents for supporting their activities. A working environment is considered as consisting of a set of artifacts organised in workspaces (containers of artifacts). The artifacts are meant to encapsulate specific functionalities and can be added, removed, and organised in the workspaces by agents at run time. Artifacts can be used by agents through their usage interfaces that allow agents to trigger and control the execution of artifacts’ operations and perceiving events from them. Different operations are supported by artifact interfaces. An agent can, for example, create, remove, or search for artifacts and workspaces. Agents can also execute operations of artifacts, for example, sense the events generated by an artifact or inspect an artifact by retrieving its description. This framework can be distributed in the sense that a working environment can consist of one or more workspaces that can be mapped onto a different nodes of a network. Cartago is implemented in Java and has been connected to various agent executions platforms such as the execution platforms for 2APL and Jason.

Beside this generic architecture and framework for the development of environments, there have been many interesting environments implemented using existing programming languages such as Java or C++. These environments are initially developed in an ad hoc manner either for an existing agent platform (e.g. the platforms for 2APL, GOAL, Jadex, and Jason) or as a simulation environment. The availability of these implemented environments raises the question how they can be (re)used and applied to arbitrary agent platforms. In practice, agent developers rebuild similar environments from scratch. Apart from these duplicating works, the interaction between agents and environments are managed in an ad hoc manner making the reuse of the environments a dedicated task that depends on the specific agent platform and the environment at hand. This problem has lead to an initiative for creating a generic environment interface that provides the required functionalities for connecting agents to environments (see e.g. Behrens et al., Reference Behrens, Dix, Hindriks, Dastani, Bordini, Hubner, Pokahr and Braubach.2010). This initiative wants to become a de facto standard. If environments were developed using such a standard, they could be exchanged freely between agent platforms that support the standard and thus would make already existing environments widely available. In order to develop a generic environment interface standard various issues should be addressed. An important issue is the right level of abstraction for modelling the interaction between agents and environments. This generic environment interface standard supports the interaction between agents and environment in two ways. On the one hand, agents can perform actions, including sense action, in the environment (the environment is assumed to realise the effect of the actions).

On the other hand, the environment can send events to individual agents. This interface provides constructs to establish and manage the relation between agents with entities (agent bodies) in the environment, the registration of agents by the interface, adding and removing entities from the environment, and performing actions and retrieving percepts from the environment. Several agent platforms such as 2APL, GOAL, and Jason have already integrated the environment interface standard.

5.1 Goal types

An advance in the field of agent programming languages concerns the concept of goals. Goals are essential for agents with pro-active behaviour (see e.g. Wooldridge, Reference Wooldridge2009). The initial focus of agent-oriented programming languages was on achievement goals, which represent a desired state that the agent aims at achieving. In due course other goal types have been studied by Braubach and Pokahr (Reference Braubach and Pokahr2009), Dastani et al. (Reference Dastani, van Riemsdijk and Meyer2006), Duff et al. (Reference Duff, Harland and Thangarajah2006), Hindriks and van Riemsdijk (Reference Hindriks and van Riemsdijk2008). Examples of goal types are perform goal (the goal to execute certain actions), test goal (the goal to test agent’s state), and maintain goal (the goal to maintain a state). In order to allow the implementation of various goal types existing agent programming languages provide a variety of constructs to represent and reason with the goal types. For example, JACK, as presented by Winikoff (Reference Winikoff2005), provides programming constructs to implement, among others, test, achieve, insist, and maintain goals. In addition, Jadex, as presented by Pokahr et al. (Reference Pokahr, Braubach and Lamersdorf2005), covers achieve, query, perform, and maintain goals. The way in which goals are treated by these programming languages differs. In Jadex goals are represented in XML in terms of a label/name and a number of other parameters, while JACK goals are particular types of events. Moreover, neither JACK nor Jadex provide the formal semantics of their goal types. Winikoff et al. (Reference Winikoff, Padgham, Harland and Thangarajah2002) provides a survey of existing literature on goal types. A more recent (theoretical) trend in this direction is to go beyond these goal types and to introduce more expressive goal types or even a language for expressing goal types. For example, Dastani et al. (Reference Dastani, van Riemsdijk and Winikoff2011) proposes six types of multiple state goals (goals expressing a property that should hold over a number of states), while other approaches propose to take arbitrary linear temporal logic (LTL) formulae as goals (see e.g. Bacchus & Kabanza, Reference Bacchus and Kabanza1998; Baral & Zhao, Reference Baral and Zhao2007; Shapiro & Brewka, Reference Shapiro and Brewka2007; Hindriks et al., Reference Hindriks, van der Hoek and van Riemsdijk2009; Khan & Lespoerance, Reference Khan and Lespoerance2009). The advantage of the approach proposed by Dastani et al. is their computational setting where the six multiple state goal types are defined in terms of achieve and maintain goals. This makes it possible to implement these goal types in the agent programming frameworks that already have an operationalisation of achieve and maintain goals.

5.2 Modular programming

From the software development point of view, the ultimate aim of multi-agent programming languages and development frameworks is to support practitioners to develop multi-agent systems for industrial applications. To this aim it is important that programming languages and development frameworks satisfy essential principles in structured programming such as modularity. Of course, the separation of concerns at the level of individual agents, organisation, and environment support modularity in multi-agent programming. However, programming languages and development frameworks for agents, organisations, and environments need to satisfy modularity as well. There have been some proposals for supporting modules in BDI-based programming languages. Examples of these proposals are Braubach et al. (Reference Braubach, Pokahr and Lamersdorf2005), Busetta et al. (Reference Busetta, Howden, Ronnquist and Hodgson2000), Hindriks (Reference Hindriks2008), van Riemsdijk et al. (Reference van Riemsdijk, Dastani, Meyer and de Boer2006), Dastani and Steunebrink (Reference Dastani and Steunebrink2010), and Madden and Logan (Reference Madden and Logan2009). In these proposals, modularisation is considered as a mechanism to structure an individual agent’s program in separate modules, each encapsulating cognitive components such as beliefs, goals, and plans that together model a specific functionality and can be used to handle specific situations or tasks. However, the way the modules are used in these programming approaches are different. For example, in JACK (Busetta et al., Reference Busetta, Howden, Ronnquist and Hodgson2000) and Jadex (Braubach et al., Reference Braubach, Pokahr and Lamersdorf2005), modules (which are also called capabilities) are used for information hiding and reusability by encapsulating different cognitive components that together implement a specific capability/functionality of the agent. In these approaches, the encapsulated components are used during an agent’s execution to process events that are received by the agent. In other approaches (e.g. van Riemsdijk et al., Reference van Riemsdijk, Dastani, Meyer and de Boer2006; Hindriks, Reference Hindriks2008), modules are used to realise a specific policy or mechanism in order to control an agent execution. More specifically, in GOAL, as proposed by Hindriks (Reference Hindriks2008), modules are considered as the ‘focus of execution’, which can be used to disambiguate the application and execution of plans. This is done by assigning a mental state condition (beliefs and/or goals) to each module. The modules whose conditions are satisfied form the focus of an agent’s execution such that only plans from these modules are applied and executed. In 3APL, as proposed by van Riemsdijk et al. (Reference van Riemsdijk, Dastani, Meyer and de Boer2006), a module is a set of planning rules that is associated with a specific goal indicating which planning rules can be applied to achieve the goal. In other words, a module implements specific means for achieving specific goals. In 2APL, as proposed by Dastani and Steunebrink (Reference Dastani and Steunebrink2010), modules are introduced for encapsulation of different cognitive components that together implement a specific agent functionality. The significant difference with other approaches is that a programmer can perform a wide range of operations on modules. These module-related operations enable a programmer to directly and explicitly control when and how modules are used. For instance, a programmer can create an instance of the module specification, query and update its internals, and execute the updated module instance. An agent that executes a module instance, stops deliberating on its current cognitive state and starts deliberating on a new cognitive state that is encapsulated by the executed module instance. The proposed notion of module can be used to implement a variety of agent concepts such as agent role and agent profile. Recently, a modularisation idea for Jason is proposed by Madden and Logan (Reference Madden and Logan2009). In this proposal, a module encapsulates a subset of an agent’s functionalities and consists of cognitive ingredients such as belief, goal, and event bases, a plan library, and a list of exported belief and goal predicates. An agent is then defined as a composition of modules (modules cannot be nested), together with a slightly modified version of the Jason’s original interpreter. Finally, it should be noted that the concept of module as used by Novak and Dix (Reference Novaik and Dix2006) is different from other approaches. In this approach, a module is considered as one specific cognitive component (e.g. an agent’s beliefs) and not as a functionality modelled by different cognitive components. Note also that behaviours in Jade, which can be used to implement an agent’s functionality, can also be seen as a kind of module.

6.1 Integration of programming languages

Respecting the separation of concerns principle advocates separate programming languages and development frameworks for the implementation of individual agents, environments, and organisations. The development of multi-agent systems therefore requires a systematic integration of the corresponding programming languages and development frameworks. Ideally, one should be able to write programs for different components of a multi-agent system separately and integrate these programs either by means of another program that indicates how the programs of different components should interact and executed, or through a platform that facilitates an integrated execution of all involved programs. It should be noted that this challenge is only relevant when different components of multi-agent systems need to be programmed separately using dedicated programming languages and development frameworks.

The first proposal for integrating programming languages for various components of multi-agent systems is based on the integration of 2APL and 2OPL, as mentioned by Tinnemeier (Reference Tinnemeier2011) and realised by Adal (Reference Adal2010). In this approach, a multi-agent program is implemented by specifying a number of agents programmed in 2APL, one or more environments programmed in Java, and an organisation programmed in 2OPL. An execution of such a multi-agent program is a concurrent execution of the specified individual agents programs, the environment program, and the organisation program. The execution of individual agents programmed in 2APL may cause agents to interact with each other and with the environment. The resulting actions are not directly effectuated in the environment, but passed to the organisation implemented in 2OPL. The organisation decides the effects of those actions in the environment based on the specified organisational norms and sanctions. In particular, the performance of actions in the environment by individual agents is effectuated by the organisation program, which allows/disallows actions and realises the effect of actions in the environment. The organisation will also evaluate the updated state of the environment with respect to the specified organisational norms and impose sanctions when violations are detected. Imposing sanctions is seen as specific updates of the environment state according to the sanctions specified by the organisation. This integration approach views an organisation as an exogenous coordination mechanism that controls and regulates the interaction among agents as well as between agents and environment.

Another integration proposal is JaCaMo as proposed by Boissier et al. (Reference Boissier, Bordini, Hbner, Ricci and Santi2013). This approach aims at integrating Jason, Cartago, and MOISE to program individual agents, environments, and organisations, respectively. The idea is to integrate on the one hand MOISE and Cartago, and on the other Jason and Cartago. The integration of MOISE and Cartago is by means of organisational artifacts and based on the earlier work on ORA4MAS as presented by Kitio et al. (Reference Kitio, Boissier, Hubner and Ricci2008). The integration of Jason and Cartago is through artifact operations performed by the Jason agents. In this integrated approach, the organisational infrastructure of a multi-agent system consists of organisational artifacts and organisational agents that together are responsible for functionalities concerning the management and enacting of the organisation. Organisational agents manage organisational activities such as observing and reasoning about organisation dynamics. The violation of norms is detected by organisational artifacts after which organisational agents have to deal with those violations. The organisational artifacts and agents are intended to implement norm regimentation and enforcement by means of sanctions, as originally proposed by Dastani et al. (Reference Dastani, Grossi, Meyer and Tinnemeier2009b). A characteristic of this integration is that the management of organisational activities such as norm enforcement is the responsibility of the so-called organisational agents. It is, however, not clear why such activities should be performed by an agent rather than, for example, by the organisation itself. After all, an agent has by definition its own objectives which is used to motivate its actions and which may not be compatible or even in conflict with the organisation objectives.

6.2 Debugging and testing multi-agent programs

Debugging is the art of finding and resolving errors or possible defects in a computer program. In general, there are various types of bugs such as syntax bugs, semantic bugs (logical and concurrent bugs), or design bugs. Design bugs arise before the actual programming and are based on erroneous design of software programs (see e.g. Dastani et al., Reference Dastani, Brandsema, Dubel and Meyer2009a). In contrast to design bugs, both syntax and semantic bugs arise during programming and are related to the actual code of the programs. Although syntax bugs depend on the specification of programming languages and are (most of the time) simple typos, which can easily be detected by the program parser (compiler), semantic bugs are mistakes at the semantic level. Because they often depend on the intention of the developer they can rarely be detected automatically by the program parsers. Therefore, special tools are needed to detect semantic bugs. The ease of the debugging experience is largely dependent on the quality of these debugging tools and the ability of developers to work with these tools.

The main challenge with respect to debugging multi-agent programs are the semantic bugs caused by the execution of autonomous agent programs, and those caused by the interaction between agents, environments, and organisations. The bugs causing by the interaction between agents are often dealt with by means of different types of visualisation tools such as sniffer and causality graphs (see e.g. Bellifemine et al., Reference Bellifemine, Bergenti, Caire and Poggi2005; Pokahr et al., Reference Pokahr, Braubach and Lamersdorf2005; Botia et al., Reference Botia, Hernansaez and Gomez-Skarmeta2006; Bordini et al., Reference Bordini, Huubner and Wooldridge2007; Dastani, Reference Dastani2008; Vigueras & Botia, Reference Vigueras and Botia2008). The visualisation tools allow the developer to browse through exchanged messages, inspect the messages, and present them using different visualisation techniques. Debugging the interaction between agents, environments, and organisations are still an unexplored research area. The semantic bugs caused by the execution of individual agent programs are dealt with by a variety of techniques such as breakpoints, assertions, and execution tracers (see e.g. Bellifemine et al., Reference Bellifemine, Bergenti, Caire and Poggi2005; Lam & Suzanne Barber, Reference Lam and Suzanne Barber2005; Bordini et al., Reference Bordini, Huubner and Wooldridge2007; Collier, Reference Collier2007; Sudeikat et al., Reference Sudeikat, Braubach, Pokahr, Lamersdorf and Renz2007; Dastani, Reference Dastani2008). A breakpoint is a marker that can be placed in the program’s code to control the program’s execution. When the marker is reached the program execution is halted. Assertions are statements that can be annotated to specific elements of the programming language. When an assertion is evaluated to false, a warning is generated to inform the developer about the agent and the element where the assertion is evaluated to false. The execution tracer is a standard tool that is present in most multi-agent development frameworks. This is a window that enables a developer to view, inspect, and trace an agent’s internal state, and to start, stop, and step through the execution of the agent program.

Debugging agent programs that are based on BDI abstractions requires additional tools to monitor and control temporal and cognitive properties of the agent program executions. In Dastani et al. (Reference Dastani, Brandsema, Dubel and Meyer2009a), a debugging approach is proposed which is based on a specification language to express temporal and cognitive execution properties of multi-agent programs and a set of debugging tools. The expressions of the specification language are related to the proposed debugging tools such that the debugging tools are activated as soon as their associated properties hold for the multi-agent program execution thus far. The specification language is based on LTL extended with the BDI operators. Given an execution of a multi-agent program, one can check if an agent drops a specific goal when it is achieved, when two or more agents have the same beliefs, whether a protocol is suited for a given task, whether important beliefs are communicated and if they are adopted/rejected once they are communicated.

Recent developments in multi-agent programming languages have proposed specific programming constructs enabling the implementation of social concepts such as norms, roles, obligations, and sanctions (see e.g. Esteva et al., Reference Esteva, Rodriguez-Aguilar, Rosell and Arcos2004; Hiibner et al., Reference Hübner, Sichman and Boissier2006; Dastani et al., Reference Dastani, Grossi, Meyer and Tinnemeier2009b; Tinnemeier et al., Reference Tinnemeier, Dastani, Meyer and van der Torre2009b). Debugging such multi-agent programs requires specific debugging constructs to specify properties related to the social aspects and to find and resolve defects involved in such programs. The presented debugging approaches assume all agents are developed on one single platform such that their executions for debugging purposes are not distributed on different platforms. One important challenge and a future work on debugging multi-agent systems remains the debugging of multi-agent programs that run concurrently on different platforms. It is, for example, not clear how the distributed executions of a multi-agent program can be monitored and controlled or how the distributed program codes can be accessed and modified.

The techniques mentioned above are helpful when errors manifest themselves directly to the developers or users. However, errors in a program do not always manifest themselves directly. For industrial applications it may be necessary to extensively test programs before deploying them. The testing should remove as many bugs (and possible defects) as possible. However, it is often infeasible to test every single situation in which the program could run. A testing approach for multi-agent programs is proposed by Poutakidis et al. (Reference Poutakidis, Padgham and Winikoff2002, Reference Poutakidis, Padgham and Winikoff2003) and Winikoff (Reference Winikoff2010). Testing is an indispensable part of evaluating multi-agent programs and should therefore be integrated in the existing debugging approaches. This allows the generation of a set of critical test traces which will be the subject of debugging in post mortem mode. Finally, we would like to emphasise that for mission and industrial critical systems, it is often necessary to formally verify the programs by means of theorem proving or model checking techniques. Unfortunately, these verification techniques are computationally expensive and cannot be applied to large-scale software systems. Some existing work on formal verification of multi-agent systems are presented in Dastani et al. (Reference Dastani, Hindriks and Meyer2010).

6.3 Towards industry-strength technology

The field of multi-agent programming can be approached from two different perspectives: academic and industry. From the academic perspective, this research field contributes to the integration of theories and techniques from artificial intelligence in the development and implementation of intelligent systems. In particular, it provides computational and executable models for high-level (social and cognitive) concepts, and proposes corresponding programming constructs that can be used to develop and implement intelligent systems. For example, computational models for beliefs and goals with their corresponding programming constructs are used to develop and implement proactive systems, and event processing mechanisms are used to develop and implement reactive systems. Moreover, the provided models and programming languages are extended and used to build prototypes of intelligent systems. For example, programming languages for BDI agents are extended with emotion models (e.g. Dastani & Meyer, Reference Dastani and Meyer2010) to prototype emotional agents, and multi-agent programming languages are extended with social concepts to prototype or simulate agent societies. The academic perspective on this research field has shown to be attractive to the researchers from the logic community. In particular, researchers from computational logics and knowledge representation and reasoning have proposed formal and computational models that can be used in multi-agent programming languages and their interpreters to allow the representation and reasoning with an agent’s (cognitive) state. In addition, researchers with specification and verification background have proposed various frameworks to specify and verify properties of individual agents and multi-agent programs.

Although the research in the multi-agent programming community has been dominated by the academic perspective and attracted more academic researchers than industrial practitioners, an important aim of this community has been the development of industry strength tools and technologies to build industrial applications. There are some unverified explanations for why the existing tools and technologies from multi-agent programming research community have not been adopted and deployed for industrial applications. For example, it is often claimed that the industry requires tools and technologies with predictable behaviours, while agents and multi-agent systems are generally seen by the industry as intelligent and adaptive systems with unpredictable behaviours. Moreover, the developers from the industry seem to prefer having full control over the execution of the developed software systems, while most of the existing multi-agent programming languages are based on interpreters that make complex decisions (e.g., deliberation and control processes) on behalf of the software developers. It is also claimed that the industry tends to use standard software technologies and is not eager to adopt paradigmatic change in software technologies and methodologies.

The adoption of multi-agent programming tools and technologies by the industry is a major challenge that still needs to be met by the multi-agent programming community. One possible way to meet this challenge is by transferring multi-agent programming tools and technologies to the standard software technologies. An idea is to start with the high-level concepts and abstractions for which multi-agent programming research field have provided computational models and programming constructs, and propose either corresponding language-level supports in the standard programming languages (e.g. C++ or Java), or alternatively propose corresponding design patterns, that is, general reusable solutions to problems such as proactivity, reactivity, adaptivity, control, and monitoring. The language-level support can either be realised by standard programming approaches such as meta-programming or aspect-oriented programming where concepts such as deliberation and control can be considered as different concerns that can be programmed either by meta-programs or aspects. Although these suggestions are not mature and need to be work out both in details and in practice, attempts along these lines can bring multi-agent community closer to the industry.