1 Introduction
The latter part of the 20th and the beginning of the 21st centuries have witnessed a significant transformation from work organized around individual jobs to team-based work structures together with a focus on organizational efficiency (Kozlowski & Bell, Reference Kozlowski and Bell2013). This is due to the increasing complexity of tasks, which in many cases cannot be performed by single individuals (Ramezan, Reference Ramezan2011). Additionally, changes in technology facilitate workers in distinct locations to communicate and collaborate at low or no cost. On that account, team composition and formation research is of interest to many fields of science, primarily to organizational psychology (OP). Moreover, it has also substantially pervaded the field of computer science (CS), mainly within the area of multiagent systems (MAS). In this paper, our understanding of team composition and formation differentiates from the definitions provided by the multiagent field. ‘Team Formation’ is a term that has been used by other authors in MAS field to describe the process which in this article is referred to as ‘Team Composition’. We define team composition as the process of deciding which agents will be part of a team. We understand team formation as the process undertaken by agents to learn to work together in a team, and through this learning decide the roles and internal organization of the team.
Team research in MAS has considered a variety of application domains (e.g. Unmanned Aerial Vehicle operations (Haque et al., Reference Haque, Egerstedt and Rahmani2013), teamwork in social networks (Lappas et al., Reference Lappas, Liu and Terzi2009), or RoboCup rescue teams (Ramchurn et al., Reference Ramchurn, Farinelli, Macarthur and Jennings2010)) wherein agents face the challenge of performing tasks that are either too complex for one single agent or limited in time, thus requiring several agents to collaborate.
Nevertheless, research on team composition and team formation in CS and OP has evolved separately. On the one hand, MAS literature has typically disregarded significant OP findings, with the exception of several recent, preliminary attempts (such as Farhangian et al., Reference Farhangian, Purvis, Purvis and Savarimuthu2015b; Hanna & Richards, Reference Hanna and Richards2015; Andrejczuk et al., Reference Andrejczuk, Rodriguez-Aguilar and Sierra2016, Reference Andrejczuk, Rodriguez-Aguilar, Roig and Sierra2017). Thus, this body of research has focussed on algorithms that help automate team formation and composition. On the other hand, the OP literature has mainly focussed on empirically investigating the factors that influence team performance to develop heuristics that help organizations handcraft their teams. OP has disregarded the algorithmic results developed by computer scientists to automate team composition and formation. Despite the common research interests shared by MAS and OP, to the best of our knowledge there has been no effort in the literature to bridge the knowledge produced by both research disciplines.
Against this background, the aim of this article is to survey both disciplines, to analyze and compare the strengths and weaknesses of their contributions, and to identify research gaps and opportunities by bringing together the knowledge of the two research strands on team composition and formation. Our analysis also pursues to identify cross-fertilization opportunities that help both disciplines benefit from one another.
In order to structure our analysis, we have identified several dimensions that help us dissect the contributions from both research fields:
1. WHO is concerned? The properties of the agents involved.
2. WHAT is the problem? The features of the task to complete by a team.
3. WHY do we do it? The objective function to optimize when composing/forming a team.
4. HOW do we do it? The organization and/or co-ordination structure adopted by the team in charge of performing a particular task.
5. WHEN do we do it? The dynamics of the stream of tasks to be completed by agent teams.
6. WHERE do we do it? The context wherein team composition/formation occurs.
Our analysis of the literature indicates that CS and OP exhibit some similarities. Indeed, one of the crucial findings in both OP and CS is that team members have to be heterogeneous to maximize team performance. When modelling agents, CS and OP agree on considering two main approaches: either there is complete information about the properties of each agent; or agents are capable of learning about their teammates through repeated interactions. Regarding tasks, both OP and CS research largely focus on finding team members whose properties make them capable of performing a given task based on its requirements. In other words, they are both concerned with matching agents (or whole teams) with tasks.
However, there are important differences between the contributions made by OP and CS that stem from the fact that OP does consider the whole complexity of: humans as team members, tasks, the context where teams perform tasks (understood as the internal and external factors influencing teamwork), and the dynamics of the actual-world scenarios where tasks appear to be serviced. Thus, OP assumes that human capabilities are necessarily dynamic (evolve along time) so that teams can successfully perform tasks in dynamic real-world scenarios and in a variety of contexts. Furthermore, OP observes that the quality of human resources (e.g. motivation, satisfaction, commitment), the ability of individuals to learn new capabilities, and the context constraining team performance significantly influence team performance. Finally, OP research also focussed on identifying correlations between task types and team types to compose the best team depending on the type of each particular task. All these findings contributed by OP research offer interesting opportunities for cross-fertilization.
The rest of the paper is organized as follows. Section 2 introduces some fundamental terminology to make clear what we mean by team composition, team formation, and teamwork. Thereafter, the paper is organized around two main sections. Section 3 reviews the MAS contributions to team composition and team formation. Next, Section 4 surveys the contributions in the OP literature. Finally, Section 5 identifies the main similarities and differences between the two bodies of research. Furthermore, it also discusses cross-fertilization opportunities between both fields that may spur future research.
2 Background
We introduce the fundamental terminology used in this survey. We refer to:
1. Team composition as the process of deciding which agents will be part of a team.
2. Team formation as the process of learning by agents to work together in a team and through this learning decide the roles and internal organization of a team.
3. Teamwork as the process of performing a task by a composed and formed team.
While there is a common understanding of teamwork within both OP and CS, the scientists do not agree on the notion of team formation. In CS it is mostly understood as the process of deciding which agents will be a part of a team (here called team composition). Our definition of team formation is in line with the OP literature (Kozlowski & Bell, Reference Kozlowski and Bell2013: 16).
Another discrepancy between the CS and the OP literature is the notion of skill and competence. Typically in CS all kinds of agents’ competences are called skills, while in OP the definition is more complex. In OP a prominent conceptualization of competence was given by Roe (Reference Roe2002: 195). He defines competence as ‘a learned ability to adequately perform a task, duty or role’. Following his definition competences ‘integrate knowledge, skills, personal values, and attitudes and are build on knowledge and skills and are acquired through work experience and learning by doing’ (Bartram & Roe, Reference Bartram and Roe2005). Hence, competences include abilities and behaviours, as well as knowledge that is fundamental to the use of a skill. An example may consist of a programming task. In order to effectively write a script one needs good logical and analytical competences as well as the skill to write a program in a specific language. Hence, Java is a skill. Although, underlying the ability to use that skill effectively is a competence.
3 Team composition and formation from a computer science perspective
Team composition and formation are critical issues for co-operative MAS. In this section, we survey the most recent and representative approaches in the MAS literature to the team composition and formation problems along the dimensions identified in the introduction above.
3.1 WHO is concerned?
The question behind team composition and formation is how to create a MAS as a group of heterogeneous agents (such as humans, robots, software agents, or even animals) and how to organize their activities. Team members must observe the environment and interact with one another in order to perform tasks or solve problems that are beyond their individual capabilities. The algorithms to create these teams take inspiration from human teamwork. We observe people working together on daily activities as well as on research and business projects. For instance, there are sport teams (e.g. football, basketball), police squads, search and rescue teams formed by dogs and humans, and we start to witness human–robot co-operation in houses, hospitals, or even in space missions (Hoffman & Breazeal, Reference Hoffman and Breazeal2004).
In general, MAS research focusses on the interaction among intelligent agents. In the team formation literature, the focus is on the interaction of co-operative and heterogeneous agents. That is, agents who share a common goal, and have different individual properties. Therefore, in this section, we would like to account for the different ways previous research has dealt with these questions. We will classify individual properties according to two dimensions:
1. capacity: individual and social capabilities of agents; and
2. personality: individual behaviour models.
3.1.1 Capacity: individual and social capabilities of agents
In many domains, a capability is defined as a particular skill required to perform an action. The capacity dimension has been exploited by numerous previous works, like Robust Team Formation (Okimoto et al., Reference Okimoto, Schwind, Clement, Ribeiro, Inoue and Marquis2015; Crawford et al., Reference Crawford, Rahaman and Sen2016) or Online Team Formation (Anagnostopoulos et al., Reference Anagnostopoulos, Becchetti, Castillo, Gionis and Leonardi2012). In these works, agents are assumed to have multiple binary skills (i.e. the agent either has a required skill or not). This is a simplistic way to model an agent’s capabilities since it ignores any skill degree. In real life, capabilities are not binary since every individual (e.g. human or robot) shows different action performance. This is why some works propose a more realistic approach by defining graded agent capabilities, for instance, by defining skill levels (Chalkiadakis & Boutilier, Reference Chalkiadakis and Boutilier2012).
On a different vein, Rangapuram et al. (Reference Rangapuram, Biihler and Hein2015) builds a weighted, undirected graph where the weight between each pair of agents reflects their degree of compatibility to jointly solve tasks. These weights are updated along multiple encounters between agents. On a somehow related vein, Peleteiro et al. (Reference Peleteiro, Burguillo-Rial, Luck, Arcos and Rodriguez-Aguilar2015) try to capture the quality of the solutions of team tasks via a model that besides using skills and compatibility between agents (called the strength of collaboration synergies within coalitions), calculates the reputation of teams (coalitions) as a whole and of single agents. These reputation values are used by the team composition process.
Typically, the capabilities of agents are assumed to be known, though there exist models that consider that an agent can learn the capability levels of other agents. For instance, Liemhetcharat and Veloso (Reference Liemhetcharat and Veloso2014) had the insight that repeated interactions allow to discover the capabilities of other agents. Agents learn a model of synergy via repeated interactions. Such synergy values are then used by individual agents to learn the capabilities of others, and hence to subsequently compose teams with improved performance. However, in open environments (i.e. when new agents and tasks are dynamically introduced), agents need more sophisticated procedures to decide which team to join. For instance, Chen et al. (Reference Chen, Chen, Timsina and Soh2015) propose an ad hoc team formation framework that considers learning other agents’ capabilities in the context of unknown tasks. In order to solve a new task, agents would prefer to team up with unknown agents instead of with agents whose known capabilities do not adjust to the task. They observe that learning the capabilities of others in the context of agent and task openness improves team composition and task resolution.
3.1.2 Personality: individual behaviour models
Personality is key to understand people’s behaviour, cognition, and emotion. The use of personality models in agents helps to create more realistic complex scenarios. Indeed, autonomy is related to how individuals behave and what makes them behave differently, even when facing the very same situation. Personality provides a mechanism for behaviour selection that is independent of social background (such as beliefs or morality). Very few MAS contributions considered the notion of personality, that is, individual behaviour model, to compose heterogeneous teams. For instance, Hanna and Richards (Reference Hanna and Richards2015) study the influence of two agent personality traits: extraversion and agreeableness, both expressed as verbal and non-verbal communication skills. They construct pairs of human users and Intelligent Virtual Agents (IVAs) and analyze how the personality traits influence the development and maintenance of a Shared Mental Model (SMM). The results confirm the importance of providing IVAs with these personality traits to succeed in jointly solving tasks. On a different vein, Andrejczuk et al. (Reference Andrejczuk, Rodriguez-Aguilar and Sierra2016) use personality traits to partition a group of humans into psychologically balanced and gender-balanced heterogeneous teams with the purpose of increasing the overall performance of the resulting teams. In the other paper, Andrejczuk et al. (Reference Andrejczuk, Rodriguez-Aguilar, Roig and Sierra2017) use personality as well as competences to partition agents into competent, personality, and gender balanced teams. The personality traits are measured by the Post-Jungian Personality Theory, which is a modified version of the Myers–Briggs Type Indicator (MBTI) (Myers et al., Reference Myers, McCaulley, Quenk and Hammer1998; Wilde, Reference Wilde2013).
Marcolino et al. (Marcolino et al., Reference Marcolino, Jiang and Tambe2013, Reference Marcolino, Xu, Gerber, Kolev, Price, Pantazis and Tambe2016; Nagarajan et al., Reference Nagarajan, Marcolino and Tambe2015) propose a new approach for action selection. A task is a sequence of actions to be decided at execution time. To choose which action to execute next, every heterogeneous agent within a team votes for its preferred candidate action. Agents vote according to a probability distribution over actions that varies for each agent. This can be understood as a way of modelling an agent’s personality, motivations, and beliefs (causing him to behave in a certain way).
In a series of papers, Farhangian et al. (Reference Farhangian, Purvis, Purvis and Savarimuthu2015a, Reference Farhangian, Purvis, Purvis and Savarimuthu2015b) use the MBTI scheme to model different agent personality types. Farhangian et al. (Reference Farhangian, Purvis, Purvis and Savarimuthu2015a) use both individuals’ skills and personality types (measured by MBTI and Belbin, Reference Belbin1993, personality tests) to compose teams. These two dimensions are used to simulate human team composition in a business environment.
Another aspect covered by the existing literature is the individual agent knowledge about the other team members’ personalities, that is, about their behaviour models. These works go beyond many ‘ad hoc’ team composition systems where information details about the behaviour of individual agents is absent. Barrett et al. (Reference Barrett, Stone, Kraus and Rosenfeld2013) focus on how a new member in a team behaves in order to co-operate well with the other team members whose behaviours are unknown. Each agent is endowed with a learning mechanism for building models of the behaviours of many distinct types of other agents via repeated interactions. A similar setting is presented by Agmon et al. (Reference Agmon, Barrett and Stone2014), though they consider that there are only two types of agents: a best response agent (choosing his action based on the current state of the world), and an ad hoc agent (has a better awareness of the team’s possible actions and the resulting joint utility). There is no a priori model, hence, similarly to Barrett et al. (Reference Barrett, Stone, Kraus and Rosenfeld2013), an ad hoc agent needs to decide his behaviour by observing his peers.
3.1.2.1 Analysis
In summary, team composition and formation research has focussed so far on co-operative, heterogeneous agents that have a set of properties. These properties can be categorized into two groups: capacity and personality. To our knowledge, besides Farhangian et al. (Reference Farhangian, Purvis, Purvis and Savarimuthu2015a), there has been no further attempts to combine capabilities and personality for team composition and formation in the area of MAS. Besides that, we observe that the capabilities of agents are always static, but the behaviour model may change with agents’ interactions. While the capabilities of humans change over time, the MAS literature typically does not consider dynamic capabilities for software agents. Finally, when modelling agents’ properties, many existing approaches typically assume extensive a priori information about teammates. This is a strong limitation for real-life settings. Notice that in many companies there is no central and extensive knowledge about all employees’ capabilities.
3.2 WHAT is the problem? The notion of task
In its most general sense, a task is a course of action to achieve a goal. The execution of a task is then usually equated to the execution of an action plan. Action plans can be rather complex as they may take into account concurrency of actions, time constraints, action order, or environment uncertainty. However, in the team formation literature it is often the case that simplifying assumptions are made and tasks are assumed to be solved by simple action plans. For instance, an action plan can be seen as a set of actions, or even as a set of competences. In this latter case the idea behind is that the task can be successfully solved by a team of individuals with expertise in a number of different fields. In this section, we review which concepts of task have been proposed in team formation and team composition. We identify two main approaches:
∙ Individual-based, that is, capacity or personality (see Section 3.1).
∙ Plan-based, for example, the set of actions or subtasks.
Next we discuss each approach in detail.
3.2.1 Individual-based approaches
Sometimes teams work less effectively than initially expected due to several reasons: a bad balance of their capacities, bad personal relations, or difficult social situations. Hence, in order to make sure a task is performed the most effectively, the large body of literature defines the action plan of the task as a set of requirements for agent individual characteristics. It is assumed that the task can be fulfilled if the task requirements are a subset of the capabilities of team members. We categorize existing work on team composition with the purpose to solve a task into two categories of individual properties: capacity and personality.
3.2.1.1 Capacity
The capabilities of team members are crucial while performing a task. For instance, it is obvious that in order to develop an online Java application, the collective team knowledge has to include Java, Java EE, front-end tools, and database and server knowledge. In the MAS literature (as discussed in Subsection 3.1.1), the majority of research work expresses capabilities as binary (they are present or they are not) (Anagnostopoulos et al., Reference Anagnostopoulos, Becchetti, Castillo, Gionis and Leonardi2012; Chen et al., Reference Chen, Chen, Timsina and Soh2015; Okimoto et al., Reference Okimoto, Schwind, Clement, Ribeiro, Inoue and Marquis2015; Crawford et al., Reference Crawford, Rahaman and Sen2016). The main shortcoming of the binary approach is the restrictive assumption that if an agent has a capability, his expertise level is sufficient to perform a given task, which implies that the quality of the task performed is not relevant.
In many cases, the definition of a task is indirectly connected to the agents’ capabilities. Peleteiro et al. (Reference Peleteiro, Burguillo-Rial, Luck, Arcos and Rodriguez-Aguilar2015) propose a model where a task is defined as a tuple that contains the specification of the task (i.e. its subtasks) and the deadline by which the task has to be completed. Each subtask is then matched with one capability. A contract net algorithm is used to compose a team of agents that covers all the required capabilities while maximizing the reputation of the team, thus leading to the best expected performance. In Chalkiadakis and Boutilier (Reference Chalkiadakis and Boutilier2012), a project is defined as a set of tasks, where each task has a complexity level (e.g. moderate or ambitious). Agents’ capabilities are graded (e.g. a good carpenter). Tasks are matched with agents’ capabilities. The probability of an agent succeeding at performing a task depends on the capability degree of the agent performing the task and the complexity level of the task. These probabilities are learned through repeated interactions between agents, and then used by them to self-organize as teams. Finally, in Roles and Teams Hedonic Games (RTHG) (Spradling et al., Reference Spradling, Goldsmith, Liu, Dadi and Li2013) each agent expresses his preferences over both his own roles within a team and on the set of roles needed in the team. This way, agents themselves jointly select a set of required capabilities to perform a given task.
3.2.1.2 Personality
Similarly, personalities of team members are crucial for performing tasks. According to Wilde (Reference Wilde2009), different types of tasks require different personalities in a team. In detail, people with different personalities approach tasks in a diverse way, resulting in better and faster solutions. Along this line, Andrejczuk et al. propose a team composition algorithm that groups agents into different teams so that the personalities in each team are as disparate as possible and gender is balanced.
In Farhangian et al. (Reference Farhangian, Purvis, Purvis and Savarimuthu2015b), the nature (structure) of a task is quantitatively characterized: from extremely structured to extremely open-ended. While structured tasks are straightforward and do not require planning, open-ended tasks require creativity and imagination from team members. In another article, Farhangian et al. (Reference Farhangian, Purvis, Purvis and Savarimuthu2015a) try to capture the dynamics of tasks by matching the required levels of creativity, urgency, social interaction, and complexity of a task to personalities of agents. For instance, teams composed of differing attitude tendencies (associated with different personalities) are believed to outperform teams composed of like-minded people when tackling tasks requiring a high level of creativity.
Finally, Hanna and Richards (Reference Hanna and Richards2015) show that when performing a task, the personality of team members influences their success. They analyze the influence of an IVA communication style (expressing its personality) on human-IVA co-operation. The task is a collaborative game that involves dodging a sequence of obstacles to reach a target.
3.2.2 Plan-based approaches
The notion of task in plan-based approaches is normally understood either as a set of actions or as a sequence of actions. Well-organized teamwork can shorten the time required for completing a particular task by distributing a set of actions across team members. Both Barrett et al. (Reference Barrett, Stone, Kraus and Rosenfeld2013) and Agmon et al. (Reference Agmon, Barrett and Stone2014) employ an indirect planning method driven by the most informed agents to solve a set of actions. Barrett et al. (Reference Barrett, Stone, Kraus and Rosenfeld2013) introduce an ad hoc team agent that learns its teammates’ models (i.e. their predictable action selection) and chooses its own actions so that they collectively maximize the likelihood of success. In detail, they use Monte Carlo sampling to simulate the long term effects of collective actions. As an extension to the previous work, in Agmon et al. (Reference Agmon, Barrett and Stone2014) the actions selected by ad hoc agents influence the actions that the other team members will choose. Each agent has a set of possible actions that it may choose in order to solve each subtask. The ad hoc agents need to predict the actions of its teammates (conditioned in this case to its own actions) and behave based on these predictions with the purpose of influencing the collective selection of actions in the team to reach a joint optimal solution.
Among the approaches considering a task as a sequence of actions, in Marcolino et al. (Reference Marcolino, Jiang and Tambe2013) a team of agents jointly playing the computer game Go plan which action to take next by voting on the possible alternatives from a discrete set of possible actions. Authors prove that under certain conditions of opinion diversity, aggregating the decisions of a team of heterogeneous agents is a better planning strategy than the decision of a team built with copies of the most competent agent (called the strongest agent). This shows that diversity improves the planning capacity of a team solving a complex task like Go. In Marcolino et al. (Reference Marcolino, Xu, Gerber, Kolev, Price, Pantazis and Tambe2016), the authors use the same technique to suggest a user a number of optimal solutions for their next action decision. The application domain of their algorithm is house design. Various design alternatives are proposed to the user in order to select one for further study.
Similarly, in Moon et al. (Reference Moon, Carley, Schneider and Shigiltchoff2005) the plan is created by team members during a game. The domain used for this study is an on-line multi-player computer game called America’s Army, which is a first-person shooting game. The game is the duel of two teams, usually an assault team and a defense team. A team consists of one to fourteen players. Every game starts with a new set of players that need to co-ordinate their activities during the game in order to win. Players are allowed to communicate in a team chat. A team wins the game either by killing all of its opposing players, or by accomplishing the goal for that mission (for instance, securing an oil pipeline or crossing a bridge).
Finally, Rochlin et al. (Reference Rochlin, Aumann, Sarne and Golosman2016) deal with self-interested agents in a team that select one agent to accomplish the task of purchasing a jointly desired item with the lowest possible cost. By doing so, the team assigns the execution of the plan to a single member of the team, becoming the buyer. The buyer’s strategy decides whether to maintain the search looking for better deals (search for a further action), or stop looking and buy at the lowest price found so far, bearing the incurred buyer’s overhead. This strategy balances the expectation of finding a better price (considering the price distribution built during the search) and the team policy to reimburse the cost of the task solution finding to the buyer.
3.2.2.1 Analysis
In conclusion, tasks are solved by the execution of action plans. How complex these action plans are depends on the focus of the reviewed contributions. Individual-based approaches understand action plans as sets of requirements on a team members’ capacity and personality. These approaches assume that the joint capabilities of agents in a team must be enough to solve a given task. Contrarily, plan-based approaches regard tasks as sets of actions or sequences of actions that are assigned to the individual members of a team. All these works propose algorithms that determine which action will be executed and by whom. However, plan-based approaches have a very simplistic notion of plan. The majority of models do not consider time constraints, action dependencies, action failure, plan robustness, or dynamic changes in a task requirements. Therefore, the vast literature on planning has not yet been integrated into team formation methods.
3.3 WHY do we do it? The objective(s)
The motivation of individual efforts or actions is to attain or accomplish a certain state of affairs: its goal. A necessary condition for a team to exist is that all team members are committed to a joint goal. Therefore, in CS an agent team is typically built of at least two co-operative agents that share a common goal; by teaming up, these goals can be achieved in a more effective way. This is the main motivation of team composition and formation. A large body of literature proposes team composition algorithms to attain at least one of the following team objectives:
1. minimizing overall cost (e.g. co-operation cost, team cost);
2. maximizing social utility; or
3. maximizing the quality of an outcome.
In this section we describe the literature on team composition per objective.
3.3.1 Minimizing overall cost
Team cost efficiency has received some attention in the literature. There are various costs associated with team composition and formation problems (e.g. communication costs or agent service costs). For instance, some results balancing cost and quality were obtained by Kargar et al. (Reference Kargar, An and Zihayat2012). They propose algorithms for composing a competent team in a social network. When composing a team, those algorithms minimize team members’ costs and communication costs within the team. Kargar et al. (Reference Kargar, An and Zihayat2012) require that agents have the necessary competences to perform a task, but do not require any specific motivation from them.
A similar approach is presented in Crawford et al. (Reference Crawford, Rahaman and Sen2016) and Okimoto et al. (Reference Okimoto, Schwind, Clement, Ribeiro, Inoue and Marquis2015). These works propose a model for robust team composition and go a step further with respect to Kargar et al. (Reference Kargar, An and Zihayat2012) since they minimize the overall cost among k-robust teams (see Section 3.4.1 for a definition of a k-robust team). That is, this model assumes that up to k agents within a team may eventually fail without affecting the achievement of the task. Thus, it assumes more realistic conditions than Kargar et al. (Reference Kargar, An and Zihayat2012). However, likewise Kargar et al. (Reference Kargar, An and Zihayat2012), agents’ motivations to work together in a team are not considered. Finally, Anagnostopoulos et al. (Reference Anagnostopoulos, Becchetti, Castillo, Gionis and Leonardi2012) propose approximation algorithms to compose teams minimizing simultaneously low co-ordination costs and agent workload.
3.3.2 Maximizing social welfare
A second objective considered in the team composition and formation literature is maximizing social welfare. That is, maximizing the utility function of a team, as a whole, while performing a task. The utility obtained is then allocated to the individual members of the team. For instance, Chalkiadakis and Boutilier (Reference Chalkiadakis and Boutilier2012) propose a Bayesian Reinforcement Learning framework where agents learn from iterated coalition compositions. Agents can choose between exploration (select coalitions to learn more about new agent types) and exploitation (rely on known agents). Exploitation enables agents to maximize their utility function by performing tasks with reliable agents (discovered during the exploration phase).
Paradoxically, the agent motivation to maximize its individual welfare may reduce the overall team cost and additionally increase the overall quality of the performed task. For instance, in Rokicki et al. (Reference Rokicki, Zerr and Siersdorfer2015) a human team competition mechanism improves cost efficiency and the quality of a solution in a team-based crowdsourcing scenario. In conventional crowdsourcing reward schemes, the payment of online workers is proportional to the number of accomplished tasks (pay-per-task). Rokicki et al. examine the possibility of getting much higher rewards by introducing strategies (e.g. random or self-organized) for team composition. Their mechanism triggers the competition among human teams as the reward is only given to the top-5 performing teams or individuals. Their evaluation shows substantial performance boosts (30% in the best scenario) for team-based settings without decreasing the quality of the outcome.
The objective of maximizing social welfare is also considered in many ad hoc settings, like the one proposed by Agmon et al. (Reference Agmon, Barrett and Stone2014). Agmon et al. consider a framework with two types of agents: best-response and ad hoc agents forming teams. On the one hand, best-response agents have limited knowledge and assume that the environment and their teammates will behave as observed in the past. On the other hand, ad hoc agents have a more complete view of a team actions, agents’ joint utilities and their action costs. Using such information, ad hoc agents try to influence joint decisions. In Agmon et al. (Reference Agmon, Barrett and Stone2014) the authors consider that ad hoc agents know with uncertainty their teammates’ behaviour. The paper analyzes the impact on optimal solutions of ad hoc agents misidentifying their teammates’ types.
The study of self-interested agents that co-operate in a team has also attracted the interest of researchers in MAS. An interesting example of this approach is presented in Farhangian et al. (Reference Farhangian, Purvis, Purvis and Savarimuthu2015b), where self-interested agents need to maximize the welfare of all team members in order to maximize their own benefit. Hence, they indirectly aim at maximizing the utility of the team. Similarly, in Chen et al. (Reference Chen, Chen, Timsina and Soh2015) agents repetitively decide which team to join by balancing both rewards from completing tasks and learning opportunities from more qualified agents. That is, each agent consider whether to sacrifice short-term rewards to acquire new knowledge that benefits himself and the whole community in the long run.
3.3.3 Maximizing quality
The last range of models propose a number of methods where agents try to maximize the quality of solutions whilst minimizing the time to achieve them, namely to maximize team performance.
Recent OP studies show that team members’ diversity is a key factor to increase team performance Wilde (Reference Wilde2009). As mentioned in Section 3.1 Marcolino et al. (Reference Marcolino, Jiang and Tambe2013) present a setting where agents in a team vote together to decide on the next joint action to execute that maximizes the team’s solution quality. The authors prove that a diverse team can overcome a stronger team (i.e. a team built of copies of the strongest agent) if at least one agent has a higher probability of taking the best action in at least one world state than the probability that the best agent has of taking that action in that state. The attempt of capturing heterogeneity is also used by Andrejczuk et al. (Reference Andrejczuk, Rodriguez-Aguilar, Roig and Sierra2017). There, instead of looking for a single heterogeneous team, Andrejczuk et al. partition a group of agents into psychologically balanced, gender-balanced, and competent heterogeneous teams with the purpose of increasing the overall performance of the resulting teams.
Hanna and Richards (Reference Hanna and Richards2015) also use personality to investigate the influence of IVAs on team collaboration. Their findings reveal that team performance boosts when the human and the IVA in a team have a SMM. Building a SMM is directly related to the psychological traits of IVA.
Carley et al. (Reference Carley, Moon, Schneider and Shigiltchoff2005) found that the most favourable size of a team is 10 because of the relatively higher survival ratio. Also, frequent usage of the weapon, precision of the weapon used, and frequency of communication, can be the distinctions between winning teams and losing teams. Moreover, frequent communication increases a team’s situation awareness, that is, gives information about where other team members are and how they can be supported.
Peleteiro et al. (Reference Peleteiro, Burguillo-Rial, Luck, Arcos and Rodriguez-Aguilar2015) introduce a decision-making mechanism that on top of improving the quality, aims at increasing the quantity of completed tasks. It uses reputation and adaptation mechanisms to allow agents in a competitive environment to autonomously join and preserve coalitions (teams). In terms of team performance, they show that coalitions keep a high percentage of tasks serviced on time despite a high percentage of unreliable workers. Moreover, coalitions and agents demonstrate that they successfully adapt to a varying distribution of incoming tasks.
Liemhetcharat and Veloso (Reference Liemhetcharat and Veloso2012) developed a model to learn and analyze capabilities of agents and synergies among them to solve the team composition problem using previous joint experiences. They define a synergy model as a graph where the distance between agents is an indicator of how well they work together. Their main contribution is that their algorithm learns from only a partial set of agent interactions in order to learn the complete synergy model. In a subsequent article (Liemhetcharat & Veloso, Reference Liemhetcharat and Veloso2014), the authors study the learning agent team formation problem with the goal of maximizing the mean performance of a team after K learning instances. There, learning agent pairs have heterogeneous rates of co-ordination improvement, and hence the allocation of training instances has a larger impact on the performance of the final team.
The notion of fairness is also considered in the context of team performance. An example of this approach is given in Liemhetcharat and Veloso (Reference Liemhetcharat and Veloso2014). Rochlin et al. analyze the correlation between efficiency and fairness in teams consisting of self-interested agents. They prove that the more fair the team the more efficient its members are.
3.3.4 Validation methods
Finally, it is worth discussing how researchers in CS evaluate and monitor the achievement of the objectives mentioned above. Omitting this information can create a false equivalence between the findings of research studies conducted in very different conditions. We distinguish among three main data sources, that is:
∙ existing databases available online containing real data;
∙ data simulation;
∙ empirical data.
3.3.4.1 Existing data
Finding ready data sets for validation of team composition and formation problems is challenging. Systems supporting team composition or/and formation are not yet in broad use and most data from them is not publicly available. For this reason, some authors use bibliography (such as Citeseer, DBLP, Bibsonomy), movie data sets (IMDb) or a software engineering environment (such as the Python Enhancement Proposals) that can demonstrate the effectiveness of their approach (Anagnostopoulos et al., Reference Anagnostopoulos, Becchetti, Castillo, Gionis and Leonardi2012; Kargar et al., Reference Kargar, An and Zihayat2012; Farhangian et al., Reference Farhangian, Purvis, Purvis and Savarimuthu2015b; Rangapuram et al., Reference Rangapuram, Biihler and Hein2015). For instance, Rangapuram et al. (Reference Rangapuram, Biihler and Hein2015) use an academic scenario (Citeseer database) to perform a qualitative and quantitative assessment of teams.
3.3.4.2 Data simulation
The most common approach to test team composition and formation algorithms is to perform a set of simulations showing the effectiveness of team methods. The majority of researchers simulating data use an abstract set of simulated tasks. Depending on the model, tasks can be static (Liemhetcharat & Veloso, Reference Liemhetcharat and Veloso2012; Okimoto et al., Reference Okimoto, Schwind, Clement, Ribeiro, Inoue and Marquis2015; Peleteiro et al., Reference Peleteiro, Burguillo-Rial, Luck, Arcos and Rodriguez-Aguilar2015; Crawford et al., Reference Crawford, Rahaman and Sen2016) or dynamic, that is, they can change over time (Chalkiadakis & Boutilier, Reference Chalkiadakis and Boutilier2012; Farhangian et al., Reference Farhangian, Purvis, Purvis and Savarimuthu2015a). Chen et al. (Reference Chen, Chen, Timsina and Soh2015) use both static and dynamic tasks to study various effects of considering agent openness and task openness in ad hoc team formation.
3.3.4.3 Empirical data
Collecting empirical data is time consuming, however, it is the most reliable way to validate team hypotheses and models. The data can be collected in real world (mostly robotics) or in virtual environments (such as online games). For many years, RoboCup has served as an excellent domain for testing teamwork, co-ordination, and co-operation. In 2013, a new competition began that serves as a testing environment for co-operation without pre-co-ordination: The Drop-in Player Competition. In this competition, instead of homogeneous teams of robots such as all robots are programmed to follow the same strategy, all robots are heterogeneous (originating from different RoboCup teams and as such running different software). Genter et al. (Reference Genter, Laue and Stone2017) present their findings from a three year experiment in the domain above that consisted of 38 games for a total playing time of 510 minutes that resulted in approximately 85 robot hours. The authors suggest improvements to the competition, and provide advice for organizing new competitive ad hoc teamwork evaluations.
An example of teamwork conducted in a virtual environment is Hanna and Richards (Reference Hanna and Richards2015), that use human-agent teams to assess the performance of teams. The results show the importance of designing agents capable of using multiple methods of communication with humans, as this tends to build SMMs with human users and improve team performance. Rokicki et al. (Reference Rokicki, Zerr and Siersdorfer2015) use a crowdsourcing scenario for a face recognition task where human agents (workers) are asked to identify a person on a given reference photo among a set of 10 test photos. The performance is evaluated by the quality of the final outcome of each team. Next, Andrejczuk et al. (Reference Andrejczuk, Rodriguez-Aguilar, Roig and Sierra2017) use an education scenario to pitch their automated team composition model with the team composition performed by experts. Authors compare both team composition models in terms of how well they predict team performance.
Many researchers use online games to do team performance studies. For instance, Marcolino et al. (Reference Marcolino, Jiang and Tambe2013) validate their hypothesis by using virtual agents playing 691 instances of the GO computer game, and Moon et al. (Reference Moon, Carley, Schneider and Shigiltchoff2005) analyze the behaviour of approximately 150 000 teams in America’s Army game. Finally, Wax et al. (Reference Wax, DeChurch and Contractor2017) use a free-to-play Web-based MMORPG game, called Dragon Nest, as a testbed to explore the attraction mechanisms that guide teaming in the virtual world and their implications for team performance. Their sample included 1568 players of the Chinese version of Dragon Nest who played on 1744 teams. Their results indicate that self-assembled teams form via three assembly mechanisms: homophily, familiarity, and proximity. They report mixed results when it comes to the relationship between team characteristics and team performance, although they establish that having a heterogeneity in skill levels on a team is good for team performance.
3.3.4.4 Analysis
In summary, the CS literature has focussed on team co-operation with various objectives that can be categorized as at least one of the following: minimizing overall cost, maximizing social utility, or maximizing team(s) performance. The models minimizing overall cost compose teams based on individual competences, though do not take into account individual motivations to complete the assigned task. This is a rather strong assumption, especially when it comes to mixed teams or human teams, making the existing approaches rather unrealistic. The literature focusing on maximizing social welfare considers both agent competences and motivation. The motivation increases by using competence mechanisms (like in crowdsourcing teams), or by giving agents the freedom to select their collaborators (like in learning agent team formation or in ad hoc teams). To maximize team performance, one of the crucial findings in both OP and CS is that team members must be heterogeneous. Further variables that have been used by computer scientists in the area of MAS to compose teams are: agent reputation, personality of humans and agents, synergy between team members, and feeling of fairness among team members. The proposed methods are validated using existing databases available online, data simulation, or empirical data.
3.4 HOW do we do it? The organization
In the existing literature, the societal structure of teams is considered crucial for effective teamwork. There are two aspects to be considered, one is which agents will be members of a team and second, how teams will be organized to solve tasks. Thus, the different approaches in the literature can be classified depending on the functionality that they tackle:
∙ Team composition: the process of deciding which agents will be part of a team. It can be an external decision or an autonomous decision by the agents themselves.
∙ Team formation: the process of learning to decide the roles and internal organization of a team. This organization can be imposed or be the result of self organization. In any case, the resulting organizations can be categorized as hierarchical or egalitarian.
Next, we look into these two dimensions in detail.
3.4.1 Team composition
Although team composition in MAS has mainly focussed on building teams of software agents, that is agent teams, there is a growing number of works considering either mixed teams (Hanna & Richards, Reference Hanna and Richards2015), where agents and humans co-operate to achieve common goals (Ramchurn et al., Reference Ramchurn, Wu, Fischer, Reece, Roberts, Rodden and Jennings2016), or human environments, where people are supported by software (Jennings et al., Reference Jennings, Moreau, Nicholson, Ramchurn, Roberts, Rodden and Rogers2014). In MAS, we distinguish between two groups of methods (or processes) to compose team(s), namely:
1. Exogenous team composition: there is an algorithm external to the agents that determines the composition of teams.
2. Endogenous team composition: agents themselves decide in a distributed manner the composition of a team.
3.4.1.1 Exogenous team composition
The team composition process uses the task requirements (i.e. constraints on teams that can be formed, such as team size (Rahwan et al., Reference Rahwan, Michalak, Elkind, Faliszewski, Sroka, Wooldridge and Jennings2011); competences and personality as discussed in Section 3.1) in order to build teams that are capable of solving the task with particular properties. For instance, Crawford et al. (Reference Crawford, Rahaman and Sen2016) and Okimoto et al.(Reference Okimoto, Schwind, Clement, Ribeiro, Inoue and Marquis2015) consider a degree of fault-tolerance to build k-robust teams. A team is k-robust if removing any k members from the team, does not affect the completion of the task. As mentioned before, Liemhetcharat and Veloso (Reference Liemhetcharat and Veloso2012) propose a learning algorithm that constructs a synergy graph from observations of the performance of pairs and triples of agent. A synergy value represents how well a pair of agents work together. The authors use this learned synergy graph as well as agent capabilities to solve the team composition problem. Their method selects teams that are capable and that maximize their internal synergy.
Similarly, Rangapuram et al. (Reference Rangapuram, Biihler and Hein2015) consider the competences of agents and their compatibility in order to identify a team that is both competent and compatible. Agent compatibility, expressed as a social network, can be understood as a set of preferences on team composition, such as: the inclusion of a certain team leader, or restrictions on team size, problem solving cost or agent locality (in a social or geographical sense).
In many systems, capabilities of agents are not widely known. Chen et al. (Reference Chen, Chen, Timsina and Soh2015) study an ad hoc setting where agents need to co-operate to recognize their capabilities. Agents bid for subtasks (parts of tasks) that they want to perform, though the final decision belongs to the exogenous algorithm that assigns each subtask to the best qualified agent bidding on the task.
Some approaches deal with the composition of multiple teams. For instance, Anagnostopoulos et al. (Reference Anagnostopoulos, Becchetti, Castillo, Gionis and Leonardi2012) use competences and communication cost in a context where tasks sequentially arrive and teams have to be composed to perform them. Each task requires a specific set of competences and the team composition algorithm is such that the workload per agent is fair across teams. Furthermore, Andrejczuk et al. (Reference Andrejczuk, Rodriguez-Aguilar and Sierra2016) compose multiple teams according to a balance of agents’ personalities and genders. Their goal is to partition a set of agents into multiple teams such that each team is internally balanced and the problem solving capabilities of the teams in the partition are similar. Besides the use of personality traits, Farhangian et al. (Reference Farhangian, Purvis, Purvis and Savarimuthu2015a) use competences and a task specification with the purpose of composing a single team.
Aside from competences and personality, team composition can also take into account agents’ preferences on teams. Indeed, hedonic coalition formation employs each agent’s hedonic preferences on its coalitions to yield a coalition structure, namely multiple teams. The defining feature of a hedonic preference is that every agent only cares about which agents are in its own team (coalition). Spradling et al. (Reference Spradling, Goldsmith, Liu, Dadi and Li2013) introduce a new model of hedonic coalition formation game, the so called RTHG. In this model, agents view coalitions as a number of available roles and have two levels of preferences: on the set of roles that are available in a coalition, and on their own role within each coalition.
Finally, there is relevant work on mixed teams by Hanna and Richards (Reference Hanna and Richards2015), which composes a team as a pair consisting of a human and an IVA. The pair plays a collaborative game that involves passing a sequence of obstacles to reach a target.
3.4.1.2 Endogenous team composition
The second group of methods for organizing teams has an endogenous nature. They incorporate algorithms enabling agents to decide on team composition by themselves. In detail, agents are equipped with negotiation and decision-making mechanisms that they employ to agree among themselves on a team structure. Therefore, team composition occurs without explicit external command.
Farhangian et al. (Reference Farhangian, Purvis, Purvis and Savarimuthu2015b) propose a model in which there are two types of agents: requesters in charge of tasks that seek for contributors to compose teams, and contributors that vote for the tasks they want to perform. Each requester runs an auction-based (first-price sealed-bid) algorithm with the purpose of composing teams with the highest chance to increase social wealth. Contributors issue bids pursuing to join the most useful requesters, namely the ones that are most likely to reward them. Peleteiro et al. (Reference Peleteiro, Burguillo-Rial, Luck, Arcos and Rodriguez-Aguilar2015) follow the similar approach but also employ reputation and adaptation mechanisms to allow agents in a competitive environment to autonomously join and preserve teams (as coalitions). Agents bid for tasks and each team is constructed and led by a mediator agent.
Similarly, in Chalkiadakis and Boutilier (Reference Chalkiadakis and Boutilier2012) each agent builds its beliefs about its peers based on prior outcomes of interactions between them, and decides on coalitional actions (which coalition to join and what task to perform). Then, agents negotiate between them to form teams taking into account their own beliefs on the probability of success when being in a team.
Another interesting scenario for endogenous team composition is gaming. For instance, Moon et al. (Reference Moon, Carley, Schneider and Shigiltchoff2005) analyze factors affecting team success in the America’s Army game. There, teams can have up to fourteen players and human agents are allowed to join teams freely at the beginning of each game. However, the authors discover that the most successful team configuration has 10 soldiers, moving in two sub teams (five players in each), and a long chain of communication (rather than star-shaped communication). Note that these findings actually indicate that two teams of five are more effective than one team of ten. This aligns with the team size recommendations from OP that we discuss further ahead in Subsection 4.4.1.
There exist also mixed approches, where researchers explore both, exogenous and endogenous methods to compose teams. For instance, Rokicki et al. (Reference Rokicki, Zerr and Siersdorfer2015) propose strategies for groupsourcing (team-based crowdsourcing), ranging from team formation processes where individuals are randomly assigned to teams, to strategies requiring self-organization where individuals participate in team building. Their results show that balanced teams (i.e. teams with the balanced number or agents in each team) combined with individual rewards for most effective team members outperforms the other strategies.
3.4.1.3 Analysis
The majority of researchers focusses on exogenous methods to compose teams. However, there are many actual-world application domains (e.g. co-working, or crowdsourcing) where endogenous team composition and formation are more appropriate for deployment.
Most of the literature on exogenous team composition assumes that there exists a centralized, detailed knowledge about all agents. This knowledge is required in order to compose teams based on agents’ capabilities, personality, or even preferences. Endogenous methods are best for dynamic environments, where team composition and formation processes are continuously performed. Furthermore, it is a good setup for agents that learn other agents’ capabilities through repeated interactions.
3.4.2 Team formation
We identify two main team organization structures to build effective teams:
1. Hierarchical.
2. Egalitarian.
We describe each team organization structure in the following sub-sections.
3.4.2.1 Hierarchical
A hierarchical structure considers a team leader who is responsible for and makes the decisions affecting the team. This structure is the traditional setting when it comes to business units.
As mentioned in Subsection 3.4.1, Farhangian et al. (Reference Farhangian, Purvis, Purvis and Savarimuthu2015b) consider two types of people within teams: requesters and contributors. Requesters adopt a leading function, they start a project and recruit the required people. Contributors perform the tasks assigned by requesters. The overall team behaviour is determined by the personality of agents in teams.
In Peleteiro et al. (Reference Peleteiro, Burguillo-Rial, Luck, Arcos and Rodriguez-Aguilar2015), each coalition is led by a mediator. This agent is responsible for leading a coalition by selecting suitable agents to be part of a coalition (called worker agents) and by evaluating the performance of workers while the coalition operates.
Agmon et al. (Reference Agmon, Barrett and Stone2014) consider ad hoc settings with two types of agents: best-response agents and ad hoc agents. In such settings a task consists of a set of actions, and each team becomes responsible for performing a task. Each best-response agent selects its next action based on its own local world view. Each ad hoc agent acts to bring out the best in its teammates by ‘leading’ them to the optimal joint action. This is an arresting example of a hierarchical structure, where agents are not aware of each other’s roles, and hence of a team’s structure. Nonetheless, an ad hoc agent has more knowledge than a best-response agent, and thus it exploits such information to lead its team. This may happen in a business setting, where both senior and junior staff form a team. Even though there is no clear division of roles, the senior employee uses his experience to make decisions that are best for the team in a long-term period (and may not look best from a short-time perspective).
3.4.2.2 Egalitarian
An egalitarian structure assumes that all workers in a team are equally informed and have the same rights. The leadership within a team is shared and existing team roles result from the team’s task requirements. An example of this structure in real-life scenario might be a team of doctors that need to join their specialized knowledge to perform a complicated surgery on a patient.
A large part of the MAS literature focusses on the egalitarian setting, trying to benefit from leaderless teams that co-operate to complete tasks. We find this team structure in groupsourcing (Rokicki et al., Reference Rokicki, Zerr and Siersdorfer2015), robust teams (Okimoto et al., Reference Okimoto, Schwind, Clement, Ribeiro, Inoue and Marquis2015; Crawford et al., Reference Crawford, Rahaman and Sen2016), ad hoc teams (Chalkiadakis & Boutilier, Reference Chalkiadakis and Boutilier2012; Barrett et al., Reference Barrett, Stone, Kraus and Rosenfeld2013; Chen et al., Reference Chen, Chen, Timsina and Soh2015), mixed teams (Hanna & Richards, Reference Hanna and Richards2015), learning teams (Liemhetcharat & Veloso, Reference Liemhetcharat and Veloso2012, Reference Liemhetcharat and Veloso2014) or online teams (Moon et al., Reference Moon, Carley, Schneider and Shigiltchoff2005).
A particular case of egalitarian structure involves members that decide collectively, usually by voting, on the appropriate course of action while performing an assigned task. The real life example for this organization structure might be a start-up with few people that make all decisions by discussion. Marcolino et al. (Reference Marcolino, Jiang and Tambe2013), Nagarajan et al. (Reference Nagarajan, Marcolino and Tambe2015), and Marcolino et al. (Reference Marcolino, Xu, Gerber, Kolev, Price, Pantazis and Tambe2016) study egalitarian structures whose agents vote to decide at every step of a task in order to choose the best course of action. They prove that teams consisting of heterogeneous agents that vote their actions are more efficient than homogeneous teams built out of the copies of the strongest agent in a team. This is because the spectrum of possible actions is wider for heterogeneous teams.
There exist also some team composition models that can produce both types of team structures. For instance, in RTHG model (Spradling et al., Reference Spradling, Goldsmith, Liu, Dadi and Li2013), the resulting structure of the teams can be either hierarchical or egalitarian depending on the relationships between roles. Typically teams in Rangapuram et al. (Reference Rangapuram, Biihler and Hein2015) are egalitarian, though the presented model includes many natural requirements that can lead to a hierarchical structure (such as inclusion of a designated team leader and/or a group of given experts).
Finally, one important question regarding team organization requires our attention, that is, what is the effect of team network and communication structure on team performance? We already discussed the article of Hanna and Richards (Reference Hanna and Richards2015), where authors show that the more informative the communication between two agents, the better the performance of the team. This result is consistent with results reported by Sukthankar et al. (Reference Sukthankar, Sycara, Giampapa and Burnett2009). There, the authors analyze the communication patterns of teams performing a collaborative search game that simulates search and rescue scenarios. Sukthankar et al. (Reference Sukthankar, Sycara, Giampapa and Burnett2009) robustly find that the less performing teams are those that communicate less. Furthermore, Moon et al. (Reference Moon, Carley, Schneider and Shigiltchoff2005) also highlight the importance of communication, as well as team movement structure on team performance. Regarding communication networks, two dominant communication network types are: star-shaped and long-chain shaped. Between these two, the long-chain-shaped communication network performed better because it reduces team members’ burden to communicate. However, the reduced communication frequency of the long-chain-shaped communication network teams with respect to star-shaped communication network teams is still higher than that of losing teams. Regarding team movement, the authors found that the most effective communication network type is a dense network (team members stay close together), and that a network with two dense subgroups has fewer casualties and less communications than others but a satisfying number of opponents being killed. Maghami and Sukthankar (Reference Maghami and Sukthankar2011) introduce an agent-based simulation for exploring the effects of stereotypes on task-oriented team composition and network evolution. The authors demonstrate that stereotype value judgements can have a negative impact on task performance, even if the agents are motivated and competent enough to perform a task. Stereotype-driven agents modify the social network from which teams are formed in a systematically suboptimal way and eliminate the skill diversity required for successful task performance. Osipov and Sukthankar (Reference Osipov and Sukthankar2012) explore the relationship between network adaptation for candidate team participants and performance of problem-solving teams. Their analysis shows that the use of a more detailed description of the agent skills (i.e. a higher number of skills per agent) is desirable as it has a net positive effect on the number of candidate teams (where an agent can contribute its skills) and the total number of teams that can be composed by a system. However, the authors do not provide a detailed, analytical treatment of the relationship between the network adaptation policies and the teams’ performance.
3.4.2.3 Analysis
The team organization structures in the MAS literature can be grouped into hierarchical and egalitarian. The majority of MAS research focusses on egalitarian structures because of simplicity reasons. In particular, there is no need for defining a role structure together with its relationship and agent-role assignments. Although structuring teams and organizations largely helps reduce complexity of interactions, by separating responsibilities, most research in team formation does not consider a clear role division. Moreover, notice that in most business settings teams work following a hierarchical structure. Finally, research suggests that teams communicating more have higher levels of performance up to a point. However, too excessive communication leads to lower levels of performance.
3.5 WHEN do we do it? The dynamics
The literature on team composition and formation mostly considers that tasks are static in the sense that their requirements do not change during their execution. However, the dynamics of task arrival is considered by many. That is, there could be multiple tasks to be solved concurrently and new tasks may arrive in an asynchronous, localized manner. The different works consider different issues in this dynamic process. For instance, the number of tasks to be serviced, task and team members localization, team size per task or time limitations. Normally, if there is only one task is to be completed, the focus will be on composing the best team for the task. On a repeated task arrival setting, the use of a history of team work experiences is key to compose new teams. Hence, the literature can be classified depending on two main aspects:
1. The succession of tasks.
2. The simultaneity of tasks.
The simplest case is a one-shot task. There is neither succession nor simultaneity, and hence the problem of team composition is normally reduced to finding the best team for the only task. When tasks come in sequence without simultaneity, then the problem can be reduced to finding the best team for each task while using the learned experiences in the composition of each new team. If tasks come in succession and can be simultaneous, the need to deal with multiple teams acting at the same time becomes a key issue. The succession of possibly simultaneous tasks is the most complex framework in which memory becomes again a key element.
We discuss each aspect in detail.
3.5.1 Non-successive and non-simultaneous tasks
In this case we face a one-shot task resolution. This is the simplest case for the team composition and formation problems. There is no long-term strategy used to compose and form teams. Thus, agents do not learn from past experiences and we cannot talk about the notion of community in this set-up.
3.5.1.1 Team composition
As mentioned above, in the team composition problem, we are looking for only one team, the best possible one to perform the task. The majority of models that consider non-successive and non-simultaneous tasks are simplistic. They assume that once the team is composed it has the needed properties and will perform the task well. For instance, Kargar et al. (Reference Kargar, An and Zihayat2012) use agents’ capabilities and team co-ordination cost to compose the most effective team. Similarly, Crawford et al. (Reference Crawford, Rahaman and Sen2016) and Okimoto et al. (Reference Okimoto, Schwind, Clement, Ribeiro, Inoue and Marquis2015) use agents’ capabilities to compose k-robust teams (see Section 3.4.1 for a definition of a k-robust team). In Rangapuram et al. (Reference Rangapuram, Biihler and Hein2015), besides agents’ capabilities, the team composition model also introduces various types of constraints (the inclusion of a specific group of agents in a team, team size, budget limitations, and maximum geographical distance between agents and between agents and tasks). This last model is more realistic, though it disregards past experiences.
3.5.1.2 Teamwork
In the teamwork phase, agents solve the task once and for all. Hence, one-shot tasks may cause self-interested behaviours, such as in Rochlin et al. (Reference Rochlin, Aumann, Sarne and Golosman2016). There, as mentioned in Section 3.2.2, one agent (called buyer) from the team is delegated to accomplish the task of purchasing a jointly desired item with the lowest possible cost. This agent operates on a one-time setting, that is, there is a single agent deciding on behalf of the team, and hence, there is no need for that agent to behave in an altruistic manner. Authors study the notion of fairness and its influence on effectiveness. They show that the selected buyer is less motivated to do the task if the cost of the goods is to be divided equally among the team members. In this case, the purchasing costs are fully assumed by the purchasing agent. Therefore, they study different methods to reimburse the purchasing costs incurred by the buyer to improve its effectiveness.
Hanna and Richards (Reference Hanna and Richards2015) study the co-operation between a human and an IVA in a one-shot task setting. Given that past experiences cannot be used, they experimentally show, by comparing many one-shot task instances, that the more informative the communication between the two agents, the better the performance of the team. The communication behaviour of an IVA is directly related to its psychological traits.
On a different vein, many models assume that given a one-shot task, agents will behave according to their knowledge and capabilities in order to benefit the whole team. In Barrett et al. (Reference Barrett, Stone, Kraus and Rosenfeld2013) and in Agmon et al. (Reference Agmon, Barrett and Stone2014), team agents are pre-designed to co-operate when solving a collective task. Then, one of the agents is replaced by an ad hoc agent that shares the team’s goals, though does not know its teammates’ behaviours. The ad hoc agent cannot control its teammates, and yet it tries to improve the team’s performance by learning to predict other agents’ actions and thus selecting its own actions to achieve an overall optimal team behaviour. Marcolino et al. (Reference Marcolino, Jiang and Tambe2013) and Nagarajan et al. (Reference Nagarajan, Marcolino and Tambe2015) perform a one-shot task study, where team agents vote for a team action leading to the task resolution. The action voted for is sampled from a fixed probability distribution over those actions appropriate in a particular world state (no learning involved). The higher the probability of an action the more preferred it is by the agent. A plurality voting mechanism is used to select the team action. Authors show that a diverse team (with different probability distributions) can outperform a uniform team (made out of copies of the best agent) and that breaking ties in favour of the best agent’s opinion in a diverse team is the optimal voting ruleFootnote 1 .
3.5.2 Non-successive and simultaneous tasks
In non-successive and simultaneous tasks, the composition and formation problem becomes more complex as it now considers a set of one-shot tasks. There is still no use of the past experiences as the tasks are non-successive.
3.5.2.1 Team composition
Researchers in the area of MAS propose algorithms to compose the best set of teams, one per simultaneous task, instead of looking for the best team for a task. For instance, Andrejczuk et al. (Reference Andrejczuk, Rodriguez-Aguilar and Sierra2016) partition a set of agents into competent, gender- and psychologicallybalanced problem-solving teams of a given size that have to solve a given task. The authors use a greedy technique to balance the psychological traits of the members of teams so that each team gets the full range of problem-solving capabilities.
In RTHG (Spradling et al., Reference Spradling, Goldsmith, Liu, Dadi and Li2013) authors propose a heuristic optimization method to partition a set of agents, again to solve different instances of the same task. The method treats as votes agents’ role preferences on team role structures. First, the role structures of the teams will be those receiving the highest social welfare (as the summation of the agent individual utilities to play any of the roles in the structure). Second, the algorithm selects the agent with the highest utility for a remaining role in the most voted team role structure, recomputes the role structure preferences without that agent’s preferences, and keeps staffing teams until the partition is complete. For instance, an agent may prefer to be a programmer in a two-agent team including a designer, but would not like to play any role in a team without a designer. Hence, an agent’s role preference is not taken in isolation, but in the context of the teams’ composition. Authors define Nash stable and individually stable solutions for RTHG in terms of possible local moves that agents could make within a given coalition partition and prove that every instance of RTHG has an individually stable partition that can be obtained with the use of local search movements (change of role within a coalition or coalition swaps). In our literature search, we could not find approaches dealing with different simultaneous non-successive tasks.
3.5.2.2 Teamwork
Similarly to team composition, Rokicki et al. (Reference Rokicki, Zerr and Siersdorfer2015) deal with the Teamwork problem over different and simultaneous instances of the same task. Agents may change their strategy during team formation in order to reach a better solution. They classify human behaviour during team self-organization in crowdsourcing tasks in two types. First, a number of users choose to join one of the leading teams, instead of selecting a weaker one and compete for a lower award. Second, small teams merge to form stronger teams and thus have a higher chance of achieving an award.
3.5.3 Successive and non-simultaneous tasks
When tasks are successive and non-simultaneous, the algorithms for team composition and formation deal with a task that has to be assigned to a team, and in many cases solved, before new tasks arrive. A successive setting can discover phenomena which we believe are important, but which are not captured when the attention is limited to static, non-successive tasks. If in the system of the same set of agents, teams are created and dismantled depending on the task, the agents may behave very differently than in a non-successive settings. For instance, a person will behave in a different manner if she repeatedly borrows a car from her friends, than when she simply rents a car. The successive setting has its advantages: it lets agents learn from the past experiences and build their beliefs based on this knowledge.
3.5.3.1 Team composition
In Anagnostopoulos et al. (Reference Anagnostopoulos, Becchetti, Castillo, Gionis and Leonardi2012), the first task arrives at the first time step and is assigned to a newly composed team of experts before the arrival of the second task. This procedure repeats until all tasks are assigned. Authors propose an algorithm to compose a set of teams to handle a set of these incoming tasks. The goal is to form a new competent team upon arrival of each task, so that the workload in the whole system is balanced. There is no learning involved in this process. Contrarily, in Liemhetcharat and Veloso (Reference Liemhetcharat and Veloso2012) a learning algorithm is proposed that constructs a synergy graph from observations of the performance of pairs and triples of agent in solving previous tasks. The synergy tells how well a pair of agents work together and they use this learned synergy graph as well as agents’ capabilities to solve the team composition problem for the next task. Their method selects teams that are capable and maximize their internal synergy.
3.5.3.2 Teamwork
To the best of our knowledge, there are no contributions on teamwork that consider successive and non-simultaneous tasks.
3.5.4 Successive and simultaneous tasks
When tasks are successive and simultaneous, the algorithms for team composition and formation deal with a set of tasks arriving, possibly overlapping in time that have to be assigned to newly composed teams.
3.5.4.1 Team composition
In Farhangian et al. (Reference Farhangian, Purvis, Purvis and Savarimuthu2015b), tasks arrive in any order, possibly overlapping in time. A team is composed for each incoming task and after execution agents assign performance values to each one of the other team members. These values are public and used by the community to compose teams for future tasks. Chalkiadakis and Boutilier (Reference Chalkiadakis and Boutilier2012) present several learning algorithms to approximate the optimal Bayesian solution to the repeated team composition. Similarly, Peleteiro et al. (Reference Peleteiro, Burguillo-Rial, Luck, Arcos and Rodriguez-Aguilar2015) compute, after teamwork, both individual agent and coalition (team) reputation values to be used in the composition of future teams.
Finally, in Chen et al. (Reference Chen, Chen, Timsina and Soh2015), for each new task arriving agents decide which team to join balancing exploitation (rewards from completing tasks learned from previous task solving) and exploration (learning opportunities from more qualified agents leading to future rewards).
3.5.4.2 Teamwork
To our knowledge, there are no contributions considering successive and simultaneous teamwork.
3.5.4.3 Analysis
One time settings (i.e. non-successive tasks) are usually simplified models that do not take into consideration the history of agent interactions. One-shot tasks may cause self-interested behaviours, where agents look for at least a fair split of costs associated with teamwork. However, the majority of the literature on team composition and teamwork considering this setting assume that the agents will always behave accordingly to their capabilities and knowledge. The successive tasks provide us with more realistic and complex scenarios. The tasks arrive either in order, one after another, or overlapping in time. The majority of the literature uses this setting to let agents build their beliefs based on the past experiences and compose new teams according to these beliefs. Regarding teamwork, there are no contributions that explore successive settings. In other words, the state of the art does not acknowledge the memory of agents as important while executing tasks.
3.6 WHERE do we do it? The context
The context is understood as the circumstances that form the setting for the team composition and formation processes. We observe that the concept of context in the reviewed CS literature has not played a major role so far. Contrarily, according to the OP literature (Guzzo & Dickson, Reference Guzzo and Dickson1996), it is one of the most important variables while composing and forming teams (see Section 4.6). There are different categorizations of context. One of them is proposed by Kozlowski and Bell (Reference Kozlowski and Bell2013), which classifies contexts as follows:
∙ Organizational context: technology used, organization structure, leadership, culture, and climate.
∙ Team context: normative expectations, shared perceptions, and compatible knowledge (generated by and emerge from individual interactions).
∙ Individual context: attributes, interactions, and responses.
In the MAS literature there are very few works that consider the social context while composing teams. Terveen and McDonald (Reference Terveen and McDonald2005) set a framework for social matching systems, which aims to bring people together on both physical and online spaces. They explain the importance of context in recommending a member of social network for collaboration. In Rangapuram et al. (Reference Rangapuram, Biihler and Hein2015), while composing teams, the context is exemplified as a social network that encodes the previous collaborations among experts. The idea behind it is that the teams that have worked together previously are expected to have less communication overhead and work more effectively as a team. Similarly, Peleteiro et al. (Reference Peleteiro, Burguillo-Rial, Luck, Arcos and Rodriguez-Aguilar2015) propose to express social context by the reputation measure. There, upon task completion, the contractor rates the quality of the service provided by a team and, also teams rate their own workers. Finally, this rating information is maintained and aggregated by a reputation module. Liemhetcharat and Veloso (Reference Liemhetcharat and Veloso2012) propose to model a social context by using the learned synergy graph (that measures how well agents work with one another) and hence, solve the team composition problem. Anagnostopoulos et al. (Reference Anagnostopoulos, Becchetti, Castillo, Gionis and Leonardi2012) include the co-ordination costs by means of a social network over the set of agents and assume a metric distance function on the edges of the network. On top of modelling preferences based on social context (such as past interactions, compatibility in collaborating, distance in a company’s hierarchy), the function may include any other kind of context (for instance, geographical proximity between agents or between task and agents within a team).
3.6.1 Analysis
To the best of our knowledge, there are only few works in MAS literature that recognize the context as an important variable. Besides Anagnostopoulos et al. (Reference Anagnostopoulos, Becchetti, Castillo, Gionis and Leonardi2012), which considers both social and geographical contexts, the methods in the literature only consider the social context (if analyzed at all).
4 Team composition and formation from an organizational psychology perspective
In this section, we discuss all above aspects in detail answering the questions asked in the introduction of this paper.
4.1 WHO is concerned?
In this section, we are going to survey the literature on OP that deals with the characteristics of humans composing teams.
We will use the structure as in Section 3.1.
4.1.1 Capacity
In OP, the most important capacity of team members that is related to team performance is their cognitive ability. Hence, the main goal is to study how cognitive abilities influence team performance. Cognitive ability refers to the ‘capacity to understand complex ideas, learn from experience, reason, solve problems, and adapt’ (Devine & Philips, Reference Devine and Philips2001: 507). Hence, cognitive ability in OP is a much wider concept than capacity in MAS as on top of skills widely used in MAS systems, it contains many other properties such as experience, competences, age, or even gender.
Moreover, in contrast to CS, where capabilities are static, psychologists deal with the dynamism of human capacity. Humans learn new capabilities and increase their level every day for whole live (Laal and Salamati, Reference Laal and Salamati2012: 399–403] for the concept of the lifelong learning). There are diverse tests and methods to examine humans capacity, such as: intelligence or cognitive competences tests, assessment centres or social and behavioural competence tests. Regarding team composition, on the one hand, Bell and Devine and Philips (Reference Devine and Philips2001) found that mean team values of cognitive ability are correlated with team performance. Moreover, she also found that the lowest and the highest team members’ cognitive abilities are correlated with team performance in lab and field settings. In addition, Devine and Philips (Reference Devine and Philips2001) found that the variance of team members’ cognitive ability did not help predict team performance. These authors also found that the mean value is twice more informative in predicting than the lowest and the highest member’s scores. On the other hand, Devine and Philips (Reference Devine and Philips2001) found that cognitive ability influences team performance differently depending on contextual variables (such as working normative procedures or human resources policies). These findings suggest that, when composing a team, organizations and managers should not only take into account the members’ cognitive ability, but also the context in which the team will operate. This will be further discussed in Section 4.6.
Woolley et al. (Reference Woolley, Aggarwal and Malone2015) discuss the existence of a measurable collective intelligence in teams that is analogous to individual intelligence. Authors suggest the existence of a general collective intelligence factor that explains a team performance on a wide variety of tasks. Woolley et al. (Reference Woolley, Chabris, Pentland, Hashmi and Malone2010) show that collective intelligence is correlated with the average social sensitivity of group members, the equality in distribution of conversational turn-taking, and the proportion of females in the group. In STEM (Science, Technology, Engineering and Math) teams, gender diversity can enhance group processes, which are increasingly important as collaboration becomes a centre piece in the production of science. The enhancement of group processes and higher levels of collective intelligence can, in turn, lead to greater innovation and scientific discovery (Bear & Woolley, Reference Bear and Woolley2011). Finally, similarly to findings in the CS literature, the concept of team properties is normally understood as a sum of humans’ individual properties.
4.1.2 Personality
In addition to the before-mentioned individual properties, the literature has examined the role of personality. The most prominent approaches have been the ‘Big Five’ personality traits theory (Mount et al., Reference Mount, Barrick and Stewart1998), Schutz’s theory of fundamental interpersonal relations orientation (FIRO) (Schutz, Reference Schutz1958) and the MBTI method (White, Reference White1984). They have been used to find the personality traits and types associated with team performance. Regarding the ‘Big Five’ theory, meta-analytic research has found that certain levels of conscientiousness, openness to experience, and agreeableness are good performance predictors (Mount et al., Reference Mount, Barrick and Stewart1998).
Another approach is that of the theory of FIRO (Schutz, Reference Schutz1958). The idea is that humans have several needs (i.e. need for inclusion, control, and affection) and that groups with team members that have compatible needs will perform better than those with incompatible ones. Nevertheless, research has found mixed support for this theory (West, Reference West2012b).
Some companies have also tried to base their team formation on cognitive styles of the members, by using the MBTI assessment instrument—(Myers et al., Reference Myers, McCaulley, Quenk and Hammer1998), which is a questionnaire that measures cognitive styles along four dimensions: Extraversion–Introversion, Sensing–Intuition, Thinking–Feeling, and Judging–Perceiving. Nevertheless, there is not enough rigorous research evidence showing its relationship with team performance (West, Reference West2012b).
There are also novel approaches created with the purpose of team composition and formation. For instance, the Post-Jungian Personality Theory, which is a modified version of MBTI (Wilde, Reference Wilde2013). It operates on the same dimensions as MBTI. The main novelty of this approach is its use of the numerical data generated by the instrument (Wilde, Reference Wilde2011). The results of this method seem promising as within a decade this novel approach tripled the fraction of Stanford teams awarded national prizes by the Lincoln Foundation (Wilde, Reference Wilde2009). However, the method is not properly validated and tested, which makes it disregarded by psychologists.
4.1.3 Analysis
Several correlations have been found between cognitive ability and team performance. The personality is also present while composing teams, although the correlation between personality and team performance is not clearly explained. The most widely used test to measure personality is the ‘Big Five’. OP studies show that besides cognitive ability and personality, experience, and gender are further properties to consider for team composition (West, Reference West2012a). Indeed, research findings on this topic suggest that diversity in those characteristics can have an effect on team performance and innovation (West, Reference West2012a). Additionally, some further research has also paid attention to values and has found collectivism and teamwork preferencesFootnote 2 to be additional good team performance predictors (Bell, Reference Bell2007).
4.2 WHAT is the problem?
When it comes to team composition, the OP literature has focussed on defining task classifications. These classifications have been employed to study the relation between task types and team performance. Hence, in this section we will review the most known task classifications and its influence on team performance.
Two of the most widely discussed task classifications are those of McGrath (Reference McGrath1984), Hackman (Reference Hackman1990), Hackman and Lawler (Reference Hackman and Lawler1971), and Hackman and Oldham (Reference Hackman and Oldham1975). While the classification of McGrath (Reference McGrath1984) is based on the cognitive requirements of tasks, the classification in Hackman (Reference Hackman1990), Hackman and Lawler (Reference Hackman and Lawler1971), Hackman and Oldham (Reference Hackman and Oldham1975) is based on the motivation characteristics of tasks (i.e. autonomy, task variety, task significance, task identity, and task feedback). The research on team composition show that the classification based on the motivation characteristics predicts more accurately the team performance (Podsakoff et al., Reference Podsakoff, MacKenzie and Michael1997).
Hackman (Reference Hackman1990) defines a task classification based on motivational requirements composed by seven work task types:
1. top management;
2. task force;
3. professional support task;
4. performing task;
5. human service task;
6. customer service task;
7. production task.
The classification of McGrath (Reference McGrath1984) based on cognitive requirement proposes three dimensions that characterize each task type:
1. choose-execute;
2. conceptual-behavioural;
3. conflict-co-operation.
Technically speaking each task type becomes a 3-tuple with qualitative values for each dimension. For instance, a routine task would be very executive, medium behavioural, and low conflicting.
After analyzing seventeen classifications in the literature Wildman et al. (Reference Wildman, Thayer, Rosen, Salas, Mathieu and Rayne2012) came out with a different classification as follows:
1. managing others;
2. advising others;
3. human service;
4. negotiation;
5. psychomotor action;
6. defined problem solving;
7. ill-defined problem solving.
As an alternative perspective, Navarro et al. (Reference Navarro, Quijano, Berger and Meneses2011) propose a task classification based on the task context (namely task complexity, interdependencies between subtasks in a task, and uncertainty about the dynamics of the environment where the task is executed and the lack of information). Their results show that in order to achieve acceptable performance, the greater the complexity, interdependence, and uncertainty, the stronger the requirements on the maturity of teams (e.g. joint experience, cohesion) and on the diversity of team members’ capabilities. For instance, to carry out highly interdependent tasks, all team members should possess co-ordination skills (maturity) and some of them the capacity to take decisions (diversity). Taking into account other task context characteristic (i.e. uncertainty and interdependence) their study results show, the greater the uncertainty and interdependence of task types, the more diverse the competences for team members to cope with complexity. On the other hand, if the team is overqualified for the task to perform, the motivation of team members decreases and the quality of the outcome is lower or the task is not completed at all.
4.2.1 Analysis
The OP literature provides many different classifications of task types, where the most important are the classifications based on the motivation of individuals, the cognitive abilities and the task context. Provided the amount of classifications and the apparent lack of consensus among them, we believe that choosing among the several classifications previously presented in order to apply them to the study of team composition is a hard decision. Nevertheless, such decision must be made in order to move forward with the understanding of how a task type can influence team composition. In an attempt to advice researchers, notice that the research show that the classification based on the motivation characteristics predicts more accurately team performance.
From OP perspective team performance cannot be assessed by simply measuring how long it takes for the group to finish a certain task or by counting the number of right answers to predefined and clear questions, which is a common approach in CS. OP rather analyzes joint team objectives and the team composition and formation setting (such as not realistic deadlines, a number of individuals in a team, the level of stress in a team, or the quality of the outcome).
The current research on OP focus has moved from task analysis so not many results are present. Although task types are defined, different task instances constantly appear because of technological development. That makes it very difficult to keep the pace. That is why the focus on OP moved to competences (understood as cognitive ability, see Section skills2). This is why not much work has appeared after defining task taxonomy. At the same time task complexity increased and hence, teams are getting more and more important. Moreover, a clear mapping between cognitive ability of individuals and task types is needed. As a major benefit such mapping would ease team composition.
4.3 WHY do we do it?
In OP the main objective for team composition and formation is to maximize team performance. When measuring it, the research on OP suggests that we should go beyond mere economic criteria, the quality of decision-making processes or other traditional performance indicators (Hackamn, 2002; Komaki, 2009).
An important difference with respect to the CS literature is that team performance is considered from two perspectives: objective and subjective. On the one hand, objective team performance refers to the features of the outcome of a team (e.g. quality, delivery time, cost, sustainability). On the other hand, subjective team performance refers to the quality of human resources in a team (e.g. motivation, satisfaction, commitment, illness rate, stress) (Quijano et al., Reference Quijano, Navarro, Yepes, Berger and Romeo2008). Therefore, while the first one refers to the delivered output of a team (what customers obtain), the latest one focusses on the inner development of team members. Objective and subjective team performance are significantly correlated (e.g. Quijano et al., Reference Quijano, Navarro, Yepes, Berger and Romeo2008). Therefore, and not surprisingly, the OP literature considers both types of performances when tackling team composition and team formation (e.g. Quijano et al., Reference Quijano, Navarro, Yepes, Berger and Romeo2008). The subjective and objective performance of a team are determined by the several aspects of the context (discussed in Section 4.6), together with individual characteristics, the task and the team processes. Following Navarro et al. (Reference Navarro, Quijano, Berger and Meneses2011) the subjective and objective performance of a team are determined by the adjustment between the maturity level of the team (e.g. in terms of group development, potential, etc.) and the groups tasks characteristics.
4.3.1 Analysis
An important difference with respect to the CS literature is that team performance is considered from two perspectives: objective and subjective. Objective and subjective team performance are significantly and directly correlated. Therefore, and not surprisingly, the OP literature considers both types of performances when tackling team composition and team formation. The CS literature can benefit from the concept of subjective team performance that currently disregarded. Therefore, current team composition models, which mainly focus on the objective team performance, need to be extended.
4.4 HOW do we do it? The organization
Similarly to Section 3.4 on CS, we divide the organization into two aspects: team composition and team formation.
4.4.1 Team composition
The OP research on team composition has been very influenced by task classification. For several authors, there is a relationship between task type and team type (structure). For example, according to Hackman (Reference Hackman1990), there are seven team types based on the task type to perform:
1. top management;
2. task force;
3. professional support;
4. performing groups;
5. human service;
6. customer service;
7. production teams.
Devine (Reference Devine2002) and Delgado Piña et al. (Reference Delgado Pña, Romero Martinez and Gomez Martinez2008) highlighted that team performance depends on a good matching between team type and task type.
On the other hand, there are multiple team type classifications in the literature based on other criteria (Gibson and Kirkman, Reference Gibson and Kirkman1999; Marks et al., Reference Marks, Mathieu and Zaccaro2001; Devine, Reference Devine2002): motivation-based, cognitive-based, or context-based (see Section 4.2), though none of them has been widely used or accepted. Also, there is agreement that team diversity must be exploited while composing teams. Diversity refers to the degree or level to which the members of a group differ or contrast in one or more properties. Diversity has been shown to have an impact on team performance (Mathieu et al., Reference Mathieu, Maynard, Rapp and Gilson2008). In their review, Mathieu et al. (Reference Mathieu, Maynard, Rapp and Gilson2008) point out the vastness of the literature featuring team diversity and draw attention to four main diversity dimensions: demographic, personality, functional background, and attitudes and values.
Horwitz and Horwitz (Reference Horwitz and Horwitz2007) conducted a meta-analysis to understand the relationships between team diversity and team performance. For this, they differentiated between two classes of diversity: bio-demographic and task-related. The former refers to diversity in individual properties that are observable and not learned (e.g. personality, gender, age, ethnicity), whereas the latter regards diversity in acquired capabilities, such as education or expertise. Using meta-analytic techniques, they found task-related diversity to be positively correlated to both qualitative and quantitative measures of team performance. However, they did not find a clear relationship between bio-demographic diversity and team performance. Although pointing out the small number of studies supporting these latest findings, their preliminary results seem to give more importance to the diversity of acquired team member properties, such as the type of education or knowledge expertise.
Finally, another factor influencing team performance is team size. The relationship between team size and productivity is a question of broad relevance across economics, psychology, and management science. Hence, the size of a team is one of the most frequently studied parameter when analyzing team performance. There is a disparity in the literature due to the fact that appropriate team size is dependent on the task and the social context in which the team operates. When it comes to athletics, sport teams have a defined number of team players: a football team needs 11, the Standard Platform League in RoboCup five players per team, and baseball teams require nine players. But when it comes to organizations, it is hard to find a golden rule to determine the optimal number of team members. For complex tasks, however, where both the potential profits and risks of teamwork increase with the number of team members, neither theoretical studies nor empirical evaluations consistently favour larger vs. smaller teams (Mao et al., Reference Mao, Mason, Suri and Watts2016). Regarding established theories, psychology (Steiner, Reference Steiner2007), economics (Holmstrom, Reference Holmstrom1982), and management (Malone & Crowston, Reference Malone and Crowston1994) studies suggest that increasing team size can be harmful to team performance. This happens because: individuals find it tempting to free ride on the efforts of teammates (Holmstrom, Reference Holmstrom1982; Steiner, Reference Steiner2007); the overhead associated with communication increases with team size (Steiner, Reference Steiner2007); and communication among team members causes partitioning into sub-teams (Lorenz et al., Reference Lorenz, Rauhut, Schweitzer and Helbing2011) and chitchat (Tetlock et al., Reference Tetlock, Peterson, McGuire, Chang and Feld1992). Therefore, in complex tasks, where all these reasons may exist simultaneously, the relationship between team size and performance is not well described by existing theories. Mao et al. (Reference Mao, Mason, Suri and Watts2016) performed a study of the dynamics of team performance and its relationship with team size in the digital volunteer setting of crisis mapping. Their findings show that although social loafing and co-ordination costs result in reduced contribution from individuals in larger teams, the potential benefits of co-ordination can outweigh this loss in performance.
However, other studies show that there is an inverse relationship between the size of the team and its performance (Bartol, Reference Bartol1977; Oyster, Reference Oyster1999). Oyster (Reference Oyster1999) and Bartol (Reference Bartol1977) show that team size is important when analyzing team performance. Yet, they have offered different recommendations concerning the best size for various types of tasks to achieve acceptable performance. Oyster (Reference Oyster1999) states that the right number of people in a team depends on the kind of tasks team members need to perform. They believe that for teams ranging from four to six, all the team members’ competences can be fully used, but for larger teams some members’ competences are under-used and this provokes that teams split up. According to the studies of Bartol (Reference Bartol1977), the optimal number of members for problem-solving tasks is five. He states that there is a limit to the team size, which, if exceeded, causes a drop in the performance of the team. Bartol (Reference Bartol1977) says that in the case of a team containing more than six people there is a tendency to split the team into two, which brings about negative effects. The cause is twofold: high co-ordination cost and loss of motivation by team members Oyster (Reference Oyster1999).
Finally, some studies have found team size to be unrelated to performance Martz et al. (Reference Martz, Vogel and Nunamaker1992) or that increasing team size actually improves performance without limit Campion et al. (Reference Campion, Medsker and Higgs1993).
4.4.2 Team formation
Once a team has been composed, there are different processes that the team carries out to execute the task and achieve the collective goal. Several classifications of team processes have been proposed in the literature, from which, the most recent and overarching one is the one proposed by Marks et al. (Reference Marks, Mathieu and Zaccaro2001) and Salas et al. (Reference Salas, Sims and Burke2005), Salas et al. (2009). Typically the research investigated the ways of implementing team processes and of measuring how well teams perform. To begin with, Marks et al. (Reference Marks, Mathieu and Zaccaro2001) distinguish between three broad types of processes: action-orientated, transition-orientated, and interpersonal. The first ones refer to actions that team members undertake to accomplish goals, namely team monitoring, systems monitoring, monitoring progress towards goals, and co-ordinating activities. Regarding transition-orientated processes, these are actions related to planning and/or evaluating in order to guide in attaining team goals, that is goal specification, mission analysis, formulation and planning, and strategy formulation. Finally, interpersonal processes are those intended to manage interpersonal relationships. They comprise motivating/confidence building, conflict management and affect management (Marks et al., Reference Marks, Mathieu and Zaccaro2001). On the other hand, Salas et al. (Reference Salas, Sims and Burke2005) built upon previous research and narrowed down the main processes into ‘Big Five’ team processes: team orientation, backup behaviour, team leadership, adaptability, and mutual performance monitoring.
Another important aspect is that team climate influences the effectiveness of processes. A team climate is defined by the degree to which a team of people possesses certain core properties that are needed for the team to work effectively. These properties include the interrelationship among team members, the identification of each person with the team and its social values, the co-ordination of team resources, behaviours and technologies, as well as the desire of each team member to achieve the objectives of the team (Meneses & Navarro, Reference Meneses and Navarro2015). A good climate assures the sharing of resources, mutual rewards, and information exchange. It promotes a high level of openness, safety, and a mix of upward, downward, and horizontal communication processes that help to increase team performance (Kozlowski and Ilgen; Mathieu et al., Reference Mathieu, Maynard, Taylor, Gilson and Rudy2007; Knapp, Reference Knapp2010; Rico et al., Reference Rico, Alcover and Tabernero2010).
A team climate that is conductive to learning requires shared perceptions of work settings (Brodbeck, Reference Brodbeck2003; James et al., Reference James, Choi, Ko, McNeil, Minton and Wright2008; Ramirez-Heller et al., Reference Ramirez-Heller, Berger and Brodbeck2014). According to Brodbeck (Reference Brodbeck2003) and Ramirez-Heller et al. (Reference Ramirez-Heller, Berger and Brodbeck2014), a team climate conductive to learning is characterized as one in which:
1. There is empathy, support, as well as a common understanding among its members, conveying an atmosphere of mutual trust.
2. There is a regular contact as well as informal and formal communication processes among its members.
3. There exists a common agreement with the goals and objectives to be achieved, and these shared goals are clear, realistic, and feasible.
4. There is a prevailing notion of democracy and equality among its members, with no one having particular control over the others.
5. Members perceive a personal development as the team enhances their creativity and provides general support in fulfilling their individual plans.
Finally, there are various studies in OP analyzing the effect of communication and network structure on team performance. Typically, teams in organizations are strategically composed by heterogeneous individuals (Osatuyi, Reference Osatuyi2012). This is based on the assumption that once team members share their information, the team as a whole can access a larger pool of information, knowledge and expertise. However, studies have shown that teams, unlike individuals, sometimes do not effectively share and use the unique information available to them. This leads to poorer decision making. Informational influence theory holds that the subjective importance of information may affect if information is shared or not. Henceforth, an important factor for performance improvement is the proactive communication of information about team members’ goals (Butchibabu et al., Reference Butchibabu, Sparano-Huiban, Sonenberg and Shah2016). It is also found that task complexity is negatively correlated with information exchange. Surprisingly, teams tend to share less information when working on complex tasks, compared to when working on simple tasks (Osatuyi, Reference Osatuyi2012). Also in an online game domain communication plays an important role on the performance of virtual team members (Leavitt et al., Reference Leavitt, Keegan and Clark2016). For instance, League of Legends enables non-verbal communication through ‘pings’, alerts that are easy to activate and provide auditory and visual hints for teammates. Leavitt et al. (Reference Leavitt, Keegan and Clark2016) analyze 10.293 matches in this popular game and test the impact of ping actions on team performance. They show that pings by players have a positive but concave relationship with player performance. That is, teams sending more pings have higher levels of performance up to a point after which sending more pings leads to lower levels of performance.
Another important factor influencing team performance is team shared belief in their collective power to produce desired results (Yildir, Reference Yildir2005). In Yildir (Reference Yildir2005), the team shared beliefs of computer game players were measured as 126 teams competed in a highly interdependent, online role-playing team game. Structural equation modelling results indicated that for all interdependent teams, as team shared belief increased, both team persistence and performance also increased positively and linearly.
4.4.2.1 Analysis
Regarding team composition, there is a strong relationship between task type and team type (structure). The type of the team depends on the features of the task to perform and so very often team types are derived from task types. Besides task type, team diversity plays an important role when composing teams. Regarding the ‘optimal’ team size, it is a complex question and future research is needed to determine the impact of team size on team performance, such as the nature of the task, the internal motivations, and the context. Some preliminary results show that the more complex the task, the larger the size of the team needs to be, but limited to an optimal size of six members. Regarding team formation, several different team processes classifications have been proposed, though no agreement has been reached. Finally, having a good team climate seems key to achieve good performance.
4.5 WHEN do we do it? The dynamics
Humans learn with every interaction. Our memory recollection and capability improvement cannot be removed or stopped. Hence, the OP research usually deals with complex scenarios, those of simultaneous and successive tasks, see Section 4.5. In OP, the dynamic properties of a team are referred to as emergent states. Emergent states develop during teamwork and have an effect on the outcomes. Several examples of emergent states (Mathieu et al., Reference Mathieu, Maynard, Rapp and Gilson2008) are team confidence, team empowerment, cohesion, team climate, collective cognition, or trust between team members.
The development of emergent states is closely connected to the process of team learning behaviours. As members of a team interact with one another and perform tasks, they learn from their experiences. That is, they learn by asking questions, seeking feedback, experimenting, reflecting on results, and discussing errors or unexpected outcomes of previous actions (Edmondson, Reference Edmondson1999). These complex tasks allow team members to acquire, share, combine, and apply knowledge (Olivera and Argote, Reference Olivera and Argote1999; Kozlowski and Ilgen, 2006). They also lead to the development of shared understanding and meaning as well as to the acquisition of mutual knowledge, skills, and performance capabilities (Garavan & McCarthy, Reference Garavan and McCarthy2008). All these developments enhance team performance (Edmondson, Reference Edmondson1999; Zellmer-Bruhn & Gibson Reference Zellmer-Bruhn and Gibson2006).
4.5.1 Analysis
Unlike CS, the reviewed OP literature does not study simple scenarios such as non-successive and non-simultaneous tasks. Typically, OP analyzes complex and realistic scenarios as human learning capabilities need to be considered. Moreover, on top of including the social network and memory about the outcomes of past experiences, the researchers in OP deal with the dynamics of individuals’ capabilities (as humans learn new capabilities and forget not used ones).
4.6 WHERE do we do it? The context
From a systemic perspective teams are part of the structure of an organization and therefore they operate within this organization. In the same way, an organization is part of the environment. The environment creates demands and requirements for an organization and influences the organization’s system. In turn, the organization tries to address these requirements by influencing the operations of its teams and their performance in diverse ways.
Research results suggest that context plays an important role in the performance of teams (Hackman, Reference Hackman1990; Guzzo & Dickson, Reference Guzzo and Dickson1996). Hackman (Reference Hackman1990) between others propose and analyze many contextual factors that have to be considered when composing a team:
∙ The uncertainty on the level of complexity of the tasks and the degree of dynamics of the environment. Both aspects influence the uncertainty within the organization and therefore its teams need to operate with incomplete knowledge. The uncertainty about external factors is determined by the available information about the customers, the suppliers, or other competing organizations. The uncertainty about internal factors is determined by the dynamics of tasks, organizational rules and objectives. In such an uncertain context, teamwork is more challenging and paradoxically teams may perform better than in a stable and predictable context.
∙ The vision and mission of an organization that determine the main rules and norms to be followed and what is to be considered as good performance.
∙ The set of values, policies and strategies of the organization. For instance, organizations supporting individual values will hinder teamwork and team performance will thus be poor. This is because teamwork is based on shared values, mutual support, constructive collaboration, mutual trust, co-ordination mechanisms, and synergies, which are collective values. On top of it, an organization promoting internal competition will lead to individual strategies of withholding information and self-interested behaviours.
∙ The organizational benefits such as the reward or the training systems. Diverse motivational theories are available to explain the relevance of the reward systems for increased performance. For example, teams will perform better with an appropriate reward system.
∙ The resources and assistance made available to the team. It is obviously easier for the team to achieve good performance when operating in a context of resource abundance.
∙ The organizational climate. A context with a perceived climate of control and low level of autonomy for the team will hinder successful teamwork and performance. As teamwork requires an individual engagement with the team, a climate is needed that facilitates information sharing or team skills development.
∙ The cultural context. The definition of a team changes across cultures: in cultures valuing individualism teams are seen more as a set of people each contributing to a different subtask, whereas in cultures valuing collectivism teams are seen as having shared goals, values and responsibility for the whole task. Research results show that teams perform better in a collective cultural context.
4.6.1 Analysis
In contrast with CS approaches, the context where teams solve tasks plays an important role in the OP literature. The context is understood as internal and external factors influencing teamwork. The internal context can be characterized as dimensions of the organization, such as vision and mission, values, policies and strategies, or organizational benefit system. The external context can be characterized as dimensions of the environment in which the organization operates, that is the culture, the available resources, and the uncertainty about other players behaviour.
5 Discussion
CS and OP have followed rather disparate approaches when it comes to team composition and team formation. However, some similarities and differences can be drawn and several new research questions can be formulated from a cross reading of the two literature corpus. In Table 1 a comparison of the main papers in CS can be found.
Comparison of the computer science contributions reviewed in this paper

5.1 Similarities in both approaches
When modelling agents’ properties in CS, there are two main approaches. There is either extensive a priori information about teammates given as input or ad hoc scenarios where agents learn their teammates’ capabilities. In OP a number of tests are proposed to acquire a-priori information about teammates, such as intelligence or cognitive competences tests, assessment centres or social and behavioural competence tests. Also, similar to CS, OP studies allow to learn human capabilities from their repeated interactions.
To maximize team performance, one of the crucial findings in both OP and CS is that team members have to be heterogeneous.
Regarding the tasks that are executed by agent teams, both OP and CS focus rather on team members’ properties required to perform a task than on a detailed planning of the task execution.
5.2 Differences in both approaches
The first difference we find between CS and OP is with respect to the complexity of individual team members. OP focusses on humans with all their intrinsic complexity while CS focusses on a limited set of human-like properties to build software agents. In CS the agent properties have been categorized as personality and capacity. In OP, although human properties can also be categorized as personality and capacity, capacity is a much wider concept. It contains not only skills, but also other properties, such as competences, experience, gender, or age. Moreover, while in OP the human capabilities are assumed to be dynamic (i.e. lifelong learning), software agents capabilities are assumed to be static and only the behaviour model may change with agents’ interactions.
In CS the majority of approaches assume that the joint capabilities of agents in a team are enough to solve a given task. However, the researchers in OP recognize also other factors as important when composing and forming a team, such as the motivation of individuals and the task context. They also show that the motivation characteristics predict more accurately the performance of a team than the other factors. Regarding OP research gaps, it lacks a mapping between cognitive ability of individuals and task types (which is an input in CS models) which complicates team composition.
The CS literature has focussed on team co-operation with various objectives that can be categorized as at least one of the following: minimizing overall cost, maximizing social utility, or maximizing the quality of the outcome (understood as maximizing team performance). In OP, the main objective for team composition and formation is just to maximize team performance. Moreover, from an OP perspective team performance cannot be assessed by the time spent to perform a task, by comparing costs or by counting the number of right answers as it would ignore some important subjective reasons. Instead, OP analyzes possible causes of failure, such as an excessive amount of work needed to execute the task given the size of the team or the lack of motivation of team members. This is why the performance is assessed from two perspectives: objective and subjective, while, CS only considers objective measures. In CS there are only early attempts to include a subjective perspective while analyzing team performance. It is shown that the motivation increases by introducing competition mechanisms (like in crowdsourcing teams) or by giving agents freedom while selecting their collaborators (like in ad hoc teams).
Since in CS agents can be modeled depending on the needs, researchers can study different settings depending on the dynamics of task arrival (one task or many, one time or many). Many MAS models are simplistic since they consider only one task arriving at a time. Unlike CS, the reviewed OP literature does not study simple scenarios, since humans have memory and improve their capabilities with every task. Hence, typically OP analyzes only complex and realistic scenarios. The CS literature uses these complex scenarios to let agents build their beliefs based on past experiences and compose new teams according to these learned beliefs. OP, on top of including the social network and memory about the outcomes of past experiences, deals with the dynamism of individuals’ capabilities (as humans learn new capabilities and forget not used ones).
5.3 Cross-fertilization opportunities
Prior sections explored a range of concepts and issues concerning team composition and formation. In this final subsection, we focus on posing research questions for the field, organized around a set of research opportunities:
∙ Establish a connection with the OP literature: We pose the following research questions:
1. What criteria to use when composing effective teams?: A goal of OP is to improve organizational performance by placing the right people in the right jobs, thus enhancing the fit between the individual and the organization. This includes manual methods for building effective teams. Nevertheless, research on team composition and team formation in CS and OP have evolved separately. The MAS literature has typically disregarded significant OP findings, with the exception of several recent, preliminary attempts (like Andrejczuk et al. (Reference Andrejczuk, Rodriguez-Aguilar and Sierra2016), Farhangian et al. (Reference Farhangian, Purvis, Purvis and Savarimuthu2015b), or Hanna and Richards (Reference Hanna and Richards2015)). This body of research has focussed on algorithms that help automate team formation and composition. Research findings from the OP literature have much potential for MAS heuristics (such as team diversity (Mathieu et al., Reference Mathieu, Maynard, Rapp and Gilson2008), team size (Mao et al., Reference Mao, Mason, Suri and Watts2016), or context (Guzzo and Dickson, Reference Guzzo and Dickson1996)).
2. Are current CS methods enough to measure team performance?: From an OP perspective, team performance cannot be assessed by simply measuring how long it takes for a group to finish a certain task or by counting the number of right answers to predefined and clear questions, which is a common approach in CS. OP rather analyzes joint team objectives and the team composition and formation setting (such as unrealistic deadlines, the number of individuals in a team, the level of stress in a team or the quality of the outcome). Also, OP focusses on the inner development of team members and analyses the quality of human resources in a team, that is, motivation, satisfaction, commitment, illness, or stress rate (Quijano et al., Reference Quijano, Navarro, Yepes, Berger and Romeo2008). When evaluating team performance, CS research should take into account team objectives, task dependencies, the feasibility of the task, etc.
3. How to exploit the factors that influence team performance?: According to OP research, in order to carry out highly interdependent tasks, all team members should possess co-ordination skills (maturity) and some of them the capacity to take decisions (diversity). Also, the greater the uncertainty and interdependence of task types, the more diverse the competences for team members to cope with complexity. However, if the team is overqualified for the task to perform, the motivation of team members decreases and the quality of the outcome is lower or the task is not completed at all. All these dependencies have been studied extensively by OP research, but they are ignored by CS. We should work to understand what is the correlation between task type and team type and what is the exact influence on team performance.
∙ Enhancing agent models: The CS literature is in need of analysing more complex examples where agents are modeled as humans. Based on our findings we form several research questions for MAS research:
1. How to develop richer information (or cognitive) agent models to enhance team composition?: In OP, the most important capacity of team members that is related to team performance is their cognitive ability. It is a much wider concept than the notion of capacity in MAS, since beyond skills, widely used by MAS research, it contains many other properties such as experience, competences, age, or even gender. While some of the human properties may not make sense in an agent context (like age or gender), some do (such as cognitive abilities, lifelong learning, or behavioural model). Also, there is a need to include more sophisticated models for agent capabilities, such as graded capabilities instead of binary ones. Richer agent models would allow the CS field to further benefit from OP findings for team composition and formation.
2. How to model and exploit competence dynamics?: The majority of CS models assume that competences are a fixed property of each agent. OP indicates that human capabilities are necessarily dynamic (evolve along time) so that teams can successfully perform tasks in dynamic real-world scenarios and in a variety of contexts. The dynamics of competences through learning and experience and the cultural values could be used by MAS research to program adaptive agents, specially when interacting in mixed teams involving humans.
3. How can we include agents’ motivation in team composition and formation models?: OP research highlights motivation as an important factor for team performance Hackman (Reference Hackman1990). The majority of the MAS literature on team composition and teamwork assumes that agents always behave according to their capabilities and knowledge. While in MAS research it is shown that motivation increases by introducing competition mechanisms (like in crowdsourcing teams, Rokicki et al. (Reference Rokicki, Zerr and Siersdorfer2015)), or by giving agents freedom when selecting their collaborators (like in ad hoc teams, Agmon et al. (Reference Agmon, Barrett and Stone2014)), these are only early attempts to include agents’ motivation as an important factor for team performance.
∙ Enhancing task execution: We are interested in the following research questions for multiagent research:
1. Are agents’ joint capabilities enough for successful task execution?: Regarding the tasks that are executed by agent teams, CS focusses on those team members’ properties required to perform a task rather than on a detailed planning of task execution. The majority of approaches assume that the joint capabilities of agents in a team are enough to solve a given task. There are some preliminary attempts to include planning, though they are very simplistic. The majority of methods do not consider time constraints, action dependencies, action failure, plan robustness, task dynamic changes and hence, the vast literature on planning has not yet been integrated into team formation methods.
2. How to endow agents with competence learning capabilities?: Since in CS agents can be engineered depending on the needs (i.e. agents can be designed with different properties, such as personality or memory, depending on the whole system design), researchers can study different settings depending on the dynamics of task arrival. The CS literature uses complex scenarios to let agents build their beliefs based on past experiences and compose new teams according to these learned beliefs. However, while executing tasks, there are no contributions that explore successive or simultaneous settings. Agent learning when executing tasks could be used to further improve the task execution.
∙ Enhancing team performance through context inclusion: Particularly, we are interested in the following question:
1. How to computationally exploit the context within team formation and composition?: OP research results suggest that context plays an important role in the performance of teams, Guzzo and Dickson (Reference Guzzo and Dickson1996), Hackman (Reference Hackman1990), Terveen and McDonald (Reference Terveen and McDonald2005). Although, to the best of our knowledge, there are only a few works in CS that would recognize context as an important factor, besides the social and geographical context considered in some papers. There is a need to perform further research on how to computationally model the context within team composition and team formation to build better performing agent teams.
∙ Enhancing team modelling: We form the following research question for multiagent research, that is:
1. Is the sum of the agents’ individual capabilities enough to predict team performance?: Although individuals’ properties have been extensively studied and considered, there is still a need for modelling the global properties of agent teams. Such modelling should go beyond considering simple properties such as the sum of the agents’ individual capabilities or the Boolean representation of whether the team can perform a task or not. One of the findings from OP that could be used is a general collective intelligence factor that explains team performance on a wide variety of tasks, Woolley et al. (Reference Woolley, Chabris, Pentland, Hashmi and Malone2010).
Acknowledgements
This work is supported by projects CollectiveMind (funded by the Spanish Ministry of Economy and Competitiveness, under grant number TEC2013-49430- EXP), Collectiveware (TIN2015-66863-C2-1-R MINECO/ FEDER), 2017 SGR 172, and MILESS (Ministerio de Economía y Competitividad) TIN2013-45039-P Financed by FEDER. The first author is supported by an Industrial PhD scholarship from the Generalitat de Catalunya (DI-060).
