Download STEAM Teamwork Model and Role Allocation in Extreme Teams and more Study Guides, Projects, Research Communication in PDF only on Docsity!
Evolution of a Teamwork Model
Nathan Schurr, Steven Okamoto, Rajiv T. Maheswaran,
Paul Scerri and Milind Tambe
1 Introduction
For heterogeneous agents working together to achieve complex goals, teamwork (Jennings, 1995; Yen, Yin, Ioerger, Miller, Xu, & Volz, 2001; Tambe, 1997a) has emerged as the dominant coordination paradigm. For domains as diverse as rescue response, military, space, sports and collaboration between human workmates, flexible, dynamic coordination between cooperative agents needs to be achieved despite complex, uncertain, and hostile environments. There is now emerging consensus in the multiagent arena that for flexible teamwork among agents, each team member is provided with an explicit model of teamwork, which entails their commitments and responsibilities as a team member. This explicit modelling allows the coordination to be robust, despite individual failures and unpredictably changing environments. Building on the well developed theory of joint intentions (Cohen & Levesque,
- and shared plans (Grosz & Kraus, 1996), the STEAM teamwork model (Tambe, 1997a) was operationalized as a set of domain independent rules that describe how teams should work together. This domain independent teamwork model has been successfully applied to a variety of domains. From combat
air missions (Hill, Chen, Gratch, Rosenbloom, & Tambe, 1997) to robot soc- cer (Kitano, Asada, Kuniyoshi, Noda, Osawa, & Matsubara, 1997) to teams supporting human organizations (Pynadath & Tambe, 2003) to rescue response (Scerri, Pynadath, Johnson, P., Schurr, Si, & Tambe, 2003), applying the same set of STEAM rules has resulted in successful coordination between heteroge- neous agents. The successful use of the same teamwork model in a wide variety of diverse domains provides compelling evidence that it is the principles of team- work, rather than exploitation of specific domain phenomena, that underlies the success of teamwork based approaches. Since the same rules can be successfully used in a range of domains, it is desirable to build a reusable software package that encapsulates those rules in order to provide a lightweight and portable implementation. The emerging standard for deploying such a package is via proxies (Pynadath & Tambe, 2003). Each proxy works closely with a single domain agent, representing that agent in the team. The second generation of teamwork proxies, called Machinetta (Pynadath & Tambe, 2003; Scerri et al., 2003), is currently being developed. The Machinetta proxies use less computing resources and are more flexible than the proxies they have superseded. While approaches to teamwork have been shown to be effective for agent teams, new emerging domains of teamwork require agent-human interactions in teams. These emerging domains and the teams that are being developed for them introduce a new set of issues and obstacles. Two algorithms that need to be revised in particular for these complex domains are the algorithms for adjustable autonomy (for agent-human interaction) and algorithms for role allocation. This chapter focuses in particular on the challenge of role allocation. Upon instantiation of a new plan, the roles needed to perform that plan are created and must be allocated to members of the team. In order to allocate a
to. For Machinetta, these policy algorithms were translated from Soar into Java in the coordination component of the proxy. For an example of some of these Soar rules, see Appendix A. Indeed, the Soar model can be viewed as a BDI architecture, enabling us to borrow from BDI theories. In the rest of this section, a mapping of Soar to BDI is presented, and readers unfamiliar with Soar may wish to proceed forward to Section 3. To see the mapping from Soar to BDI, let us consider a very abstract defi- nition of the Soar model. Soar is based on operators, which are similar to reac- tive plans, and states (which include the agent’s highest-level goals and beliefs about its environment). Operators are qualified by preconditions which help select operators for execution based on an agent’s current state. Selecting high- level operators for execution leads to subgoals and thus a hierarchical expansion of operators ensues. Selected operators are reconsidered if their termination conditions match the state. While this abstract description ignores significant aspects of the Soar architecture, such as (i) its meta-level reasoning layer, and (ii) its highly optimized rule-based implementation layer, it will be sufficient for the sake of defining an abstract mapping between BDI architectures and Soar as follows:
1: Intentions are selected operators in Soar
2: Beliefs are included in the current state in Soar
3: Desires are goals (including those generated from operators which are sub- goals)
4: Commitment strategies are strategies for defining operator termination con- ditions. For instance, operators may be terminated only if they are achieved, unachievable or irrelevant
In Soar, a selected operator (commitment) constrains the new operators (op- tions) that the agent is willing to consider. In particular, the operator constrains the problem space that is selected in its subgoal. This problem space in turn constrains the choice of new operators that are considered in the subgoal (un- less a new situation causes the higher-level operator itself to be reconsidered). Interestingly, such insights from Soar have parallels in BDI architectures. Both Soar and BDI architectures have by now been applied to several large-scale ap- plications. Thus, they share concerns of efficiency, real-time, and scalability to large scale applications. Interestingly, even the application domains have also overlapped. For instance, PRS and dMARS have been applied in air-combat simulation, which is also one of the large-scale applications for Soar. Despite such commonality, there are some key differences between Soar and conventional BDI models. Interestingly, in these differences, the two models appear to complement each other’s strengths. For instance, Soar research has typically appealed to cognitive psychology and practical applications for ratio- nalizing design decisions. In contrast, BDI architectures have appealed to logic and philosophy. Furthermore, Soar has often taken an empirical approach to architecture design, where systems are first constructed and some of the underly- ing principles are understood via such constructed systems. Thus, Soar includes modules such as chunking, a form of explanation-based learning, and a truth maintenance system for maintaining state consistency, which as yet appear to be absent from BDI systems. In contrast, the approach in BDI systems appears is to first clearly understand the logical and philosophical underpinnings and then build systems.
Figure 1: Proxy software architecture.
and need only know the ultimate decision made by the proxy, whether that decision was made autonomously or by the team member. The RAP interface component is the only part of the proxy that needs to be designed for a specific type of team member. For example, the RAP interface for a person playing the role of fire chief in the disaster rescue domain is a large graphical interface, while for agents a simple socket communicating a small, fixed set of messages is sufficient. With some extensions, these techniques were used to allow Machinetta to scale up to run 200 proxies on two desktop computers.
3.2 TOP
A team of proxies implements Team Oriented Plans (TOPs). A TOP is a team- level description of the activities that need to be performed in order to achieve the goals of the team. It consists of reactive team plans, roles, relationships between roles, and conditions for initiating a plan and terminating a plan. The proxies dynamically instantiate plans when, during the course of execution, their current states match a plan’s required trigger conditions. The proxy communi- cation policy determines precisely which messages should be sent among proxies to ensure that cohesion is maintained. In developing Machinetta, much of the focus has been on joint intentions
theory (Cohen & Levesque, 1991) due to its detailed formal specification and prescriptive power. The joint intentions framework provides a modal logic spec- ification of a team’s mental state, called a joint intention. A team has a joint intention to commit a team action if its team members are jointly committed to completing that team action, while mutually believing that they are completing it. A joint commitment in turn is defined as a joint persistent goal (JPG). The team T ’s JPG to achieve p, where p stands for completion of a team action, is denoted (JP G T p q). The variable q is a relevance term and is true if and only if p is still relevant; if the team mutually believes q to be false, then there is no need to achieve p (i.e., no need to perform the team action) and so the JPG can be abandoned. For illustrative purposes, only teams with two members x and y will be considered here, with their JPG to achieve p with respect to q denoted (JP G x y p q). The following definitions can be extended in a straightforward manner to larger teams. The joint intentions framework uses temporal operators such as 3 (even- tually) and 2 (always), individual propositional attitude operators such as (BEL x p) and (GOAL x p) (agent x has p as a belief and as a goal, re- spectively), and joint propositional attitude operators such as (M B x y p) and (M G x y p) (agents x and y have p as a mutual belief and as a mutual goal, respectively) to build more complex modal operators to describe both individ- ual and team mental states. Two other operators, the weak achievement goal (WAG) operator and the weak mutual goal (WMG) operator, are needed to define a JPG.
JointP ersistentGoal
(JP G x y p q) , (M B x y ¬p) ∧ (M G x y p) ∧ (U N T IL [(M B x y p) ∨ (M B x y 2 ¬p) ∨ (M B x y ¬q)] (W M G x y p q))
In order for a team with members x and y to have p as a JPG with respect to q, four conditions must hold:
1: All team members mutually believe that p is currently unachieved;
2: All team members have p as their mutual goal, i.e., they mutually know that they want p to be true eventually; and
3: Until p is mutually known to be achieved, unachievable or irrelevant, the team holds p as a WMG.
To enter into a joint commitment (JPG) in the first place, all team members must establish appropriate mutual beliefs and commitments. The commitment to attain mutual belief in the termination of p is a key aspect of a JPG. This commitment ensures that team members stay updated about the status of team activities, and thus do not unnecessarily face risks or waste their time. These principles are embodied in Machinetta in the following way. When a team plan is instantiated, the proxies may communicate with their respective RAPs about whether to participate in the plan. Upon successfully triggering a new plan, the proxies perform the “establishJointCommitment” procedure specified by their coordination policy to ensure that all proxies agree on the plan. Because each proxy maintains separate beliefs about these joint goals, the team can detect (in a distributed manner) any inconsistencies among team
members’ plan beliefs. The proxies then use termination conditions, specified in the TOP, as the basis for automatically generating the communication necessary to jointly terminate a team plan when those conditions are met.
3.3 Role Allocation
Roles are slots for specialized execution that the team may potentially fill at runtime. Assignment of roles to team members is of critical importance to team success. This is especially true for heterogeneous teams, where some team members have little or no capability to perform certain roles. However, even for homogeneous teams, team members can usually only perform a limited number of roles simultaneously and so distributing roles satisfactorily throughout the team is of great importance. Upon instantiation of a newly triggered plan, Machinetta proxies also instan- tiate any associated roles. The initial plan specification may name particular team members to fill these roles, but often the roles are unfilled and are then subject to role allocation. The proxies themselves have no ability to achieve goals at the domain level; instead, they must ensure that all of the requisite domain-level capabilities are brought to bear by informing team members of their responsibility to perform instantiated roles that are allocated to them. One role allocation algorithm successfully used in Machinetta is described in Section 5.
3.4 Example
To see how joint intentions and role allocation affect team behavior, consider an example of personal assistant proxies in an office environment. A group of three researchers, Scientist1, Scientist2, and Scientist3, need to make a joint presentation of their work at a meeting. Each person has a proxy (Proxy1 for
to another proxy in the hopes of it being allocated to someone more capable. For the purposes of this example, suppose that the roles are successfully allocated, with Scientist1 presenting the introduction, Scientist2 presenting the demonstration, and Scientist3 presenting the conclusion. The researchers begin preparing their respective portions of the presentation. The proxies all have the JPG of making the presentation. Now consider four ways in which this joint commitment can be terminated. In the first case, suppose that the meeting time arrives and the three scientists present their respective portions. As each completes his part of the presentation, his proxy is updated of the status. Once Proxy3 is notified that the conclusion has been presented, it knows that the presentation has been completed and so the JPG has been achieved. It now communicates this fact to the other proxies, so that all members of the team mutually believe that the presentation has been completed. In the second case, suppose that Scientist3 becomes sick on the day of the presentation. He informs his proxy that he will be unable to attend. Proxy realizes that without Scientist3’s participation the JPG is unachievable, and so it drops its goal of making the presentation. Under its joint commitment, it then communicates this information to the other proxies, who can then notify their users. This allows team members to stop preparations for the presentation and attend to other business. Once mutual belief that the goal is unachievable is established, the joint commitment dissolves. Because Scientist3 is the only team member capable of presenting the conclusion, there is no way to salvage the team plan. The third case is similar to the second, but it is Scientist1 who falls ill. Proxy1 then notifies Proxy2 and Proxy3 that the goal is unachievable, and so they drop the JPG. In this case, however, Proxy2 and Proxy3 recognize that it
may be possible to still make the presentation; Proxy2 and Proxy3 then enter into a new joint commitment to repair the team plan. They do so by reallocating the introduction presentation to someone other than Proxy1; for the sake of this example, say that Proxy2 accepts this role. The new, repaired team plan can now be instantiated and Proxy2 and Proxy3 enter into a JPG to perform the presentation. Scientist2 is informed that he must present the introduction as well as the demonstration, and the meeting can go on as scheduled. In the last case, Proxy3 learns that the meeting has been cancelled and so the presentation has become irrelevant. As a result, it drops its goal of presenting, and the JPG of presenting becomes false as well. However, as in the case of the goal being unachievable, the team behavior is not completely dissolved, because only Proxy3 knows that the presentation is irrelevant; a WAG to make the presentation persists. Proxy3 now must take action to achieve mutual belief among all team members that the presentation is irrelevant. To achieve this, it notifies the other two proxies that the meeting has been cancelled. These proxies in turn notify their users of the cancellation. Only when there is mutual belief that the presentation is irrelevant are the proxies fully released from their joint commitment.
4 Domains
The proxy approach has been applied earlier to several domains such as battle- field simulations (Tambe, 1997b) and RoboCup soccer simulations (Pynadath & Tambe, 2003; Kitano et al., 1997). This section will describe three additional domains that have been used to explore proxy-based teamwork. In each of these domains the same teamwork approach has been applied and been shown to be effective without changes to the key ideas. The first domain is that of a team of personal assistant agents. Individual
Figure 3: Disaster Response using Machinetta Proxies.
extension of the RoboCup Rescue simulation environment (Kitano, Tadokoro, Noda, Matsubara, Takahashi, Shinjoh, & Shimada, 1999) that enables human- robot interactions. Fire brigade agents act in a virtual city, while human and robot team members act in the physical world. The fire brigades can search the city after an earthquake has hit and can extinguish any fires that are found. The agents try to allocate themselves to fires in a distributed manner, but can call on the expertise of the human fire chief if required. The fire chief can al- locate trucks to fires easily both because of a more global view of the situation and because the spatial, high-level reasoning required is well suited to human capabilities. Thus, the fire chief’s proxy must carefully adjust its own autonomy when accepting and rejecting roles. The third domain involves a type of Unmanned Aerial Vehicle (UAV) known as Wide Area Search Munitions (WASMs), which are part UAV and part mu-
Figure 4: Unmanned Aerial Vehicle Simulator.
nition (Scerri, Xu, Liao, Lai, & Sycara, 2004). Experiments were performed using a simulation environment. Figure 4 shows a screenshot of the simulation environment in which a large group of WASMS (small spheres) are flying in pro- tection of a single aircraft (large sphere). Various surface to air missle sites are scattered around the environment. Terrain type is indicated by the color of the ground. As many as 200 WASMs were simulated, each with its own Machinetta proxy. In the experiments, a team of WASMs coordinate to find and destroy ground based targets in support of a manned aircraft that they are guarding.
5 Novel Role Allocation Method
To allocate unfilled roles to team members, a novel role allocation algorithm has been developed that draws upon ideas from distributed constraint optimization problems (DCOPs). Based on valued constraints, DCOP is a powerful and natural representation for the role allocation problem. Mapping the problem to
quality of the output, the speed of task performance or other factors affecting output. Each role requires some resources of the team member in order to be performed. Resource requirements of a team member ek for a role rj are written as Resources(ek, rj ) and the available resources of an agent, e, as e.resources. Following convention, we define a matrix A, where ai,j is value of the ith row and jth column and
ai,j = 1 if ei is performing rj otherwise ai,j = 0.
Thus, the matrix A defines the allocation of roles to team members. The goal in GAP is to maximize:
f (A) = ∑ e∈E
r∈R
Cap(e, r) × ae,r
such that
∀i(∀e ∈ E(
r∈R
Resources(e, r) × ae,r ≤ e.resources))
and ∀r ∈ R
e∈E
ae,r ≤ 1.
Intuitively, this says that the goal is to maximize the capabilities of the agents assigned to roles, subject to the resource constraints of team members, ensuring that at most one team member is assigned to each role but potentially more than one role per team member.
5.1.2 Extended GAP
To introduce the dynamics of extreme teams into GAP, make R, E, Cap and Resources functions of time. The most important consequence of this is that a single allocation A is no longer sufficient; rather, a sequence of allocations is needed, A→, one for each discrete time step. A delay cost function, DC(ri, t), captures the cost of not performing ri at time t. Thus, the objective of the E-GAP problem is to maximize:
f (A→) = ∑ t
e∈E
r∈R
(Cap(e, r, t) × ae,r,t) − ∑ t
r∈R
e∈E
ae,r,t) × DC(r, t)
such that ∀i(∀e ∈ E(∑ r∈R
Resources(e, r) × ae,r,t ≤ e.resources))
and ∀r ∈ R ∑ e∈E
ae,r,t ≤ 1
Thus, extreme teams must allocate roles rapidly to accrue rewards, or else incur delay costs at each time step.
5.2 LA-DCOP
Given the response requirements for agents in extreme teams, they must solve E-GAP in an approximate fashion. LA-DCOP is a DCOP algorithm that is being proposed for addressing E-GAP in a distributed fashion. LA-DCOP ex- ploits key properties of extreme teams that arise due to large-scale domains and similarity of agent functionality (e.g., using probability distributions), while simultaneously addressing special role-allocation challenges of extreme teams
