Strategic and Tactical Behaviour in Automated Negotiation Fernando Lopes1 and Helder Coelho2 1

LNEG − National Research Institute Estrada do Pac¸o do Lumiar 22, 1649-038 Lisbon, Portugal [email protected] 2

University of Lisbon Department of Computer Science Bloco C6, Piso 3, Campo Grande, 1749-016 Lisbon, Portugal [email protected]

ABSTRACT Traditional negotiation, conducted face-to-face and via mail or telephone, is often difficult to manage, prone to misunderstanding, and time consuming. Negotiators are typically satisfied with the final outcome and, in many instances, proudly describe it. However, closer inspection usually reveals that money is squandered, resources wasted, and potential joint gain untapped. Automated negotiation promises a higher level of process efficiency, and more importantly, a faster emergence and a higher quality of agreements. The potential monetary impact has led to an increasing demand for systems composed of software agents representing individuals or organizations and capable of reaching efficient agreements (e.g., the industrial trend toward agent-based supply chain management). Work to date on automated negotiation has generated many useful ideas and concepts leading to important theories and systems. Yet, the field is still immature. The design of software agents with negotiation competence largely lacks systematic, traceable, and reproducible approaches, and thus remains more an art than a science. Against this background, this paper presents a model for software agents that handles two-party and multi-issue negotiation, manages important activities that negotiators often perform before starting to negotiate, and formalizes relevant procedures studied in the social sciences and frequently used in real-life negotiation. The model incorporates various strategies and tactics. Strategies are computationally tractable functions that define the tactics to be used both at the beginning and during the course of negotiation. They are based on rules-of-thumb distilled from behavioral practice in human negotiation. Tactics, in turn, are functions that specify the short-term moves to be made at each point of negotiation. They are structured, directed, and driven by strategic considerations. Keywords: Automated negotiation, Bargaining, Negotiation strategies, Negotiation tactics, Autonomous agents, Multi-agent systems, Negotiation systems. 2010 Mathematics Subject Classification: 68T42, 68T35, 68T20, 91A, 91E

1

Introduction

Negotiation is a discussion among conflicting parties with the aim of reaching agreement about a divergence of interests (Pruitt, 1998). The list of situations that can be handled by negotiation is endless. Some situations are purely competitive, as when the parties have completely opposed interests. Other situations are purely cooperative, as when the parties have perfectly compatible interests. Most situations are mixed-motive, containing elements of both competitive and cooperative situations − the parties’ interests are imperfectly correlated (Pruitt and Carnevale, 1993). There are, however, several characteristics common to most negotiation situations, including (Lewicki et al., 2003): (i) two or more parties, (ii) a conflict among the parties − that is, what one party wants is not necessarily what the other parties want, and (iii) an individual preference to search for agreement rather than to appeal to a higher authority, to permanently break off contact, or to fight openly. Negotiation may involve two parties (bilateral negotiation) or more than two parties (multilateral negotiation) and one issue (single-issue negotiation) or many issues (multi-issue negotiation). Also, negotiation may proceed through several distinct phases or stages, notably a beginning or initiation phase, a middle or problem-solving phase, and an ending or resolution phase (Lewicki et al., 2003). The initiation phase focuses on preparation and planning for negotiation − it is marked by each party’s efforts to emphasize points of difference and to posture for positions. The problem-solving phase seeks a solution for a dispute − it is characterized by extensive interpersonal interaction, strategic maneuvers, and movement toward a mutually acceptable agreement. The resolution phase focuses on details and implementation of a final agreement. The heart of negotiation is reciprocal offer and counter-offer in an attempt to agree upon outcomes mutually perceived as beneficial (Tutzauer, 1992). In “good faith” negotiation, offers are made, and are either accepted or returned with counter-offers. There is an unstated assumption that the parties will show their commitment to the process of finding a solution by submitting offers and receiving counter-offers, and not simply by rejecting the offers of the others out of hand. To do so is often seen as “bad faith” bargaining. Hopefully, through the exchange of offers and counter-offers a point is reached on which the parties will agree. The literature on negotiation commonly identifies three different agreement types (Thompson, 2005): 1. a compromise agreement − a settlement on some middle ground on an obvious dimension or dimensions that link the parties’ initial offers; 2. an integrative agreement − a settlement that reconciles (i.e., integrates) the parties’ interests and thus provides higher joint benefit (collective utility) than a simple compromise; 3. a Pareto optimal agreement − a settlement that lies along the Pareto optimal or efficient frontier, i.e., the locus of achievable joint evaluations from which no joint gains are possible (Raiffa, 1982).

Integrative agreements are generally believed to have a number of advantages over compromises (Pruitt and Kim, 2004): they are mutually rewarding, they are more likely to be complied with, and they can foster harmonious relations between the parties. Furthermore, Pareto-optimal agreements produce the highest joint benefit of the three types of agreement. Also, Pareto optimality ensures that resources are not wasted and money is not squandered. Real-life negotiators are typically satisfied with the final agreement and, in many instances, proudly describe it. However, they frequently view conflict-laden situations with a fundamentally more distrustful, win-lose attitude than is necessary, and settle for outcomes that are worse for them than other available solutions (Thompson, 2005). They overlook opportunities for mutually beneficial agreements and instead, settle for suboptimal outcomes. Simply put, they often fail to achieve agreements on the Pareto optimal frontier or even close to this frontier. The primary cause of this failure is the desire to satisfy one’s own goals without regard to the other’s goals. Three additional factors contribute to this failure (Lewicki et al., 2003): (i) the history of the relationship between the parties, (ii) a belief that the issues can only be resolved distributively, and (iii) the mixed-motive nature of most negotiating situations. Traditional negotiation, conducted face-to-face and via mail or telephone, is often difficult to manage, prone to misunderstanding, and time consuming (Bichler et al., 2003). Automated negotiation promises a higher level of process efficiency, and most importantly, a faster emergence and a higher quality of agreements. The potential monetary impact has led to an increasing demand for systems composed of software agents representing individuals or organizations and capable of reaching mutually beneficial agreements. Examples, to mention a few, include (Zhou et al., 2007; Pechoucek and Marik, 2008): • supply chain management − agents representing business units or facilities negotiate the conditions for purchasing raw materials, decide and execute the scheduling, and negotiate the terms under which the products are delivered; • deregulated energy markets − competing power stations offer their electricity output to retailers, and end-user customers choose their supplier from competing electricity retailers. Yet, the field of automated negotiation is still immature − there is much further work to be done, and some current ideas and concepts are likely to be substantially altered as researchers move ahead (but see Lopes et al., 2008). Against this background, the purpose of this paper is to present a negotiation model for software agents. The model handles two-party and multi-issue negotiation, manages important activities that negotiators often perform before starting to negotiate, and formalizes relevant procedures studied in the social sciences and frequently used in real-life negotiation. Conceptually, the model incorporates a bilateral negotiation protocol, a set of negotiation strategies, and a set of negotiation tactics. The protocol formalizes the set of possible tasks that the agents can perform during the course of negotiation. The strategies and tactics formalize the tasks that each agent should perform to negotiate effectively. In particular, the strategies define the tactics to be used both at the beginning and during the course of negotiation. The tactics formalize the individual moves to be made at each point of the negotiation process.

This paper builds on our previous work in the area of automated negotiation. In particular, Lopes et al. (2002) address the problem of developing agents with competence for detecting and validating conflicts and, mainly, for resolving conflicts through negotiation. They introduce a model that handles bilateral and multi-issue negotiation. They also describe an experiment to assess the feasibility of building agents equipped with a version of the model that handles two-party and single-issue negotiation. Lopes et al. (2004) extend the model by introducing important negotiation strategies and tactics motivated by bargaining procedures typical of integrative or win-win negotiation (i.e., the parties are not strict competitors and attempt to explore options to enlarge the “pie” of available resources). Lopes et al. (2005) present an experiment to investigate the performance of agents equipped with a version of the model that handles two-party and multi-issue negotiation. This paper extends our previous work by introducing precise definitions for the key components of the model, notably by formalizing strategies as functions that specify the tactics to be used both at the outset and throughout negotiation. The functions are computationally tractable, which means that agents are able to compute them in a reasonable amount of time. Also, at every period of negotiation, strategies state whether bargaining should continue or terminate. In particular, bargainers have the ability to unilaterally opt out of negotiation when responding to a proposal. Finally, this paper fixes a few technical problems associated with some components of the model presented previously. The remainder of the paper is structured as follows. For illustrative purposes, section 2 introduces a multi-agent supply chain system and describes a mixed-motive negotiation situation. Section 3 is devoted to negotiation between autonomous agents − it presents a negotiation model and gives a summary of the notation used throughout the paper. Section 4 illustrates how software agents equipped with the model operate in a negotiation setting. Section 5 discusses related work and compares the negotiation model with other existing models.

Finally, section 6 presents concluding remarks and indicates future avenues of

research.

2

Agents for Supply Chain Management

Multi-agent systems (MAS) are ideally suited to represent problems that have multiple problem solving entities and multiple problem solving methods (Jennings et al., 1998). The major motivations for the increasing interest in MAS research include the ability to solve problems in which data, expertise, or control is distributed, the ability to allow inter-operation of existing legacy systems, and the ability to enhance performance along the dimensions of computational efficiency, reliability, and robustness. A supply chain is a network of facilities that performs the functions of procurement of raw materials from suppliers, transformation of these materials into intermediate goods and final products, and the delivery of these products to customers. Supply chain functions range from the ordering and receipt of raw materials to the distribution and delivery of final products, via the scheduling, production, and warehousing of intermediate goods and final products (Fox et al., 2000). A multi-agent supply chain system is a collection of software agents, each

responsible for one or more supply chain functions, and each interacting with other agents in the execution of their responsibilities. Central to the design and effective operation of a multi-agent supply chain system system are a core set of problems and research questions, notably: 1. the design problem − how to describe, decompose, and allocate different supply chain functions among a group of software agents? 2. the coordination problem − how to ensure that agents act coherently in making decisions or taking action, accommodating the local decisions or non-local effects and avoiding harmful interactions? The design problem consists mainly in distributing different supply chain functions across a number of autonomous agents. A typical distribution involves at least the following agents: 1. sales agent − responsible for acquiring orders from customers, negotiating with customers, and handling customer requests for modifying or canceling orders; 2. logistics agent − responsible for coordinating the plants and distribution centers of a manufacturing enterprise: it manages the movement of materials and products across the supply chain, from the suppliers of raw materials to the customers of finished goods; 3. scheduling agent − responsible for scheduling and rescheduling the activities of a manufacturing enterprise; 4. resource management agent − responsible for dynamically managing the availability of resources in order to execute the scheduled activities; 5. supplier agents and customer agents − the suppliers sell raw materials and the customers buy finished goods. The coordination problem is focussed on ensuring that autonomous agents act in a tightly coordinated manner in order to effectively achieve their goals. This problem is addressed, at least in part, by designing agents that are able to coordinate their activities through negotiation. Let us introduce a specific situation involving negotiation between the sales agent and the logistics agent. David, the director of Sales, has lined up two new orders for a total of 15000 men’s suits: one for 10000 and the other for 5000 men’s suits. Martin, the director of Logistics, has already stated that it will take four months to make the suits. Together, they will gross over a million Euros, with a fine profit for the company. The problem is that Martin insists that the job will take four months and David’s customer wants a two-month turnaround. Also, David claims that he can’t afford to lose the customer. David and Martin are discussing and, so far, have accomplished little more than making each other angry. The agents can resolve their differences by negotiating a mutually beneficial agreement. To this end, they should be equipped with a negotiation model enabling them to come to efficient agreements.

3

A Negotiation Model for Autonomous Agents

Negotiation, like other forms of social interaction, often proceeds through distinct phases or stages. A phase is a coherent period of interaction characterized by a dominant group of communicative acts that serves a set of related functions in the movement from initiation to a resolution of a dispute (Holmes, 1992). Phase models provide a narrative explanation of the negotiation process, i.e., they identify sequences of events that constitute the story of negotiation. Most models fit into a general structure of three phases (Lewicki et al., 2003): 1. a beginning or initiation phase − focuses on preparation and planning for negotiation and is marked by each party’s efforts to acknowledge a dispute, formulate an agenda, emphasize points of difference, and posture for positions; 2. a middle or problem-solving phase − seeks a solution for a dispute and is characterized by strategic maneuvers, extensive interpersonal interaction, and movement toward a mutually acceptable agreement; 3. an ending or resolution phase − centers on details and implementation of a final agreement; the parties often demand a gesture of good faith commitment to the agreement (close the deal) and determine who needs to do what once the documents are signed (implement the agreement). This section presents a negotiation model for autonomous agents. The model slightly manages important activities that negotiators often perform before starting to negotiate, and primarily focuses on the problem-solving phase of negotiation.

3.1

Negotiation Issues, Negotiating Agenda and Targets

Effective negotiators often make efforts to perform a number of activities prior to actual negotiation, including (Bazerman and Neale, 1992; Lewicki et al., 2003; Thompson, 2005): 1. defining the issues; 2. establishing the negotiating agenda; 3. defining realistic, pessimistic and optimistic targets. First, every negotiator should define the issues to be discussed. In any negotiation, a detailed list of issues can be derived from a number of sources (e.g., an analysis of the conflict that has created the need to negotiate, or past experience in similar situations). Second, every negotiator should prioritize the issues. Prioritization usually involves two steps: (i) determining which issues are most important and which are least important, and (ii) determining whether the issues are connected or separate. Priorities can be set in a number of ways (e.g., to rankorder the issues, or to use standard techniques, such as the nominal group technique). For the sake of simplicity, we consider that negotiators set priorities by ranking-order the issues. Priorities are often concealed, though in some situations negotiators can disclose information about their priorities (e.g., to make effective trade-offs across issues).

Third, every negotiator should assemble all the issues that have been defined into a comprehensive list. The combination of lists from each side in the negotiation determines the negotiating agenda. This task often involves interaction with the opponent − every negotiator discloses his list of issues in order to reach agreement about what will be discussed during actual negotiation. The next step is to define two key points for each issue at stake in negotiation: • the resistance point or limit − the point where every negotiator decides to stop the negotiation rather than to continue, because any settlement beyond this point is not minimally acceptable; • the target point or level of aspiration − the point where every negotiator realistically expects to achieve a settlement. Negotiators willingly disclose their target point, but work hard to conceal their limit. After setting the limits and target points, skilled negotiators frequently define a third key point for each issue at stake: the optimistic point or asking price, i.e., the best deal they could possibly hope to assume. The resistance point will be the most pessimistic (but acceptable settlement) that is achievable, the target point will be synonymous with what a negotiator realistically expects to achieve, and the optimistic point will be the best possible outcome, an ideal solution. Now, the question arises, where should negotiators start: at the most optimistic point, the likely target point, or the most pessimistic resistance point? From a technical perspective, however, it is not very important where one starts. The important point to remember is that by defining one target and then determining the other two, bargainers will be better prepared to frame offers and evaluate counter-offers. At this stage, we hasten to add two explanatory and cautionary notes. First, we assume that these activities can proceed linearly, in the order in which they were presented. Information often cannot be obtained and accumulated quite this simply and straightforwardly, however, and information discovered in performing later activities may force negotiators to reconsider earlier activities. Second, the literature on negotiation discusses a number of different templates, which tend to emphasize different activities in slightly different sequences (see, e.g., Lewicki et al., 2003). Nevertheless, we try in this section to discuss the most important activities in the preparation process. To this end, we present below precise definitions for the aforementioned intuitions. We consider a set A = {a1 , a2 } of autonomous agents (negotiating parties). Both the number of agents and their identity are fixed and known to all the participants. The negotiation issues {x1 , . . . , xn } are quantitative in nature and defined over continuous domains {D1 , . . . , Dn }, respectively. For each issue xk , the range of acceptable values is represented by the interval Dk = [mink , maxk ]. The issues are also known to all the participants. Definition 3.1 (Issue, Agenda). A negotiation issue is a resource to be allocated or a consideration to be resolved in negotiation. The negotiating agenda is the set I = {x1 , . . . , xn } of issues to be deliberated during negotiation.

”

Table 1: Summary of notation A Ci

set of agents (usually ai and aj ) concession strategy of agent ai

D

set of issue domains (the domain of issue xk is Dk = [mink , maxk ])

fki

concession factor of agent ai for issue xk

I Ibi

negotiating agenda (the set of issues {x1 , . . . , xn })

Ii+ Ii− Ii±

set of issues of higher priority to ai (and believed to be of lower priority to aj )

Li

logrolling strategy of agent ai

limki

limit of agent ai for issue xk

Oi (xk )

opening negotiation tactic of agent ai for issue xk

optki prtki pti→j

optimistic point of agent ai for issue xk negotiation proposal (sent by ai to aj in period t of negotiation)

S

set of possible agreements {s1 , s2 , . . . }

Si

set of acceptable agreements for agent ai

sˆi

least-acceptable agreement for agent ai

T

set of time periods {1, 2, . . . , t, . . . }

Ti

set of negotiation tactics of agent ai (the set {Oi , Yi , Yi+ , Yi− , Yi± , . . . })

trgki

target point of agent ai for issue xk

Ui (pti→j ) Vki (xk )

utility of negotiation proposal pti→j

vk

value of issue xk

wki

weight of agent ai for issue xk

Yi (xk , fki )

concession (or yielding) tactic of agent ai to apply to issues in the agenda

set of issues that are of lower priority to agent ai set of issues of lower priority to ai (and believed to be of higher priority to aj ) set of distributive and compatible issues of ai (neither in Ii+ nor Ii− )

priority of agent ai for issue xk

scoring or (marginal) utility function of ai for issue xk

Definition 3.2 (Priority, Weight). The priority prtki of an agent ai ∈ A for an issue xk ∈ I is a number that represents the importance of xk . The weight wki is a number that represents the preference for xk .

”

Definition 3.3 (Limit, Target Point, Optimistic Point). The limit limki of an agent ai ∈ A for an issue xk ∈ I is the ultimate fallback position for xk , the point beyond which ai is unwilling to concede on xk . The target point trgki is the point at which ai is satisfied with the value of xk . The optimistic point optki is the most preferred or ideal value for xk .

3.2

”

The Negotiation Protocol and Negotiators’ Preferences

The protocol specifies the rules that govern the interaction between the agents (Jennings et al., 2001). Specifically, the protocol defines the negotiation states (e.g., accepting proposals), the valid actions of the agents in particular states (e.g., which messages can be sent by whom, to whom, at what stage), and the events that cause states to change (e.g., proposal accepted).

The negotiation literature describes several protocols that vary significantly depending on the type and amount of information exchanged between agents (see, e.g., Rahwan et al., 2004; Lopes et al., 2008). Simple protocols allow agents to exchange only proposals, i.e., solutions to the problem they face. Richer protocols allow agents to provide feedback on the proposals they receive. This feedback often takes the form of critiques, i.e., comments on which parts of proposals are acceptable or unacceptable. Sophisticated protocols allow agents to provide arguments to support their negotiation stance. Thus, in addition to generating proposals and critiques, agents seek to make proposals more attractive (acceptable) by providing additional information in the form of arguments for their positions. Most complex protocols make, however, considerable demands on any implementation, mainly because they appeal to very rich representations of the agents and their environments. Therefore, we consider a simple alternating offers protocol (Osborne and Rubinstein, 1990). Two agents or players bargain over the division of the surplus of n ≥ 2 distinct issues. The players determine an allocation of the issues by alternately submitting proposals at times in T = {1, 2, . . . }. This means that one proposal is made per time period t ∈ T , with an agent, say ai ∈ A, offering in odd periods {1, 3, . . . }, and the other agent aj ∈ A offering in even periods {2, 4, . . . }. The agents have the ability to unilaterally opt out of the negotiation when responding to a proposal. The negotiation process starts with ai submitting a proposal p1i→j to aj in period t = 1. The agent aj receives p1i→j and can either accept the offer (Yes), reject it and opt out of the negotiation (Opt), or reject it and continue bargaining (No). In the first two cases the negotiation ends. Specifically, if p1i→j is accepted, negotiation ends successfully and the agreement is implemented. Conversely, if p1i→j is rejected and aj decides to opt out, negotiation terminates with no agreement. In the last case, negotiation proceeds to the next time period t = 2, in which aj makes a counter-proposal p2j→i . The tasks just described are then repeated. Once an agreement is reached, the agreed-upon allocations of the issues are implemented. Now, there are several procedures for multi-issue bargaining that the players could adopt, notably (Pruitt, 1981; Pruitt and Kim, 2004): 1. joint-offer procedure − involves bargaining over the allocation of the entire endowment stream at once; the issues are fully bundled and discussed jointly; 2. sequential procedure − involves a sequential determination of the allocations; the issues are discussed sequentially, one at at time. The main argument for bundling issues is efficiency, since there are typically mutual gains available when coordination and externalities across issues are taken into account. Specifically, the joint-offer procedure permits agents to exploit the benefits from trading-off concessions on their less preferred issues for concessions by their opponent on the more preferred issues. The sequential procedure, by contrast, allows no trading of concessions nor any exploitation of the benefits from so doing. Since this work seeks to develop autonomous negotiating agents able to make effective trade-offs across issues, the joint-offer procedure is adopted. A proposal (or offer) is a vector specifying a division of the surplus of all the issues.

Definition 3.4 (Proposal). Let A be the set of negotiating agents and I the set of issues at stake in negotiation. Let T be the set of time periods. A proposal pti→j submitted by an agent ai ∈ A to an agent aj ∈ A in period t ∈ T is a vector of issue values: pti→j = (v1 , . . . , vn ) where vk , k = 1, . . . , n, is a value of an issue xk ∈ I.

”

Negotiators should express their own preferences to rate and compare incoming offers and counter-offers.

The most common way to model the preferences of the negotiating

agents is probably to define a utility function over all possible outcomes (Keeney and Raiffa, 1976; Raiffa, 1982). Let I = {x1 , . . . , xn } be the agenda and D = {D1 , . . . , Dn } the set of issue domains. We consider that each agent ai ∈ A has a continuous utility function: Ui : {D1×. . .×Dn } ∪ {Opt, Disagreement} →

R.

Accordingly, when the utility for ai from one

outcome is greater than from another outcome, we assume that ai prefers the first outcome over the second. The outcome Opt is interpreted as one of the agents opting out of the negotiation in a given period of time. Perpetual disagreement is denoted by Disagreement. As noted earlier, negotiation proceeds by an iterative exchange of proposals and countert−1 proposals. Let pj→i be the offer that aj has proposed to ai in period t−1. Likewise, let pti→j

be the offer that ai is ready to propose in the next time period t. The decision to accept t−1 t−1 t t or reject pj→i depends on both the utility Ui (pt−1 j→i ) of pj→i and the utility Ui (pi→j ) of pi→j . t−1 t Thus, ai receives pt−1 j→i and rates it using his own utility function. If Ui (pj→i ) ≥ Ui (pi→j ) then ai

accepts pt−1 j→i at period t−1 and negotiation ends successfully in an agreement. Otherwise, if ai decides to continue bargaining, negotiation passes to period t and the offer pti→j is submitted. These concepts will be stated formally in the definition of the negotiation strategies (see next subsection). A definition of an agreement is now presented. Definition 3.5 (Agreement, Possible Agreements). An agreement is a proposal accepted by all the agents in A. The set of possible agreements is: S = {(v1 , . . . , vn ) ∈ where vk is a value of an issue xk ∈ I.

Rn : vk ∈ Dk , for k = 1, . . . , n} ”

Negotiation may end with either agreement or no agreement. Failure to agree can occur in two ways: (i) either party decides to opt out unilaterally, or (ii) the two do not agree to any proposal. The resistance points or limits play a key role in reaching agreement when the parties have the ability to unilaterally opt out of the negotiation − they define the worst agreement for a given party which is still better than opting out. For each agent ai ∈ A, we will denote this agreement by sˆi ∈ S. Hence, sˆi will be the least-acceptable agreement for ai , i.e., the worst (but still acceptable) agreement for ai . The set of all agreements that are preferred by ai to opting out will be denoted by Si .

Definition 3.6 (Least-acceptable Agreement, Acceptable Agreements). The leastacceptable agreement for an agent ai ∈ A is defined as: sˆi = (limi1 , . . . , limin ), where limik , k = 1, . . . , n, is the limit of ai for an issue xk ∈ I. The set of acceptable agreements for ai is: Si = {s : s ∈ S, Ui (s) ≥ Ui (ˆ si )} where Ui (ˆ si ) is the utility of sˆi for ai .

”

Perpetual disagreement is the least-preferred or worst outcome, i.e., disagreement is even worse than opting out. Thus, the agents prefer any agreement in any given time period over the continuation of the negotiation process indefinitely. Formally, and more precisely, we state the following: (1) (Acceptable agreements versus opting out). For every agent ai ∈ A and acceptable agreement s ∈ Si , Ui (s) ≥ Ui (Opt). (2) (Opting out versus Disagreement). For every agent ai ∈ A, Ui (Opt) > Ui (Disagreement). Now, the additive model is probably the most widely used in multi-issue negotiation − the parties assign numerical values to the different levels on each issue and add them to get an entire offer evaluation (Raiffa, 1982; Raiffa et al., 2002). This model is simple and intuitive, and therefore well suited to the purposes of this work. We consider that each agent ai has a scoring or single-issue (marginal) utility function for each issue at stake in negotiation, i.e., a function that gives the score ai assigns to a value of an issue xk . For convenience, scores are kept in the interval [0,1]. Additionally, as mentioned above, we consider that ai has a multi-issue utility function to rate offers. A formal description of this function follows (a particular single-issue utility function will be presented in section 4). Definition 3.7 (Multi-Issue Utility Function). Let A = {a1 , a2 } be the set of negotiating agents and I = {x1 , . . . , xn } the negotiating agenda. The utility function Ui of an agent ai ∈ A to rate offers and counter-offers takes the form: Ui (x1 , . . . , xn ) =

n X

wki ×Vki (xk )

k=1

where: (i) wki is the weight of ai for an issue xk ∈ I; (ii) Vki (xk ) is the (marginal) utility function of ai for xk .

”

Three explanatory and cautionary notes are in order here. First, the literature commonly makes a distinction between situations involving independent and interdependent issues. In the former situations, preference ranking of levels within one issue does not depend on the levels of the remaining issues. In the latter situations, by contrast, preference ranking of levels within one issue depends on the levels of other issues. The additive model is only appropriate when mutual preference independence exists between issues (Raiffa, 1982). Accordingly, this work

considers situations involving independent issues. Second, representing and reasoning with non-linear preferences handling dependencies between issues is an active area of research in artificial intelligence in general and multi-agent systems in particular. Clearly, multi-issue negotiation with non-linear utility functions raises complex problems, even for the case of binary issue dependencies. The literature proposes various models that exhibit fairly different features and make use of a diverse range of techniques, including simulated annealing (Klein ´ 2008). However, despite the et al., 2003) and utility graph formalisms (Robu and La Poutre, power and elegance of these and other relevant pieces of work, most of them are either computationally expensive or do not scale well for settings with many issues and complex issue dependencies. Finally, the literature discusses a number of different ways to model preferences over time (see, e.g., Keeney and Raiffa, 1976; Fishburn and Rubinstein, 1982). Two specific ways have attracted much attention in strategic negotiation (Osborne and Rubinstein, 1990; Kraus, 2001): constant discount rate and constant cost of delay. Time preferences with a constant discount rate assume that each player ai discounts future payoffs at some given rate δit (referred to as the discount factor). Time preferences with a constant cost of delay consider that each offer costs to each player ai some given amount ci > 0 (referred to as the cost of delay or bargaining cost). Both forms of preferences are simple and powerful − they represent the observation that money today can be used to make money tomorrow, commonly supplemented with the argument that consumption today is better than consumption tomorrow. Despite this, preferences expressing the satisfaction of agents when faced with a choice between different alternatives whose consequences may be considered time-independent can offer much potential value in enhancing our understanding of negotiation.

3.3

Negotiation Strategies

Negotiation strategies account for the individual decisions of the negotiating agents. While the negotiation protocol helps to restrict the possible actions to perform, it often does not specify any particular action. Rather, it frequently marks branching points at which negotiators have to make decisions according to their strategies. Thus, at each step of negotiation, agents often need to follow their strategies to choose among different possible actions to execute. The following two fundamental groups of strategies have attracted much attention in negotiation research (Lax and Sebenius, 1986; Bazerman and Neale, 1992; Pruitt and Kim, 2004): 1. concession making or yielding − negotiators who employ strategies in this group reduce their demands or aspirations to (partially) accommodate the opponent; 2. problem solving or integrating − negotiators maintain their aspirations and try to find ways of reconciling them with the aspirations of their opponent; they work toward solutions that appeal to both sides. Concession strategies are essentially unilateral strategies − the decision to concede is fundamentally a unilateral one. By contrast, problem solving strategies are essentially social strategies − they work as intended when they are accompanied by effective social influence.

Most negotiation strategies are implemented through a wide variety of tactics. Generally speaking, the line between strategies and tactics often seems indistinct, but one major difference is that of scope (Pruitt and Carnevale, 1993; Pruitt and Kim, 2004). Tactics are short-term moves designed to enact broad (or high-level) strategies − they are structured, directed, and driven by strategic considerations (Lewicki et al., 2003). Accordingly, in this work strategies are computationally tractable functions that define the tactics to be used both at the beginning and during the course of negotiation. The words “computationally tractable functions” presume that agents are able to compute the strategies in a reasonable amount of time. Also, at every period of negotiation, strategies state whether bargaining should continue or terminate. Specifically, strategies state whether agents should opt out of negotiation or continue bargaining. Tactics, in turn, are functions that specify the short-term moves to be made at each point of negotiation.

3.3.1

Concession Strategies

The opening offer and the initial concessions are two central elements of negotiation (Raiffa et al., 2002; Thompson, 2005).

When negotiation begins, the parties are faced with a

fundamental question − should the opening offer be exaggerated, more toward the optimistic point, or modest, somewhat closer to the limit? The main advantages of an exaggerated initial offer are (Pruitt, 1981; Lewicki et al., 2003): (i) negotiators can concede further and hence elicit more counterconcessions from their opponent (by the principle of reciprocity), and (ii) negotiators’ later demands are likely to look generous. However, an exaggerated opening offer frequently communicates an attitude of toughness that may be harmful to long-term relationships. Also, it may be seen as too high by the other party and therefore summarily rejected. By contrast, an opening offer seen as reasonable or modest by the other party could perhaps have been higher, either to leave more room for movement or to achieve a higher settlement. After the first round of offers, other fundamental question is, what concessions are to be made? Negotiators can choose to make none, holding firm and insisting on their original positions. By taking a firm position, negotiators attempt to capture most of the initial bargaining or settlement range (defined by the opening offers of both parties). However, there is the very real possibility that firmness will be reciprocated − one or both parties may become intransigent and withdraw completely. Negotiators can also choose to make some concessions, being flexible and changing their original positions. Flexibility often keeps negotiation going − the more flexible one party seems to be, the more the other party will believe that a settlement is possible. Hence, if concessions are to be made, another fundamental question is, how large should they be? Concession strategies are functions that model significant opening positions and typical patterns of concessions. They specify the tactics to be used at the outset of negotiation (to prepare the initial offers). Also, at each step of negotiation, they specify the concession tactics to be used in preparing counter-offers. A formal definition of a generic concession strategy follows.

Definition 3.8 (Concession Strategy). Let A be the set of negotiating agents, I the negotiating agenda, T the set of time periods, and S the set of possible agreements. Let ai ∈ A be the first agent to submit a proposal and Ti his set of tactics. A concession strategy Ci : Ti ×Ti ×T → S ∪ {Yes, No, Opt} for ai is a function with the following general form:  1  if ai ’s turn and t = 1  apply Oi (xk ) and offer pi→j ,     t−1  reject pj→i and quit , if aj ’s turn and Ui (pt−1 si )  j→i ) < Ui (ˆ   Ci (Oi , Yi , t) = apply Yi (xk , fki ) and prepare pti→j if aj ’s turn and Ui (pt−1 si ) j→i ) ≥ Ui (ˆ      if Ui∗ ≥ 0 accept pt−1  j→i else reject,     offer compromise pti→j , if ai ’s turn and t > 1 where: (i)

p1i→j is the opening offer of ai , pt−1 j→i is the offer of aj for time period t−1 of negotiation, and pti→j is the offer of ai for the next time period t of negotiation;

(ii)

for each issue xk ∈ I, Oi (xk ) is an opening negotiation tactic, Yi (xk , fki ) is a concession tactic, and fki ∈ [0, 1] is a real number that defines the magnitude of a concession on xk , referred to as the concession factor of ai for xk (see subsection 3.4, below);

(iii) (iv)

Ui (ˆ si ) is the utility of the least-acceptable agreement for ai ; t−1 Ui∗ = Ui (pj→i ) − Ui (pti→j )

”

Two explanatory and cautionary notes are in order here. First, notation is being abused somewhat, by using Ci (Oi , Yi , t) rather than Ci (Oi (xk ), Yi (xk , fki ), t). The abuse helps improve readability, however, and meaning will always be clear from context. Also, all specifications written in this way can be expanded out into the strictly correct form if necessary. Second, tactics are functions of a single issue rather than a vector of issues. This permits great flexibility, since it allows agents to model a wide range of concession behaviors. Typical behaviors include large initial demands and slow concession making, overt concessions that seek unilateral concessions, and fractionated concessions (Pruitt, 1981; Thompson, 2005). The remainder of this subsection slightly discusses these behaviors and introduces specific concession strategies. Bargainers frequently start with ambitious demands, well in excess of limits and aspirations, and make small concessions during negotiation. Large initial demands and slow concession making are often motivated by concern about position loss and image loss (or face-saving). Position loss is the abandonment of desirable alternatives, whereas image loss is the fear of appearing ready to make substantial concessions (Pruitt, 1981). Position loss is of concern because norms of “good faith” negotiation make it difficult to reverse concessions once made. After conceding, bargainers are forever forsaking levels of demand that eventually might be accepted and taking moves that eventually might be exchanged for comparable concessions. Image loss is also of concern because it is likely to inspire the opponent to maintain high demands and to adopt a competitive stance in an effort to coerce concessions from bargainers.

The strategy of starting high and conceding slowly prevents position loss by preserving the possibility of reaching agreement at the present level if the opponent eventually concedes to that point. It also guards against image loss by promoting an image of firmness in the opponent’s mind. Interestingly, bargainers sometimes demonstrate good will to keep the negotiation going − they make a few substantial concessions that seek reciprocal concessions (such moves are different from unilateral concessions that seek no quid pro quo). Yet overt concessions are risky and entail the possibility of image loss and position loss. Hence, overt concessions are more likely to be made when the other party is trusted. Clearly, bargainers turn to less risky moves when trust is weak, mainly because there is less concern about both forms of loss. These include fractionated concessions − negotiators start with relatively riskless actions and move on toward increasing levels of risk. They make a few small concessions and wait to see if the opponent follows suit. If so, they then may feel sufficiently confident to venture larger concessions. Practically speaking, three different opening positions (extreme or high, reasonable or moderate, and modest or low) and three levels of concession magnitude (large, substantial, and small) are commonly discussed in the negotiation literature (Pruitt and Carnevale, 1993; Lewicki et al., 2003). They may be associated with a number of concession strategies, notably: 1. starting high and conceding slowly − negotiators adopt an optimistic opening position and make small (initial) concessions; 2. starting high and conceding moderately − negotiators adopt an optimistic opening position and make substantial (initial) concessions; 3. starting reasonable and conceding slowly − negotiators adopt a realistic opening position and make small (initial) concessions. These and other similar strategies are defined by simply considering different tactics. For instance, the strategy “starting high and conceding slowly” is defined by considering the opening negotiation tactic “starting optimistic” and the concession tactic “tough” (but see subsection 3.4). Now, bargainers typically view the world differently − they are not identical in their values, needs and preferences. In particular, they frequently have different strengths of preference for the issues at stake − they place greater emphasis on some key issues and make significant efforts to resolve them favourably. Hence, they often yield on less important or low-priority issues, in the hope that their opponent will make compensating concessions (Raiffa, 1982; Thompson, 2005). A generic low-priority concession making strategy for ai takes the form:   apply Oi (xk ) and offer p1i→j , if ai ’s turn and t = 1      t−1  reject pj→i and quit , if aj ’s turn and Ui (pt−1 si )  j→i ) < Ui (ˆ   i CLP (Oi , Yi , t) = apply Yi (xk , fki ) to Ibi and prepare pti→j if aj ’s turn and Ui (pt−1 si ) j→i ) ≥ Ui (ˆ    t−1   if Ui∗ ≥ 0 accept pj→i else reject,      offer compromise pt , if ai ’s turn and t > 1 i→j

where Ibi ⊂ I is the set of issues that are of lower priority to ai . The definition of a specific strategy for a given time period t > 1 involves basically the specification of a key concession tactic to apply to the issues of low priority (e.g., the tactic “moderate”).

3.3.2

Logrolling Strategies

Most well-intended negotiators tend to believe that, above all, success depends on the creativity to devise agreements that yield considerable gain to both negotiating parties (Thompson, 2005). They see the essence of negotiation as expanding the “pie” of available resources, as pursuing joint gains. To this end, they often create the conditions for effective information exchange, seek insight into the goals and interests of the other party, focus on similarities rather than differences, cultivate common interests, and frequently use problem solving strategies. Simply put, they are essentially value creators − they attempt to probe below the surface of the other party’s true needs to locate mutually superior solutions (Lax and Sebenius, 1986). The host of existing problem solving strategies includes (Pruitt and Kim, 2004): 1. expanding the “pie” − negotiators increase the available resources in a way that all sides can achieve their goals; 2. logrolling − the parties agree to exchange concessions on different issues, with each party yielding on issues that are of low priority to himself and high priority to the other party (such exchanges of concessions are often called trade-offs); 3. nonspecific compensation − one party achieves his goals and pays off the opponent for accommodating his interests. These strategies are major routes − though not the only routes − to the development of integrative and Pareto optimal agreements. Probably, the simplest and most widely used route consists of assessing the differences in valuation of the negotiation issues and to make mutually beneficial trade-offs among issues. Accordingly, logrolling will receive the preponderance of our attention in this paper. Logrolling is possible only when several issues are under consideration and the parties have different priorities among these issues. The parties then agree to exchange concessions on (part or all) of the issues, each party winning on the issues he places greater emphasis. In this way, each party gets the fraction of his demands that he deems most important. Clearly, a theory of logrolling in complex agendas is of particular importance to both human and automated negotiation. However, there are important questions still waiting to be addressed more thoroughly. We highlight the following: which issues will be grouped for the exchange of concessions? Relevant efforts to answer this questions include the theory of appropriate exchange and the principle of equivalence (Pruitt, 1981; Pruitt and Kim, 2004). But it is clear that much more research still needs to be performed. In this work, we consider the following three subsets of the negotiating agenda for each agent ai ∈ A:

• a subset Ii+ , containing the issues of higher priority to ai (and are also believed to be of lower priority to his opponent aj ); • a subset Ii− , containing the issues of lower priority to ai (and are also believed to be of higher priority to aj ); • a subset Ii± , containing the remaining issues of the agenda (I = Ii+ ∪ Ii± ∪ Ii− ). Negotiators have frequently something to offer that is relatively less valuable to them than to their opponent, and thus, the subsets Ii+ and Ii− are typically non-empty. These two subsets contain the logrolling issues, i.e., the issues that can be logrolled to make profitable tradeoffs. By contrast, the subset Ii± contains both the distributive issues (the parties’ interests are directly opposed) and the compatible issues (the parties have coordinated interests). A formal definition of a generic logrolling strategy follows (the definition slightly abuses notation, in the interests of readability). Definition 3.9 (Logrolling Strategy). Let A be the set of negotiating agents, I the negotiating agenda, T the set of time periods, and S the set of possible agreements. Let ai ∈ A be the first agent to submit a proposal, Ti his set of tactics, and aj ∈ A his opponent. Ii+

Let

be the set of issues that are of higher priority to ai (and are believed to be of lower

priority to aj ), Ii− the set of issues that are of lower priority to ai (and are believed to be of higher priority to aj ), and Ii± the remaining issues of the agenda. A logrolling strategy Li : Ti × Ti × Ti × Ti × T → S ∪ {Yes, No, Opt} for ai is a function with the following general form:   apply Oi (xk ) and offer p1i→j ,     t−1   reject pj→i and quit ,       apply Yi+ (xk , fki ) to Ii+    Li = apply Yi± (xk , fki ) to Ii±      apply Yi− (xk , fki ) to Ii− and prepare pti→j      t−1  if Ui∗ ≥ 0 accept pj→i else reject,      offer logrolling solution pti→j ,

if ai ’s turn and t = 1 if aj ’s turn and Ui (pt−1 si ) j→i ) < Ui (ˆ if aj ’s turn and Ui (pt−1 si ) j→i ) ≥ Ui (ˆ

if ai ’s turn and t > 1

where: (i)

p1i→j is the opening offer of ai , pt−1 j→i is the offer of aj for time period t−1 of negotiation, and pti→j is the offer of ai for the next time period t of negotiation;

(ii)

for each issue xk ∈ I, Oi (xk ) is an opening negotiation tactic;

(iii)

Yi+ (xk , fki ), Yi± (xk , fki ) and Yi− (xk , fki ) are concession tactics, and fki is the concession factor of ai for xk (see subsection 3.4, below);

(iv)

Ui (ˆ si ) is the utility of the least-acceptable agreement for ai ;

(v)

t Ui∗ = Ui (pt−1 j→i ) − Ui (pi→j )

”

Clearly, to develop solutions by logrolling, it is useful to have some information about the two parties’ priorities so that concessions can easily be matched up. This information is not always easy to get. The main reason for this is that negotiators often try to conceal their priorities for fear that they will be forced to concede on issues of lesser importance to themselves without receiving any repayment. Noticeably, solutions by logrolling can also be developed by a process of trial and error − the parties systematically offer different packages, keeping their own aspirations as high as possible, until an alternative is found that is acceptable to everyone involved (Lewicki et al., 2003). Hence, logrolling can be insightful or simply emerge from concession making.

Typical

strategies that lead to logrolling solutions either reasonably integrative or fully integrative include: 1. starting high and conceding strategically − negotiators adopt an optimistic opening position, slightly reduce their low-priority demands (and they believe are of high priority to their opponent), and hold firm on their high-priority demands (and they believe are of low priority to their opponent); 2. starting high and negotiating creatively − negotiators adopt an optimistic opening position, substantially reduce their low-priority demands (and they believe are of high priority to their opponent), and hold firm on their high-priority demands (and they believe are of low priority to their opponent); 3. starting high and acting cooperatively − negotiators adopt an optimistic opening position, drastically reduce or drop their low-priority demands (and they believe are of high priority to their opponent), and hold firm on their high-priority demands (and they believe are of low priority to their opponent). The definition of these and other relevant strategies involves basically the specification of particular tactics. For instance, the strategy “starting high and negotiating creatively” is defined by considering the opening negotiation tactic “starting optimistic” and the concessions tactics “moderate” and “stalemate” (but see next subsection).

3.4

Negotiation Tactics

Negotiation tactics are functions that model the short-term moves designed to enact high-level strategies. The following groups of tactics will receive the preponderance of our attention in this paper: 1. opening negotiation tactics − functions that specify the demands to be made at the outset of negotiation; 2. concession tactics − functions that model the concessions to be made throughout negotiation. As mentioned earlier, tactics are functions of a single issue rather than a vector of issues.

3.4.1

Opening Negotiation Tactics

Skilled negotiators often start with high demands to leave room for later movement and hence elicit counterconcessions from their opponent (Pruitt, 1981). High initial demands also protect limits from detection and underestimation (this is a concern about image loss). If limits are detected by the opponent, he may become unwilling to accept a better offer than the leastacceptable one, dooming all higher aspirations. If limits are underestimated, the opponent may become committed to unacceptable demands, fostering breakdown of negotiation. Thus, to avoid these dual dangers, bargainers typically place their demands well above their limits as a sort of smoke screen. Furthermore, high initial demands are also partly designed to protect target points (this is a concern about position loss). Clearly, bargainers often need to move in concert with their opponent toward mutually acceptable agreements. This means starting higher than targets and only moving down to them in coordination with the opponent (Pruitt and Carnevale, 1993). Noticeably, starting high frequently communicates an attitude of toughness that can be reciprocated by the opponent, thus making negotiation “difficult to resolve” (Lewicki et al., 2003). Hence, should bargainers start with a firm, determined stance, or adopt a position of moderateness and understanding? It follows that bargainers often decide how much to demand on the basis of the concessions they expect from their opponent − the farther the opponent is expected to concede, the more will be demanded (this phenomenon is referred to as tracking). In general, three levels of initial demand are commonly discussed in the negotiation literature (Lewicki et al., 2003; Pruitt and Kim, 2004): extreme or high, reasonable or moderate, and modest or low. They have motivated the definition of the following opening negotiation tactics: 1. starting optimistic − specifies a value for an issue close to the optimistic point; 2. starting realistic−specifies a value for an issue in the range defined by the target and the optimistic points; 3. starting pessimistic − specifies a value for an issue in the range defined by the target and the resistance points. A formal definition of the tactic “starting optimistic” follows. Definition 3.10 (Starting Optimistic). Let A = {a1 , a2 } be the set of negotiating agents and I = {x1 , . . . , xn } the negotiating agenda. Let D = {D1 , . . . , Dn } be the set of issue domains. The tactic starting optimistic of an agent ai ∈ A for an issue xk ∈ I takes the form: Oi (xk ) = optki +  where: (i)  > 0 is small; (ii) optki is the optimistic point of ai for xk .

”

The definition of the other two tactics is essentially identical to that of “starting optimistic”, and is therefore omitted.

3.4.2

Concession Tactics

Concessions are a powerful aspect of negotiation − without them, in fact, some researchers consider that negotiation would not exist (Thompson, 2005). If bargainers are not prepared to make concessions, either the opponent must capitulate or negotiation will deadlock. Noticeably, concessions ordinarily result from the belief that they will hasten agreement, prevent the other party from leaving negotiation, or encourage the other to make reciprocal concessions. A concession is usually defined as a change of offer in the supposed direction of the other party’s interests that reduces the level of benefit sought. Concession rate is the speed at which demand level declines over time. A bargainer’s demand level can be thought of as the level of benefit to the self associated with the current demand or offer (Pruitt, 1981; Pruitt and Kim, 2004). Bargainers often enter negotiation expecting concessions (Raiffa et al., 2002). Their opening position may be good for both sides and might have been the final settlement if the parties started negotiation from different points. Even so, bargainers generally resent a take-it-orleave-it approach − an offer that may have been accepted had it emerged as a result of concession making may be rejected when it is thrown on the table and presented as a fait accomply (Bazerman and Neale, 1992; Lewicki et al., 2003). Ample research evidence indicates that the parties feel better about a settlement when negotiation has involved a progression of concessions (Pruitt, 1998; Pruitt and Carnevale, 1993). In practice, bargainers sometimes maintain their aspirations while making a new offer that attempts to achieve those aspirations in a different way. Such an offer is not necessarily a concession, even if it provides greater benefit to the other party (recall that logrolling can be done by trial and error, as discussed in subsection 3.3.2). A formal definition of a generic concession tactic follows (in the interests of readability, and without loss of generality, we consider that ai ∈ A wants to maximize xk ∈ I). Definition 3.11 (Concession Tactic). Let A = {a1 , a2 } be the set of negotiating agents, I = {x1 , . . . , xn } the negotiating agenda, and D = {D1 , . . . , Dn } the set of issue domains. A concession tactic Yi : Dk ×[0, 1] → Dk of an agent ai ∈ A for an issue xk ∈ I is a function with the following general form: Yi (xk , fki ) = xk − fki (xk −limki ) where: (i)

fki is the concession factor of ai for xk ;

(ii)

limki is the limit of ai for xk .

”

Negotiators may consider strikingly different patterns of concessions as negotiation unfolds (Thompson, 2005).

However, as noted earlier, the following three levels of concession

magnitude are commonly discussed in the negotiation literature: large, substantial, and small. To this we would add two other levels: null and complete. Accordingly, we consider the following five concession tactics:

1. stalemate − models a null concession on an issue xk at stake; 2. tough − models a small concession on xk ; 3. moderate − models a substantial concession on xk ; 4. soft − models a large concession on xk ; 5. accommodate − models a complete concession on xk . These and other similar tactics are defined by considering specific values for the concession factor fki . In particular, the “stalemate” tactic is defined by fki = 0 and the “accommodate” tactic by fki = 1. The other three tactics are defined by considering values for fki in different ranges (e.g., the “tough” tactic by fki ∈ ]0.00, 0.05], the “moderate” tactic by fki ∈ ]0.05, 0.15], and the “soft” tactic by fki ∈ ]0.15, 0.20]). At this stage, we hasten to add two explanatory and cautionary notes. First, concession tactics are essentially simple linear functions enabling agents to compute new values for the issues under discussion. The literature presents slightly different functions for modelling concessions during the course of negotiation (see, e.g., Koperczak et al., 1992; Faratin et al., 1998). However, despite the power and elegance of these and other relevant pieces of work, we are aware of no work on explicitly modeling concessions using a concession factor. Second, the successive application of concession tactics allows negotiators to model a wide range of concession patterns (e.g., successive concessions getting smaller to signal that the resistance point is being approached, and thus there is not much left to concede).

3.5

Opponent Information

Effective negotiators typically gather information about their opponent and make use of that information to gain a clear sense of direction on how to proceed. The information which is most relevant depends on the opposing negotiator and the negotiation situation. However, the following key pieces of data are often of great importance (Raiffa, 1982; Roloff and Jordan, 1992; Lewicki et al., 2003): 1. the likely priorities and targets of the opponent; 2. the intended strategy of the opponent; 3. the negotiating history of the opponent. Negotiators may speculate about the priorities and limits of the other party. They can and often do think stereotypically using their own preferences and limits as a guide and assuming that the other is like themselves and wants similar things. In addition, negotiators may speculate about the target and optimistic points of the opponent. However, they can also get relevant information directly from him. To this end, skilled negotiators often exchange information prior to actual negotiation and, if this is not possible, they frequently plan to collect important information during the opening stage of actual negotiation.

Also, it would be most helpful to gain information about the intended strategy (and tactics) of the other party. Although it is unlikely that the opponent reveals any information about his initial strategy outright, negotiators can infer some information from whatever data they collect about him. Thus, style, authority, and objectives may tell negotiators a great deal about what strategy the other party intends to pursue. Furthermore, negotiators should attempt to understand the past behaviour of the other party. How the opponent has acted in the past is usually a good indicator of how he is likely to behave in the future. Therefore, a careful assessment of the negotiating history of the opponent can provide useful clues. To this end, negotiators can communicate to others who know the opponent, or to others who have been in his situation before. However, there is a potential danger in drawing conclusions from this information. Assuming that the opposing negotiator will act in the future as he has acted in the past is just an assumption − he can and do act differently in different circumstances at different times. In theory, it would be extremely useful to have as much information about the other party as possible before opening the negotiation. Although it does take some time and effort to get relevant information, the results are usually more than worth the investment. In reality, however, it may not be possible to obtain any information through either direct communication or other research sources (e.g., asking others who know the opposing negotiator or have negotiated with him). If not, negotiators should attempt to see the negotiation from the perspective of the other party and anticipate what he would want if he was negotiating from that point of view. To this end, we consider that each agent ai ∈ A has a set Bi of beliefs representing information about the agent himself, the opponent aj , and the environment where negotiation takes place. The beliefs about aj can be approximately decomposed into the following categories: • beliefs about the decision-making process (e.g., the utility function); • beliefs about meta-level issues (e.g., the negotiation style and reputation). Three explanatory and cautionary notes are in order here. First, beliefs are expressed in a well defined language L, which is a logical language (the precise nature of L is not relevant to this paper). Second, agents are able to perform both private and communicative actions. The distinguishing feature of private actions is that they cannot be observed by the other negotiating agent. Also, the agent performing a private action has control over it, and thus it is not possible for the other agent to prevent its successful completion. By contrast, communicative actions correspond to sending messages, notably submitting offers and counter-offers. Agents can affect the beliefs of their opponent by communicating with him through message passing. Finally, negotiation is a learnable process − most bargainers can gain greater confidence and improve their skills with repeated experience. Also, several studies suggest that learning plays a key role in negotiation (see, e.g., Lopes et al., 2008). Noticeably, this work seeks to develop autonomous agents with negotiation competence, computationally bounded, and able to explore information about their opponent to exhibit more effective behaviours. To this end, the agents can update their beliefs using a Bayesian updating mechanism, after observing the environment and the behaviour of the other negotiating agent. At this stage, however, the formal treatment of the learning mechanism is postponed to future work.

4

Negotiation in a Multi-Agent Supply Chain System

This section illustrates how the proposed model can be utilized in a multi-agent supply chain system. To this end, let us revisit the Sales-Logistics scenario to demonstrate both how negotiation evolves and how different strategies (and their associated tactics) can be used in a negotiation situation. Sales and Logistics continue at it. David, the director of Sales, is trying to arrange for production of his two new orders, one for 10000 and the other for 5000 men’s suits. Martin, the director of Logistics, is stating that the job will take four months, instead of the two months desired by David’s customer. David and Martin have developed well-laid plans before starting to negotiate and are now seeking a solution for their dispute. For illustrative purposes, we consider the negotiation process from the viewpoint of David. There are four major issues of concern: quantity 1, date 1, quantity 2 and date 2. The first two issues are the most important to David due to the inherent customer demands − he wants fast action on the 10000 suit order. By being clear on preferences, David is more likely to yield on issues that are less important to him, i.e., issues associated with the 5000 suit order, rather than to yield on points strategically demanded by his opponent. Also, after a period of consultation with the customer, David concludes that he is overly firm about the 10000 suit order (and is willing to wait up to five weeks for 10000 suits, or ultimately 9500 suits), but he is moderately firm about the 5000 suit order (and is willing to wait up to six weeks for only 4000 suits). Table 2 shows the four issues, the (normalized) weights, the limits, and the target points of the Sales agent. The negotiation process involves an iterative exchange of offers and counter-offers. As noted earlier, the decision to accept or reject an offer in a given period t depends on both the utility of the offer and the utility of the counter-offer to submit in period t+1. Also, the agents rate offers using both a multi-issue utility function (recall definition 3.7) and a scoring or single-issue (marginal) utility function. The literature discusses a number of different single-issue functions (see, e.g., Raiffa, 1982; Osborne and Rubinstein, 1990; Kraus, 2001). In this case study, we simply consider a continuous, strictly monotonic, and linear function. A formal definition of this function follows (without loss of generality, we consider that an agent ai wants to maximize an issue xk at stake). Definition 4.1 (Single-Issue Utility Function). Let A = {a1 , a2 } be the set of negotiating agents and I the negotiating agenda. Let D be the set of issue domains. The (marginal) utility function Vki : Dk → [0, 1] of an agent ai ∈ A for an issue xk ∈ I takes the form: Vki (xk ) =

xk − limki maxk − mink

where: (i)

limki is the limit of ai for xk ;

(ii) mink and maxk are the minimum and maximum values for xk , respectively.

”

Table 2: Major issues, preferences, limits and targets (Sales agent) Negotiation Issue

Weight

Limit

Target Point

quantity 1

0.350

9500

10000

date 1

0.300

1.25 (5 weeks)

1.00 (4 weeks)

quantity 2

0.175

4000

5000

date 2

0.175

1.50 (6 weeks)

1.00 (4 weeks)

Now, we take up a few strategies, one at a time, and examine their nature and their impact on the negotiation outcome. Concession strategies aim at (partially) reducing negotiators’ demands. Generally speaking, negotiators who demand too much will often fail to reach agreement and thereby do poorly. Those who demand too little will usually reach agreement but achieve low benefits (Pruitt and Carnevale, 1993). For instance, if David is overly firm and keeps on demanding a two-month schedule, negotiation may break down, leaving him in the lurch. But if he is overly soft and concedes a lot, Martin will win and he will lose his customer. The most successful negotiators are often those who are moderately firm and hence are between these two extremes (Pruitt and Kim, 2004). However, if negotiators are moderately firm and do not try to devise new alternatives by means of problem solving, the result will probably be a compromise agreement with low benefits to both sides. For instance, David and Martin can agree on a five-week schedule for 10000 men’s suits and a six-week schedule for 5000 men’s suits. Figure 1 shows the joint utility space for David and Martin. The abscissa represents the utility to David, and the ordinate represents the utility to Martin. The solid line OCO’ represents the Pareto optimal frontier. The small squares depict a few options for settling the issues at stake. The aforementioned compromise agreement is represented by point A and provides a (normalized) benefit of 0.525 to David and 0.475 to Martin. Problem solving strategies formalize specific ways to find mutually acceptable solutions. As noted, it is of higher priority for Sales to get fast action on the 10000 suit order than the 5000 suit order. Suppose now that it is of higher priority for Logistics to handle the 5000 suit order (and to avoid the 10000 suit order). These two departments have the makings of a logrolling deal − each party can yield on issues that are of low priority to himself and high priority to the other party. Accordingly, David and Martin can reach the following solution: a 4-week schedule for 9750 suits and a 6-week schedule for 4500 suits. This integrative agreement is represented by point B in Figure 1 and provides a (normalized) benefit of 0.562 to each agent. Hence, it is better for both agents than the compromise agreement represented by point A. As we have constantly stressed, compromising is a conflict management strategy for satisficing − making sure that each party gets something out of the solution − but not everything.

Figure 1: Joint utility space for the Sales-Logistics negotiation situation Noticeably, logrolling strategies can permit negotiators to fully exploit the differences in the valuation of the issues to capitalize on fully integrative or Pareto optimal agreements. In this way, David and Martin can pursue specific logrolling strategies (e.g., starting high and acting cooperatively) and agree on a four-week schedule for 10000 suits and a six-week schedule for 4000 suits. This agreement lies along the efficient frontier and is represented by point C in Figure 1 − it provides a (normalized) benefit of 0.65 to each party.

5

Related Work

Artificial intelligence (AI) researchers have investigated the design of agents with negotiation competence from two main perspectives: a theoretical or formal mathematical perspective and a practical or system-building perspective. Researchers following the theoretical perspective have attempted mainly to develop formal models of negotiation, i.e., models for describing, specifying, and reasoning about the key features of negotiating agents. To this end, they have drawn heavily on game-theoretic and economic methods (see, e.g., Rosenschein and Zlotkin, 1994; Sandholm, 1999; Kraus, 2001; Fatima et al., 2005). On the other hand, researchers following the practical perspective have attempted mainly to develop computational models of negotiation, i.e., models for specifying the key data structures of negotiating agents and the processes operating on these structures. They have drawn heavily on social sciences techniques for understanding interaction and negotiation (see, e.g., Muller, 1996; Jennings ¨ et al., 2001; Rahwan et al., 2004; Lopes et al., 2008). The theoretical models have some highly desirable properties such as maximizing social welfare, Pareto efficiency, stability, and the ability to guarantee convergence.

However,

most models are essentially static in the sense that they primarily focus on the outcome of negotiation rather than on the negotiation process itself − negotiation is mainly viewed as a process used to select a solution from a set of candidate solutions. Simply put, the negotiation

process does not itself define the set of candidate solutions. Also, most models work with abstract problems and often fail to capture the richness of detail that would be necessary to successfully apply them in realistic domains. Furthermore, most models make the following restrictive assumptions: (i) the agents are rational, (ii) the set of candidate solutions is fixed and known by all the agents, (iii) each agent knows the other agents’ potential payoffs for all candidate solutions, and (iv) each agent knows the other agents’ potential attitudes toward risk. These assumptions fail in most realistic environments due to the limited processing and communication capabilities of existing systems. Most computational models are being used successfully in a wide variety of real-world domains. These models exhibit the following desirable features: (i) they focus on both the negotiation process and the negotiation outcome, (ii) they are based on realistic assumptions, and (iii) they make use of moderate computational resources to find acceptable solutions (according to the principles of bounded rationality, Simon 1981). However, most models have a number of limitations. Firstly, they often lack a rigorous theoretical underpinning − they are essentially ad hoc in nature. Secondly, they often lead to outcomes that are sub-optimal. Finally, there is often no precise understanding of how and why they behave the way they do. Consequently, they need extensive evaluation. Overall, various researchers have developed models that incorporate specific protocols (notably, the alternating offers protocol) and libraries of negotiation strategies (notably, concession strategies).

Faratin et al.

(1998), for example, have presented a model for

bilateral service-oriented negotiation that defines a range of strategies (functions that map a matrix of real numbers ranging from zero to one into another similar matrix) and three groups of concession tactics: time dependent (functions of time), resource dependent (functions of limited resources), and behavior dependent or imitative (functions of the opponent’s behavior).

The authors have also defined a linear algorithm for making issue trade-offs

(Faratin et al., 2002). They have performed several experiments to empirically evaluate these components of the model in various types of negotiation situations. However, despite the power and elegance of this and other existing negotiation models, we are aware of no similar efforts to define strategies as functions that specify the tactics to be used both at the outset and throughout negotiation. Tactics, in turn, are defined as functions that specify the short-term moves to be made at each point of negotiation. Our interest lies mainly in formalizing important strategies and tactics motivated by rules-of-thumb distilled from good behavioral practice in real-life negotiations.

6

Conclusion

This paper has presented the key features of a negotiation model for software agents. The model handles two-party and multi-issue negotiation, manages important activities that negotiators often perform before starting to negotiate, and formalizes relevant procedures studied in the social sciences and frequently used in real-life negotiation. Conceptually, the model incorporates a bilateral negotiation protocol, a set of negotiation strategies, and a set of negotiation tactics. The protocol is an alternating offers protocol. The strategies are

computationally tractable functions that define the tactics to be used both at the beginning and during the course of negotiation. The words “computationally tractable functions” presume that agents are able to compute the strategies in a reasonable amount of time. Furthermore, at every period of negotiation, the strategies state whether bargaining should continue or terminate. Specifically, the strategies state whether agents should opt out of negotiation or continue bargaining. The tactics are functions that specify the individual moves to be made at each point of the negotiation process. They are structured, directed, and driven by strategic considerations. Autonomous agents equipped with the negotiation model are currently being developed. Our aim for the future is to perform a number of experiments to empirically evaluate the key components of the agents. Also, notice that the task of designing and implementing agents with negotiation competence involves the consideration of insights from multiple relevant research areas. Accordingly, we also intend to develop an interdisciplinary framework for automated negotiation − game-theoretic (strategic) and behavioural negotiation theories should mutually reinforce each other and contribute to richer negotiators.

Acknowledgments The authors are grateful to two anonymous reviewers of this paper. Their valuable comments and suggestions helped us to substantially improve the accuracy and readability of the paper.

References Bazerman, M. and Neale, M. 1992. Negotiating Rationality, Free Press, New York. Bichler, M., Kersten, G. and Strecker, S. 2003. Towards a structured design of electronic negotiations, Group Decision and Negotiation 12: 311–335. Faratin, P., Sierra, C. and Jennings, N. 1998. Negotiation decision functions for autonomous agents, Journal of Robotics and Autonomous Systems 24(3-4): 159–182. Faratin, P., Sierra, C. and Jennings, N. 2002. Using similarity criteria to make issue trade-offs in automated negotiations, Artificial Intelligence 142: 205–237. Fatima, S., Wooldridge, M. and Jennings, N. 2005. A comparative study of game theoretic and evolutionary models of bargaining for software agents, Artificial Intelligence Review 23: 185–203. Fishburn, P. and Rubinstein, A. 1982. Time preferences, International Economic Review 23(3): 677–694. Fox, M., Barbuceanu, M. and Teigen, R. 2000. Agent-oriented supply chain management, The International Journal of Flexible Manufacturing Systems 12: 165–188. Holmes, M. 1992.

Phase structures in negotiation, in L. Putnam and M. Roloff (eds),

Communication and Negotiation, Sage, London, pp. 83–105.

Jennings, N., Faratin, P., Lomuscio, A., Parsons, S., Wooldridge, M. and Sierra, C. 2001. Automated negotiation: Prospects, methods and challenges, Group Decision and Negotiation 10: 199–215. Jennings, N., Sycara, K. and Wooldridge, M. 1998.

A roadmap of agent research and

development, Autonomous Agents and Multi-Agent Systems 1: 7–38. Keeney, R. and Raiffa, H. 1976. Decisions with Multiple Objectives: Preferences and Value Tradeoffs, John Wiley & Sons, Chichester. Klein, M., Faratin, P., Sayama, H. and Bar-Yam, Y. 2003. Negotiating complex contracts, Group Decision and Negotiation 12: 111–125. Koperczak, Z., Matwin, S. and Szpakowicz, S. 1992. Modelling negotiation strategies with two interacting expert systems, Control and Cybernetics 21(1): 105–130. Kraus, S. 2001. Strategic Negotiation in Multi-Agent Environments, MIT Press, Cambridge. Lax, D. and Sebenius, J. 1986. The Manager as Negotiator, The Free Press, New York. Lewicki, R., Barry, B., Saunders, D. and Minton, J. 2003. Negotiation, McGraw Hill, New York. Lopes, F., Mamede, N., Novais, A. Q. and Coelho, H. 2002.

A negotiation model for

autonomous computational agents: Formal description and empirical evaluation, Journal of Intelligent & Fuzzy Systems 12: 195–212. Lopes, F., Mamede, N., Novais, A. Q. and Coelho, H. 2004.

Negotiation strategies for

autonomous computational agents, European Conference on Artificial Intelligence (ECAI04), IOS Press, Amsterdam, pp. 38–42. Lopes, F., Mamede, N., Novais, A. Q. and Coelho, H. 2005. Negotiation among autonomous agents: Experimental evaluation of integrative strategies, Portuguese Conference on Artificial Intelligence, IEEE Computer Society Press, Los Alamitos, CA, pp. 280–288. Lopes, F., Wooldridge, M. and Novais, A. Q. 2008.

Negotiation among autonomous

computational agents: Principles, analysis and challenges, Artificial Intelligence Review 29: 1–44. Muller, H. 1996. Negotiation principles, in G. O’Hare and N. Jennings (eds), Foundations of ¨ Distributed Artificial Intelligence, John Wiley & Sons, Chichester, pp. 211–229. Osborne, M. and Rubinstein, A. 1990. Bargaining and Markets, Academic Press, San Diego. Pechoucek, M. and Marik, V. 2008. Industrial deployment of multi-agent technologies: Review and selected case studies, Autonomous Agents and Multi-Agent Systems 17: 397–431. Pruitt, D. 1981. Negotiation Behavior, Academic Press, New York. Pruitt, D. 1998. Social conflict, in D. Gilbert, S. Fiske and G. Lindzei (eds), The Handbook of Social Psychology, Vol. 2, McGraw-Hill, New York, pp. 470–503.

Pruitt, D. and Carnevale, P. 1993. Negotiation in Social Conflict, Open University Press, Philadelphia. Pruitt, D. and Kim, S. 2004. Social Conflict: Escalation, Stalemate, and Settlement, McGraw Hill, New York. Rahwan, I., Ramchurn, S., Jennings, N., McBurney, P., Parsons, S. and Sonenberg, L. 2004. Argumentation-based negotiation, The Knowledge Engineering Review 18: 343–375. Raiffa, H. 1982. The Art and Science of Negotiation, Harvard University Press, Cambridge. Raiffa, H., Richardson, J. and Metcalfe, D. 2002. Negotiation Analysis, Harvard University Press, Cambridge. ´ H. 2008. Retrieving the structure of utility graphs used in multiRobu, V. and La Poutre, item negotiation through collaborative filtering of aggregate buyer preferences, in T. Ito, H. Hattori, M. Zhang and T. Matsuo (eds), Rational, Robust and Secure Negotiations in Multi-agent Systems, Vol. 89, Springer Series on Computational Intelligence, Heidelberg, pp. 147–168. Roloff, M. and Jordan, J. 1992.

Achieving negotiation goals: The fruits and foibles of

planning ahead, in L. Putnam and M. Roloff (eds), Communication and Negotiation, Sage Publications, London, pp. 21–45. Rosenschein, J. and Zlotkin, G. 1994.

Rules of Encounter: Designing Conventions for

Automated Negotiation Among Computers, MIT Press, Cambridge. Sandholm, T. 1999.

Distributed rational decision making, in G. Weiss (ed.), Multi-Agent

Systems – A Modern Approach to Distributed Artificial Intelligence, MIT Press, Cambridge, pp. 201–259. Simon, H. 1981. The Sciences of the Artificial, MIT Press, Cambridge. Thompson, L. 2005. The Mind and Heart of the Negotiator, Prentice-Hall, Englewood Cliffs. Tutzauer, F. 1992. The communication of offers in dyadic bargaining, in L. Putnam and M. Roloff (eds), Communication and Negotiation, Sage Publications, London, pp. 67–82. Zhou, Z., Chan, W. and Chow, J. 2007. Agent-based simulation of electricity markets: a survey of tools, Artificial Intelligence Review 28: 305–342.