A Knowledge Based Methodology for Designing Robust Electronic Markets

A research and education initiative at the MIT Sloan School of Management A Knowledge Based Methodology for Designing Robust Electronic Markets Paper...
Author: Shannon Lang
3 downloads 2 Views 692KB Size
A research and education initiative at the MIT Sloan School of Management

A Knowledge Based Methodology for Designing Robust Electronic Markets Paper 126

Mark Klein Chrysanthos Dellarocas Juan Antonio Rodriguez-Aguilar

August 2001

For more information,

please visit our website at http://ebusiness.mit.edu or contact the Center directly at [email protected]

or 617-253-7054

A Knowledge-Based Methodology for Designing Robust Electronic Markets Mark Klein Center for Coordination Science Massachusetts Institute of Technology [email protected] Chrysanthos Dellarocas Sloan School of Management Massachusetts Institute of Technology [email protected] Juan Antonio Rodriguez-Aguilar Center for Coordination Science Massachusetts Institute of Technology [email protected]

1. The Challenge The ‘new economy’ of ubiquitous electronic markets entails new risks and demands novel responses. Such markets, for one, give unprecedented scope to open participation by software agents, whose unique strengths and weaknesses enable a whole range of novel failure modes. Software agents can, for example, be replicated almost indefinitely, operate extremely rapidly, and can be very difficult to trace back to their humans ‘owners’. These attributes make electronic markets susceptible to such phenomena as denial of service attacks, which have halted operations at many major businesses [1] [2], in addition to many others such as ‘bid snatching’ (wherein a set of malicious colluding agents effectively halt another agent by using attractive bids to fraudulently ‘snatch’ and non-perform on all the targets’ subcontracts), ‘bid collision loops” (where a pair of agents halt a Dutch auction by establishing an infinite loop of colliding bids), and so on. The range of attack types and corresponding responses is growing (see [3] for example) and seems limited only by human creativity. Even the mechanisms used to help avoid malicious agent behavior, such as reputation servers, are themselves prone to such attacks as collusive reputation manipulation [4]. Electronic markets also give unprecedented scope to the deployment of relatively untried market mechanisms whose vulnerabilities have not been fully understood. Large-scale remote bidding has been enabled by ubiquitous telecommunications, for example, and combinatoric auctions have only

recently made practical by the identification of tractable winning bid identification algorithms [5]. The combination of software agents and novel market mechanisms, finally, leads to the potential for emergent dysfunctions, such as resource allocation thrashing (when agents spend a disproportionate amount of time re-allocating resources among them) [6], wild inventory swings [7], and price oscillations [8]. It has been argued, for example, that the 1987 stock crash was due in part to the action of computer-based “program traders” that were able to execute trade decisions at unprecedented speed and volume, leading to unprecedented stock market volatility [9]. The consequences for unwary electronic market

developers thus include the potential for serious disruptions in the operations of targeted market participants and/or the markets themselves. This paper describes a general methodology for identifying and finding suitable responses for potential failure modes (henceforth called ‘exceptions’) in multi-agent coordination mechanisms, and shows how it can be applied to the particular context of electronic markets. We present the methodology itself in section 2, and then describe (in section 3) how an augmentation of the MIT Process Handbook captures exception handling expertise in a way that can greatly increase the speed and comprehensiveness of exception analysis. Both sections will make use of many examples concerning the market mechanisms known as auctions. 2. Our Exception Analysis Methodology Our exception analysis methodology is based on the insight that coordination fundamentally involves the making of commitments [10] [11] [12], and that exceptions (i.e. coordination failures) can as a result be viewed as violations of the commitments agents require of one another. Exception analysis thus consists of the following three steps: 1. Identify the commitments agents require of one another in the coordination mechanism 2. Identify the ways these commitments can be violated (i.e. the exceptions) 3. Identify the ways these exception can be handled (i.e. the exception handlers) We consider each of these steps in the paragraphs below. To make the discussion more concrete, we will describe them in context of the well-known auction-based task-sharing mechanism known as the “Contract Net” (CNET) [13] [14] [15] [16] [17]:

Contractor Role

Subcontractor Role

Create RFB (Request For Bids)

Sen

dR FB

d Sen

Select Bid

Aw

ard

d Sen

Create Bid

Bid

Con

trac

t

ults

Res

Perform Work

Receive Results

Figure 1. The Contract Net Coordination Mechanism. In this protocol, an agent (the “contractor”) identifies a task that it cannot or chooses not to do locally and attempts to find another agent (the “subcontractor”) to perform the task. It begins by creating a Request For Bids (RFB) which describes the desired work, and then sending it to potential subcontractors (typically identified using an agent known as a ‘matchmaker’). Interested subcontractors respond with bids (specifying such issues as the price and time needed to perform the task) from which the contractor selects a winner. This is thus a first-price sealed-bid auction. The winning agent, once notified of the award, performs the work (potentially subcontracting out its own subtasks as needed) and submits the results to the contractor. We can see that there are thus at least three key agent types in CNET: the contractor, subcontractor, and matchmaker. Identifying Commitments: Commitments can be identified by finding all the places in the protocol where one agent depends on some other agent in order to discharge its own responsibilities successfully [10]. There are in general two kinds of commitments: design-time commitments that are part of the mechanism definition (e.g. that a bidding agent will send only one bid at a given price level), as well as run-time commitments created as a result of the operation of the mechanism (e.g. that an agent will in fact perform the task it was allocated with the contracted cost quality and delivery time). The first commitment in CNET arises when the contractor queries the matchmaker to get the list of agents potentially appropriate for performing a subtask: C1:

matchmaker provides contractor with correct candidates in a timely way

In order for the matchmaker to discharge this commitment, it in turn requires commitments on its own, including: C2: C3: C4: C5: C6:

contractor provides matchmaker with correct skills requirements subcontractors provide matchmaker with up-to-date skills descriptions messaging-mechanism provides matchmaker with timely and correct delivery of results to contractor host-computer provides matchmaker with sufficient computing cycles coder provides matchmaker with correct code

Each of these commitments, in turn, may imply further commitments that must be honored in order for them to be discharged correctly. For example, in order to discharge commitment C5, a host computer requires that: C7:

system-operator ensures that host has sufficient available capacity

The process of tracing what commitments are required to discharge others allows us, in principle, to identify commitments exhaustively, even those that are often left implicit (e.g. those involving the messaging-mechanism and system-operator). Note that it is critical that commitments be enumerated from the idealized perspective of each agent. For example, in an auction the seller ideally wants the following commitment: C8:

bidders provide seller with bids up to their true valuation for good

even though in many situations the commitment target (the bidder) may have no intention of fulfilling it (e.g. in auctions where true-value revelation is not the dominant strategy1). Many important exceptions represent violations of ideal-case commitments. Identifying Commitment Violations (Exceptions): The next step is to identify, for each commitment, how that commitment can be violated, i.e. what its characteristic exceptions are. The basic set of possible exceptions can be identified simply as the possible negations of the commitment itself. For example, the message-sender is responsible for discharging three commitments: delivering the right message to the right place at the 1

A dominant strategy is a strategy that it is rational for an agent to pursue no matter what strategies are pursued by the other agents.

right time. There are thus three main exceptions: wrong message (the message is garbled), wrong place (the message is delivered to other than the intended target), and wrong time (the message is either too early or, more likely, excessively delayed). As another example, the subcontractor commitment to perform a task with a given quality cost and duration can be violated by late results, sub-contractual quality, or higher-thanproduced cost (it is logically possible, though unlikely, that early results, unexpectedly high quality or low costs could also be exceptions). It is not enough to identify these basic exceptions, however, since by the time the commitment is violated it is often too late to deal effectively with it. Ideally, one would prefer to identify, as early as possible, the events that might lead to commitment violations, i.e. the causes of the root exceptions. We consider these causes to be exceptions as well, since they also represent system states that can lead to commitment violations. Unfortunately, it is not possible to exhaustively identify all possible exception causes deductively. For example, commitment C5 above (host-computer provides matchmaker with sufficient cycles) could be violated because the host computer has experienced a software virus or distributed denial of service attack. Before such attacks were invented, we could not expect to have simply inferred them from the commitment description. Another example is emergent dysfunctions like those mentioned above. There is as yet no complete analytic theory concerning which kinds of coordination mechanisms are apt to face which kinds of emergent dysfunctions. These limitations are a fundamental property of open systems (i.e. ones that are built from ‘black box’ components). This introduces a experiential component into exception analysis; one must rely on one’s previous experience in order to be able to fully identify possible exception types, and an exhaustive identification may not be possible. Identifying Exception Handlers: Once we have identified the exceptions that potentially characterize a given coordination mechanism, we are ready to identify possible techniques for handling these exceptions. If the coordination mechanism does not currently provide any handlers for an important exception type, then the mechanism designer needs to select one or more handlers and incorporate them into the coordination mechanism being analyzed. There is a vast range of possible exception handling techniques. One can argue in fact that a very significant proportion of human organizational innovations, including the police, law courts, disaster relief agencies, insurance, contracts and so on, can all be viewed as being exception handling techniques. As with exception causes, however, the range of possible handlers appears to be limited only by human creativity and there appears to be no way to identify all potentially useful exception handlers from first principles.

3. Exploiting A Repository of Exception Handling Knowledge The fact that exception and handler identification can be time-consuming as well as at least partly experiential in nature has led us to explore whether it is possible to systematically accumulate this knowledge so that we can analyze new mechanisms more quickly and completely. We have been able to do so using a substantively extended version of the MIT Process Handbook , a process knowledge repository which has been under development at the Center for Coordination Science (CCS) for the past six years [18] [19]. In the following sections we will present the core concepts underlying the Handbook, describe how these concepts and associated tools were extended to enable more effective and efficient exception analysis, and conclude with an extended example of how this methodology was applied successfully to a real-life case. 3.1. Underlying Process Handbook Concepts The Handbook takes advantage of three simple but powerful concepts to capture and organize process knowledge: attributes, decomposition, dependencies, and specialization. Decomposition: Also like most process modeling techniques, the Handbook uses the notion of decomposition: a process is modeled as a collection of activities that can in turn be broken down (“decomposed”) into subactivities that themselves, potentially, are processes. Dependencies: Another key concept we use is that coordination can be viewed as the management of dependencies representing flows (of control, data, material …) between activities [18]. Dependencies can be arranged into a taxonomy, where the three primitive types include: “flow” which represent one-to-one resource flows (e.g. transferring money between accounts), “sharing” which represent one-to-many resource flows (e.g. allocating investment money to stocks) , and “fit” which represent many-to-one resource flows (e.g. fitting together the designs for multiple subsystems of a given system). Every dependency is “managed” by an associated coordination mechanism, which is simply the process that controls the resource flow and thereby coordinates the activities connected by the dependency. Different kinds of dependencies have different potentially appropriate coordination mechanisms, e.g.:

Dependency Type Flow Fit Sharing

Possible Coordination Mechanisms Make to order (just-in-time) Make to forecast and stockpile Pre-defined subsystem design interfaces Post-hoc design conflict resolution First-come, first-served Market-like bidding

The key advantage of this conceptualization is that it allows us to distinguish the domainspecific activities of a process from the generic coordination functions, thereby enabling cross-domain sharing of coordination mechanism ideas. Process Specialization: The final key concept is that processes can be arranged into a taxonomy, with very generic processes at one extreme and increasingly specialized processes at the other. Processes are organized based on their function, so that processes with similar purposes appear close to each other. This facilitates finding and comparing alternative ways for performing functions of interest, thereby fostering easy transfer of ideas. Sibling processes can be grouped into “bundles” with tradeoff tables that capture the relative pros and cons of these alternatives. 3.2. Extending the Handbook to Capture Exception Handling Knowledge While the Handbook as described above is well-suited for describing processes, it does not capture information concerning possible exceptions in these processes. The key innovation required to do so is the addition of the exception taxonomy, a specialization hierarchy of exception types [20] [21]. Using this additional element we can extend the Handbook schema as follows (Figure 2):

processes

requires

coordination mechanisms

exception handlers

includes dependency

is handled by

dependencies (commitments)

exception types has exception

is caused by

Figure 2. The Schema for Capturing Exception Handling Knowledge. A coordination mechanism is simply a kind of process that involves multiple agents, and resides therefore in the Handbook process taxonomy. Commitments are inter-agent dependencies and reside in the Handbook’s dependency taxonomy. Every commitment can be mapped (using the has-exception link) to the characteristic ways it can be violated (its exceptions); these exceptions reside in the exception taxonomy. Finally, all exceptions can be mapped (using an is handled by link) to the processes (exception handlers) that can be used to deal with them. Handlers are themselves members of the process taxonomy, and potentially have their own characteristic commitments, exceptions, and exception handlers. This simple augmentation of the Handbook schema has proven adequate, in our experience, to capture all important aspects of exception handling expertise.

3.3. Using the Knowledge Base for Exception Analysis Let us now consider how an exception handling knowledge base organized using the schema described above can improve the speed and completeness of exception analysis. The key idea is that one can learn much about an entity (e.g. a process or exception type) by looking at generalizations of that entity. In other words, we can expect that

information (e.g. exceptions or handlers) associated with the generalization is often inherited by its specializations. The Coordination Mechanism Taxonomy: In order to exploit this effect, we have been building a taxonomy of coordination mechanisms (Figure 2):

Figure 3. A fragment of the coordination mechanism taxonomy. Every coordination mechanism is linked to the exception types that characterize it; a mechanisms’ exceptions are inherited by its specializations unless explicitly over-ridden. Every exception is annotated with its typical impact on the associated coordination mechanism. Auctioneer fraud, for example, typically results in increased buyer costs in sealed-bid second-price auctions (when the auctioneer inserts a bogus second bid that is just below to the winning bid). Our recent efforts have focused on developing the branch of the coordination mechanism taxonomy that concerns auctions. We have defined an abstract auction model (building on [22] [23] [24]) consisting of the following steps: • • • • •

Bid call. How buyers are invited to submit bids. Ask call. How sellers are invited to submit “asks” (desired bids). Bid collection. How bids are collected by auctioneer. Ask collection. How asks are collected by auctioneer. Winner determination. How the buyer-seller match is found.

• • •

Clearing. How the end of the bidding round is determined. Information revelation. How/when bid information is revealed to bidders. Closing. When to close the auctioning permanently.

All known auction types can be represented as specializations of this abstract model. A typical Dutch auction [25] for example, uses downward bidding for 'bid call', includes no 'ask call' step, determines the winner as the first bid received as the auctioneer calls prices downwards, clears the bidding round when a bid above the reservation price is received or the reservation price is reached, reveals the winning bid (and possibly the winner's identity) after clearing, and resolves ties by selecting randomly one of the bidders involved. A typical multi-unit Japanese auction [26] involves a calls for bids every time the auctioneer raises the price, includes no 'ask call' step, collects bids as the number of units requested by each bidder, all bidders which do not explicitly request the auctioneer to drop out as part of the winning set, does not reveal any information about bidders' bids during bidding, and clears the bidding round when the total demand from bidders matches the supply (number of units of the lot at auction). Our analysis to date shows that many auction exceptions are inherited by all specializations of the abstract auction model. All auctions, for example, are potentially liable to the ‘auctioneer agent crashes’ exception, as well as impersonated bids (where a bidder presents a bid as if it came from someone else), multiple bids from a single buyer at a given price (a protocol violation that may represent a denial of service attack), unsupported bids (where the winner is unable to pay the winning price), or too few bidders (which may reduce the competitiveness of the auction below a level the auctioneer or seller is willing to accept). In other cases we are able to identify exceptions that apply to an entire subclass of auction protocols. One example concerns information revelation. In any auction where bid information is not revealed, for example (e.g. as with closed bid auctions such as the Vickrey [25]), we have the possibility of buyer-auctioneer collusion, since bidders will not be in a position to determine the correct winners on their own [27]. For another example, all ascending price auctions (e.g. English or Japanese protocols) are prone to non-termination of the winner determination step if enough agents (due either to bugs or malice) simply do not ever drop out. Some exceptions, however, are idiosyncratic to a quite specific class of auction mechanism because that mechanism involves unusual commitment types. Japanese auctions, for example, rely on each bidder explicitly indicating that it is reducing its demand or dropping out entirely, as the price rises. If an active bidding agent crashes then

the auctioneer will still consider its bids active, prices will be driven above the appropriate level, possibly infinitely (i.e. non-terminating) if the dead agent’s last demand exceeded the total supply, and the dead agent may be inappropriately selected as a winner. The exception handling knowledge base stored in the Handbook thus does not guarantee that all exceptions can be identified simply by inheritance from the abstract auction model. The fact that so many exceptions apply to all or most auction types is a good result because it means that we immediately know a lot about the potential failure modes in auctions we have never analyzed before. Even for unusual auction variants, much of the analysis effort is obviated and the market designer can focus on identifying exceptions involving the commitment types idiosyncratic to that variant. Note that even though an unusual action variant may not already reside in the process taxonomy, some important portion(s) of it may. For example, the Handbook taxonomy includes a simple generic model for a query/response interaction between two agents, whose constituent commitment include the following: C10: C11: C12: C13: C14: C15:

agent1 provides agent2 with correct query agent2 provides agent1 with correct response messaging-mechanism provides agent2 with timely and correct delivery of results host-computer provides agent2 with sufficient computing cycles coder provides agent2 with correct code system-operator ensures that host has sufficient available capacity

Any query in a coordination mechanism can be viewed as an instance of this abstract process, and inherits as a result the exceptions associated with it. The Exception Taxonomy: Exceptions, like coordination mechanisms, are arranged into a taxonomy (Figure 4):

Figure 4. A fragment of the exception type taxonomy. Exception types are linked, in turn, to the exception handling processes suitable for dealing with them. Handlers are inherited down the exception taxonomy unless specifically over-ridden. Since different handlers have different strengths and weaknesses, exception types can include tradeoff tables that present the relative pros and cons of the different handlers for dealing with that exception. There are three main classes of exceptions: •

Infrastructure commitment violations, which represent violations of commitments made by the infrastructure to provide dependable communication and computation.

Examples include crashed host computers, and unreliable or slow communication links. •

Agent commitment violations, which represent violations of commitments agents make to each other. Examples include agents performing tasks late, doing sub-quality work, or not adhering to the agreed-upon coordination protocols.



Systemic commitment violations, which represent violations of commitments made by the system operator to create an environment well-suited to the tasks at hand. Examples include not populating the market with agents containing all the skills needed to perform the tasks at hand, or allowing emergent dysfunctions.

A key benefit of the exception taxonomy is that one can identify potentially useful handlers for an exception by looking at those linked to its generalizations in the taxonomy. For example, an agent not paying for a purchased good can be classified as an ‘agent commitment violation’ exception, which is linked to the generic exception avoidance mechanism called ‘reputation mechanisms’. To give another example, excessive bandwidth use in a network can be classified as a ‘tragedy of the commons’ exception (a type of emergent dysfunction exception wherein agents over-use a public resource). The tragedy of the commons exception can be avoided by ‘privatizing’ the resource e.g. by charging use fees. It is our experience that a relatively compact corpus of generic exception handlers covers a wide range of exception types, which is good because that means a relatively small investment in creating the exception and exception handler taxonomies can be amortized over a broad range of uses. The Exception Handler Taxonomy: The exception handing process taxonomy (Figure 5) is where the bulk of the exception handling knowledge base content resides:

Figure 5. A subset of the exception handling process taxonomy. There are four main classes of exception handlers, divided into two pairs. If a exception has not yet occurred, we can use: • Exception anticipation processes, which uncover situations where a given class of exception is likely to occur. • Exception avoidance processes, which reduce or eliminate the likelihood of a given class of exception. If the exception has already occurred, we can use: • •

Exception detection processes, which detect when an exception has actually occurred. Exception resolution processes, which resolve an exception once it has happened.

One of the key advantages of arranging exception handlers into a process taxonomy is that it allows us to generate new and potentially useful handlers by re-combining the elements of the taxonomy into novel configurations. The methodology for generating novel process alternatives is described in detail in [28].

Closing the Loop: Exception handlers can themselves have their own characteristic exceptions, which can be of great practical importance. We have already encountered, for example, the idea that reputation mechanisms can be sabotaged. This is because they typically assume the following idealized commitments: C16: C17:

agents provide all other agents with honest ratings agents perform for other agents without discrimination i.e. they are no more likely to non-perform for one agent than another

We can easily imagine agents that violate these commitments, however. Imagine for example a clique of agents that (perhaps fraudulently) provide each other with a large number of highly positive ratings, while selectively non-performing for a few targeted customers. The average ratings of such misbehaving agents will remain high, especially if, as is sometimes done, outlier reputation scores are discarded by the reputation service. Handlers exist for dealing with reputation manipulation exceptions, however, including for example anonymizing the rated agents to foil ‘bad-mouthing’ [29]. A second, multi-tiered, example of this concerns tie bids in Dutch auctions. A typical response to this exception is to reset the price to a higher level and restart the descending price clock. A potential exception with this handler is infinite bid collisions, wherein two or more agents collide indefinitely at the same price and as a result the auction does not terminate. A typical response to the infinite bid collision exception is to terminate the bidding after a pre-determined number of collisions and then rely on the auctioneer to break the tie, for example by selecting the first bid, last bid, or a random bid. This tie breaking mechanism is, however, potentially subject to the ‘buyer/auctioneer collusion’ exception, wherein the auctioneer favors one bidder over others, e.g. by always rewarding tie bids to the favored bidder. This exception can be avoided by making bids public so bidders can verify for themselves who the correct winner is. Our exception handling knowledge base captures knowledge about handling such exceptions transparently, since exception handlers are themselves just processes that can be linked to their own exceptions and exception handlers, allowing recursion as needed. 3.4. The Barings Bank Case We will now illustrate our exception analysis methodology in the context of a wellknown real-life case: the Barings Bank failure.

In February 1994, 233-year old Barings Bank, one of the oldest and most respected investment houses in the United Kingdom, went bankrupt [30] [31]. The entire bank collapsed because of losses of $1.4 billion incurred in a matter of days by a single young trader, Nicholas Leeson. Nicholas Leeson was a futures trader in the Singapore branch of the bank. For a number of reasons, which are still not entirely clear, Leeson began to engage in unauthorized futures trading in the Singapore exchange. Due to inadequate internal controls and other process failures, Leeson was able to maintain his unauthorized and highly risky activity undetected by the bank headquarters in London until the very end. As we shall see, our exception analysis methodology can be used to systematically point out the gaps in the Barings trading process controls that allowed disaster to occur, as well as to suggest ways for closing those gaps. As noted above, the first step is to capture the ‘ideal’ business process. Figure 6 depicts a simplified but accurate model of the Barings futures trading process; boxes correspond to process activities, and lines correspond to dependencies (commitments) between activities: Customer Place Request

Trader

Buy Futures Contract

Receive Certificate

Send Payment

Bank Treasury Transfer Funds

Prerequisite Flow

Figure 6. The Barings Futures Trading Process When a customer requests a futures trade, the trader asks the bank headquarters for an advance of funds in order to cover the customer’s margin account2. Once the funds have arrived, the trader performs the trade, waits to receive the corresponding security certificate and finally pays the exchange. In an “ideal” world, a trader only performs trades when authorized to do so by customers, correct certificates are always received, and payment for trades exactly match the funds forwarded to the trader by the bank

headquarters. These conditions are implied by the “prerequisite” and “exact flow” commitments. The next step is to identify the possible exceptions that are associated with each key commitment in the “ideal” process model. By consulting the exception repository one can see that one possible exception for any prerequisite commitment is a prerequisite violation (“B without A”), that is, the possibility of activity B happening without a prior occurrence of activity A. In the context of the Barings trade process such violations would translate into unauthorized trading, unwanted security receipts and unnecessary payment (Figure 7). Customer Place Request

Trader

Unauthorized trade

Buy Futures Contract

Wrong certificate

Receive Certificate

Send Payment

Bank Treasury Transfer Funds

Prerequisite Funds misused

Flow

Figure 7. The Barings Futures Trading Process with Associated Exceptions Likewise, one possible exception for the mechanism managing an “exact flow” commitment is mismatch between the amount produced and the amount consumed. In the context of the Barings process this would translate into a misuse of headquarter funds. After possible exceptions have been identified, the next step is to find handlers suitable for mamaging the possible exceptions identified in step 1. It turns out that, because the trading process at Barings involves several independent entities (customer, bank, exchange) and requires some initiative from the part of the trader, there are were no practical mechanisms for avoiding the exceptions. There were, however, several mechanisms for detecting them.

Main Process

A

Update LogA

B

Exception Avoidance Process

Update LogB

Prerequisite Periodic Consistency Check

Exclusive access

Figure 8. Logging is a Generic Process for Detecting Prerequisite Violations For example, logging is one (out of several) generic mechanism for detecting prerequisite relationship violations (Figure 8). Logging involves recording all occurrences of activities A and B in some reliable storage medium and periodically conducting checks for prerequisite violations. In order for logging to be successful it is, in turn, required that (a) all occurrences of A and B are reliably logged and (b) the log can only be modified by the processes that do the logging. If we insert a logging process for all dependencies listed in Figure 8 we get a model of a properly instrumented trading process (Figure 9). Flow Prerequisite Exclusive access

Customer Place Request Update Commits

Trader Update Reqs

Bank Treasury

Update Funds

Buy Futures Contract

Update Recv

Receive Certificate

Update Paid

Send Margin Payment

Transfer Margin Funds Daily Consistency Check

Figure 9. Barings Process Properly Instrumented with Logging Processes

At this point, we can compare the process derived using our approach with the actual Barings process. It can immediately be seen that, although Barings did log some information about trades, it had two crucial gaps relative to the properly instrumented process of Figure 10 (see Figure 10): Customer Barings failed to compare funds transferred against funds used for client transactions

Place Request

Trader Update Reqs

Bank Treasury

Update Funds

Buy Futures Contract

Update Commits

Barings failed to safeguard against exclusive access Updatetrader violations because Recv was given log modification privileges

Receive Certificate

Update Paid

Send Margin Payment

Transfer Margin Funds Daily Consistency Check

Figure 10. Comparison between Ideal and Actual Barings Process First, it failed to log and compare the amount of funds forwarded by headquarters to the trader to the amounts actually paid by the trader for customer trades (in other words, the log labeled “Funds” in Figures 9-10 was missing from the Barings process). Second, Nick Leeson, in addition to being a trader, was also in charge of the back room operations in the Singapore branch. This gave him the authorization to modify the trades logs (and thus violated requirement (b) above of the logging process). Nick Leeson was able to use these two gaps to his advantage as follows: Whenever he received a trade request from a customer, he requested an amount of funds far greater than what was required for the customer trade. He then performed the customer trade, as well as some additional unauthorized trades on his behalf. All of these trades were automatically logged into logs “Commits”, “Received” and “Paid” (see Figures 9-10). Leeson then erased the records of his unauthorized trades from logs “Commits”, “Received” and “Paid”. Therefore, at the end of each day, the log of “Requests” matched perfectly the other three logs. By not checking for discrepancies between the funds forwarded to Leeson and the total funds recorded at the “Paid” log, headquarters remained unaware of Leeson’s activities until it was too late.

It is probably too simplistic to claim that the Barings disaster would have been avoided if the management of Barings had at their disposal our exception analysis methodology. Nevertheless, this exercise demonstrates that these methodologies and tools can be used in real-life cases to identify potential weaknesses and suggest ways alleviating them. 4. Conclusions This paper has described a generic knowledge-based methodology for the important problem of identifying possible failure modes and responses in multi-agent coordination mechanisms such as auctions. Our experience, based on the application of this methodology to problems in such domains as futures trading, multi-agent system task allocation, auction mechanisms, and back-office telephone operations, is that the methodology can be highly effective in helping designers define more robust coordination mechanisms. 4. References 1.

Gomes, L., Online Trading Halted at Broker After Site Attack, in Wall Street Journal. 2000. p. 6.

2.

Richtel, M., Yahoo Blames a Hacker Attack for a Lengthy Service Failure, in New York Times. 2000.

3.

Sandberg, J., Internet Vandals Pose Threat by Using New Mode of Attack Called "Smurfing", in The Wall Street Journal. 1998.

4.

Dellarocas, C. Immunizing online reputation reporting systems against unfair ratings and discriminatory behavior. in Proceedings of the 2nd ACM Conference on Electronic Commerce. 2000. Minneapolis, MN.

5.

Andersson, A., M. Tenhunen, and F. Ygge. Integer Programming for Combinatorial Auction Winner Determination. in International Conference on Multi-Agent Systems (ICMAS-2000). 2000. Boston, MA USA: IEEE Computer Society.

6.

Youssefmir, M. and B. Huberman. Resource contention in multi-agent systems. in First International Conference on Multi-Agent Systems (ICMAS-95). 1995. San Francisco, CA, USA: AAAI Press.

7.

Sterman, J.D., Learning in and about complex systems. 1994, Cambridge, Mass.: Alfred P. Sloan School of Management, Massachusetts Institute of Technology. 51.

8.

Kephart, J.O., J.E. Hanson, and J. Sairamesh, Price and niche wars in a freemarket economy of software agents. Artificial Life, 1998. 4(1): p. 1-23.

9.

Waldrop, M., Computers amplify Black Monday. Science, 1987. 238: p. 602-604.

10.

Singh, M.P., An Ontology for Commitments in Multiagent Systems: Toward a Unification of Normative Concepts. Artificial Intelligence and Law, 1999.

11.

Jennings, N.R., Coordination Techniques for Distributed Artificial Intelligence, in Foundations of Distributed Artificial Intelligence, G.M.P. O'Hare and N.R. Jennings, Editors. 1996, John Wiley & Sons. p. 187-210.

12.

Gasser, L., DAI Approaches to Coordination, in Distributed Artificial Intelligence: Theory and Praxis, N.M. Avouris and L. Gasser, Editors. 1992, Kluwer Academic Publishers. p. 31-51.

13.

Smith, R.G. and R. Davis, Distributed Problem Solving: The Contract Net Approach. Proceedings of the 2nd National Conference of the Canadian Society for Computational Studies of Intelligence, 1978.

14.

Baker, A. Complete manufacturing control using a contract net: a simulation study. in Proceedings of the International Conference on Computer Integrated Manufacturing. 1988. Troy New York USA: IEEE Computer Society Press.

15.

Boettcher, K., D. Perschbacher, and C. Wessel, Coordination of distributed agents in tactical situations. Ieee, 1987(87CH2450): p. 1421-6.

16.

Bouzid, M. and A.-I. Mouaddib, Cooperative uncertain temporal reasoning for distributed transportation scheduling. Proceedings International Conference on Multi Agent Systems, 1998.

17.

Smith, R.G. and R. Davis, Applications Of The Contract Net Framework: Distributed Sensing. Distributed Sensor Nets: Proceedings of a Workshop, 1978.

18.

Malone, T.W. and K.G. Crowston, The interdisciplinary study of Coordination. ACM Computing Surveys, 1994. 26(1): p. 87-119.

19.

Malone, T.W., et al., Tools for inventing organizations: Toward a handbook of organizational processes. Management Science, 1999. 45(3): p. 425-443.

20.

Klein, M. and C. Dellarocas, A Knowledge-Based Approach to Handling Exceptions in Workflow Systems. Journal of Computer-Supported Collaborative Work. Special Issue on Adaptive Workflow Systems., 2000. 9(3/4).

21.

Klein, M. Towards a Systematic Repository of Knowledge About Managing Collaborative Design Conflicts. in Proceedings of the International Conference on AI in Design (AID-2000). 2000. Boston MA: Kluwer Academic Publishers.

22.

Sandholm, T., eMediator: A Next Generation Electronic Commerce Server. 1999, Washington University at St. Louis: St Louis, MO, USA.

23.

Wurman, P.R., M.P. Wellman, and W.E. Walsh, The Michigan Internet AuctionBot: a configurable auction server for human and software agents. Proceedings of the Second International Conference on Autonomous Agents. ACM., 1998.

24.

Rodriguez-Aguilar, J.A., et al. Competitive Scenarios for Heterogeneous Trading Agents. in Second International Conference on Autonomous Agents (AGENTS'98). 1998.

25.

McAfee, R.P. and J. McMillan, Auctions and Bidding. J. Economics Lit., 1987. XXV: p. 699--738.

26.

Cassady, R., Auctions and Auctioneering. 1967: University of California Press.

27.

Priest, C. Commodity Trading Using An Agent-based Iterated Double-Auction. in Third International Conference on Autonomous Agents (AGENTS'99). 1999.

28.

Bernstein, A., M. Klein, and T.W. Malone. The Process Recombinator: A Tool for Generating New Business Process Ideas. in Proceedings of the International Conference on Information Systems (ICIS-99). 1999. Charlotte, North Carolina USA.

29.

Dellarocas, C. Mechanisms for coping with unfair ratings and discriminatory behavior in online reputation reporting systems. in International Conference on Information Systems (ICIS-00) (under review). 2000.

30.

Fay, S., The collapse of Barings. 1997, New York: W.W. Norton.

31.

Zhang, P.G., Barings bankruptcy and financial derivatives. 1995, Singapore: World Scientific Publishing Co.

Suggest Documents