Omega 35 (2007) 604 – 622 www.elsevier.com/locate/omega

Intelligence gathering for decision making夡,夡夡 Paul C. Nutt∗ Fisher College of Business, The Ohio State University, 644 Fisher Hall, 2100 Neil Ave., Columbus, OH 43210, USA Available online 7 February 2006

Abstract Empirical studies of decision making seldom consider the intelligence gathering activities required for decision making. In an attempt to fill this void, this study set out to identify and assess some of the key steps in gathering intelligence, considering the difficulty of the decision and available resources. The study found performance gapping and premising to be crucial activities and explored how each is carried out. A variety of premising and gapping tactics were uncovered, with some having better success than others. These tactics were found to influence the search approach selected to uncover alternatives and the success of the resulting decision. The best results were noted when search efforts are guided by needs documented with a quantitative performance gap; and when formal search or negotiation is used to identify alternatives. These findings hold for decisions that have high and low difficulty and for those with high and low resource support. The implications of these findings for decision makers and decision making are discussed. 䉷 2006 Elsevier Ltd. All rights reserved. Keywords: Organizational decision making; Intelligence gathering

1. Introduction According to Eisenhardt and Zbaracki [1], Harrison and Phillips [2], Mintzberg et al. [3], and Nutt [4] decision making is made up of activities that span intelligence gathering, direction setting, uncovering alternatives, selecting a course of action, and implementation. The paper addresses a neglected aspect in the study of decision making processes—intelligence gathering [5]. Decision making begins when stakeholders see a triggering trend (declining sales) or event (a strike) as significant, prompting steps to obtain intelligence. 夡 The paper received the Best Theoretical/Empirical Research Paper at the Decision Sciences National Meeting, November 18–21, 2000 in Orlando, Florida; a shorter version of the paper appeared the 2000 DSI proceedings. 夡夡 This manuscript was processed by Area Editor L. Sciford. ∗ Tel.: +1 614 292 4605; fax: +1 614 292 1272. E-mail address: [email protected].

0305-0483/$ - see front matter 䉷 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.omega.2005.12.001

This study identifies the steps taken to gather intelligence that clarifies the trend or event and explores the relationship between the steps followed to gather intelligence and the effectiveness and efficiency of the decision. A triggering trend or event draws attention. When seen as important, intelligence is gathered to elucidate the issues provoked [6,7]. The goal of this work is to improve our understanding of what goes on to gather intelligence. To do this, intelligence gathering steps are identified, tracing actions undertaken by decision makers that begin with signal recognition and end after a remedy has been uncovered. This study documents the steps taken and their sequence, identifying optional ways to gather intelligence. The approaches uncovered are then linked to success, documenting the value, timeliness, and use of the resulting decisions. Such a study requires qualitative and quantitative approaches: qualitative to uncover intelligence gathering

Paul C. Nutt / Omega 35 (2007) 604 – 622

steps and outcomes and quantitative to link these steps to success indicators in a decision outcome. Two research questions are addressed. First, how do decision makers go about gathering intelligence? Second, do the approaches used to gather intelligence influence decision success? To address these questions, nearly 400decisions made in organizations are examined to uncover the nature of intelligence gathering and its impact on success. Decision difficulty, resource support, and threats are measured and included as intervening factors in the success assessments. 2. The intelligence of intelligence Organizational leaders are bombarded by signals from users, customers, judicial renderings, new industry practices, regulators, and suppliers; calling attention to trends and events. When a signal denoting the trend or event is seen as significant by a manager with decision making authority, an organizational decision begins [8–10]. Most researchers agree that such signals or events create attention and motivate action [1–3]. What happens next is controversial. Some contend that objective information is sought [11,55,56]; others believe that an influence game is initiated [12,57]. In either case, intelligence is required to mobilize support and clarify what is at issue. This study probes intelligence gathering activities, concentrating on the steps employed by decision makers to provide “intelligence” in the “intelligencedesign-choice” conception of a decision making process offered by Simon [13]. Virtually every conception of decision making concedes that intelligence is collected (e.g., [14–17]), but the process of intelligence acquisition is rarely considered in empirical research. Past research efforts say little about how intelligence is gathered, or its effects (e.g., [5]). This is understandable and regrettable. It is understandable because the signalintelligence link is hard to document. Signals can take many forms. And, because signals arrive early-on in a decision making effort, decision makers and key participants find it hard to recall them and to trace their effects. These and related problems of specification and classification seem to have excluded intelligence gathering from empirical research efforts. This has left researchers little but vague impressions about what happens in these early decision making activities, making it hard to identify the tactics employed. This is regrettable because the intelligence gathered provides an important source of assumptions, motivations, and expectations that influence the search for answers in the decision making process.

605

When research ignores intelligence gathering, researchers are pulled away from the decision maker’s motivations, assumptions, and expectations. The motivation to act may be forgotten or used to form assumptions and expectations about what to do, and why, that become hidden. These motivations, assumptions, and expectations can have considerable influence on what is done to take corrective action. Because each is tacit, they remain implicit and out of a researcher’s reach. This has created a void, prompting researchers to speculate about how signals arise and their influence. Several streams of work have made conjectures about how assumptions, motivations, and expectations are made as managers confront a decision provoking issue. The influence of learning and memory has been empirically documented in the “information processing” research stream (e.g., [58,59]). This research offers problem-solving experiments, which have identified the representation and organization of stimuli (called signals here) as key activities. Such studies find that limited memory capacity makes encoding selective, dependent on signals with features that have salience to an observer (e.g., [18]). Vivid claims in a signal that agree with and confirm prior knowledge are more apt to be recognized ([60]). Pallid ones tend to be ignored. Decision makers seem to locate the salient signals along a continuum that runs from gain to loss. A possible loss is found to produce a stronger motivation to act than does a possible gain [19–21]. This stream of work is both interesting and informative, but says more about the biases in signal coding than how signal coding is, or should be, carried out. Researchers with a prescriptive posture suggest that signals should be decoded as a performance gap (e.g., [6,7,4,22,23]). This literature contends that a signal will be seen as significant when a performance indicator, such as market share or revenue, falls below a preset norm. The signal would be ignored if performance equals or exceeds the norm [24,7,4]. When a performance gap, a shortfall, is identified, a concern is “sensed” [25,26]. The performance shortfall uncovered identifies the magnitude of the concern to be overcome, such as the amount of market share or revenue decline. Decision making is undertaken to find ways to deal with the concern by closing the performance gap. Social motivation researchers make very different assertions. Here the decision maker’s actions are prompted by dissonance [27], equity [28], or related consistency theories. Such theories contend that decision makers’ beliefs, motivations, and drives entice them to look for information that attends to these beliefs, motivations,

606

Paul C. Nutt / Omega 35 (2007) 604 – 622

and drives. This prompts decision makers to seek information that bolsters their position by maintaining images of self, attitudes, and the like. Decision makers create the impression that they have arrived at an unbiased inference by making their interpretation of trends and events seem equitable to all concerned. Such decision makers may seek information about a diversion to avoid decoding signals that could invalidate a preferred interpretation. Some kinds of performance gaps would be recognized and others ignored. Social motivation explanations suggest that a performance gap need not be expressed in measurable terms, such as the decline in market share or loss in sales. Instead, impressionistic information would be marshaled. This would allow the decision maker to maneuver for position, taking steps to insure that a personal interest or an interest upon which the decision maker is dependent can be served. Interpretations take shape as subjective statements that call for dealing with dissatisfactions, conflicts, or values that have vague and ambiguous expectations [4]. Search would be undertaken to overcome a concern described in impressionistic terms, such finding ways to reduce conflict or to increase value. Other research efforts have attempted to capture how decisions are initiated by exploring case studies that document how a decisions was made (e.g., [14,15,8,29,30]). Studies of this type describe what was done to make a decision, indicating what appears to be a triggering trend or event. For example, Mintzberg et al. [3] explored 24 cases and discovered an “identification” phase, in which signal recognition and diagnosis initiates decision making. During recognition, decision makers examine factual signals to measure differences between a current situation and some standard, again looking for a performance gap. Past and projected trends, standards drawn from comparable settings, top management’s expectations, and the like, provided norms. If a gap seems to be frequent, clear, and consistent it is assumed to be real and action to close it is called for. This implies that signals are documented with one or more quantitatively determined performance gaps. When the gap exceeds a threshold, directions are set to realize an opportunity or to solve a problem. To respond, search efforts take shape formally—creating a task force or hiring a consultant—and implicitly, with no observable steps. In some instances, ready-made solutions, either bought or copied, identified the direction with an opportunity (a ready-made solution). In others, a problem, which specifies a performance gap to be closed, is used to provide direction for subsequent decision making activity.

Quinn [31] conducted studies of decision making in 10 major corporations, discovering activities and processes that occurred as decision making begins. Managers were observed drawing on networks of people to get information suggesting whether a change is needed. In Quinn’s studies, decision makers tended to ignore information provided by traditional means, such as an MIS or formal reports. Instead ‘screens’ were used. The screen uses subjective information depicting proliferation, exposure, overlap, lack of focus, low motivation, inconsistencies, and anomalies to compare a current position with a perception of future needs. This differs from the steps discovered by Mintzberg et al. [3], by calling for informal information sources and subjective measures to form performance gaps. Decision makers use the gap information to signify what is wanted as an outcome. In some instances, overly stringent norms are applied to make performance shortfalls seem worthy of attention. Search is not directed by an opportunity in these studies. Instead, search is guided by the ends sought, which is made clear by stating a goal. Other research finds decision makers to be buffeted by streams of loosely coupled problems, solutions, stakeholders, and choice situations that flow through an organization [32–34]. These streams meet and couple due to accidents of timing, not any causal logic. Solutions seek problems, problems and solutions are looking for choice situations, and people with a stake in the outcome are looking for action. Decision makers respond by making choices according to their work load, how decisions “bunch up”, and the power of idea champions; and do not collect intelligence to clarify matters. The choice situation becomes a “garbage can” in which problems and solutions are dumped. Performance gaps are recognized (or problems rationalized) after a solution is offered. After the fact rationalizations, such as carefully crafted problem descriptions, are used to defend a solution believed to provide an opportunity. In this conceptualization, decision making is initiated by the opportunity to act, accompanied by justifications expressed as performance shortfalls that the opportunity could correct. The literature seems to agree that intelligence is gathered to identify some kind of a performance gap, but not when this is done during the decision making effort. There is also agreement that the perceived size of a gap provides the motivation to close it. Decision making can take shape as a formal process directed a goal made up of norms to be met. In other instances, decision making can be displaced by a solution to be tested, with a performance gap formed after the fact. This suggests that intelligence gathering during decision

Paul C. Nutt / Omega 35 (2007) 604 – 622

making is a two-step process, and that the order of the steps is contentious. In one step, signals are decoded with a performance gap that can be set quantitatively, qualitatively, or impressionistically. Arguments for using each type of gap have been made, suggesting the following research question: Q1: Is decision success influenced by whether quantitative, qualitative, or impressionistic performance gaps are extracted from a signal? In the other step, an interpretation is made that offers premises that frame what is to be done. The literature suggests that this can be done in four ways. First, a premise can frame the effort as either meeting a need or realizing an opportunity. Opportunities reference a solution that call for a particular course of action. An opportunity translates a seemingly desirable way to respond, found in a signal, into questions about its merit. The guidance offered emerges as a mandate to adopt the solution—an action statement. Needs offer a more general view of guidance in which a concern or difficulty is noted. Guidance takes shape by specifying the nature of the need, such as turning around a decline in market share or increasing utilization. A need statement identifies the kind of performance improvement sought, such as lower cost or better utilization. Both action and need statements can be moderated by threats. Threats emerge when the signal also conveys both high importance and considerable urgency [20,35,61]. Combining this with a need or opportunity statement suggests two additional categories: a defined threat, an action statement accompanied by considerable urgency and importance, and an undefined threat, a need statement with high importance and urgency. Thus, premises can take shape in four ways: as a need, an opportunity, a defined threat, or an undefined threat, suggesting: Q2: Is decision success influenced by whether the premise used to direct action takes shape as a need, opportunity, defined threat, or undefined threat? The way in which a performance gap is set and how an interpretation is made may be compatible, such as a need and a quantitative gap, or incompatible, such as a need and an impressionistic performance gap. The signal can be recognized and premised in ways that do and do not facilitate search in a decision making effort, suggesting: Q3: Is decision making success influenced by the joint effect (interaction) of the performance gap and premise types?

607

Premises (need, opportunities, defined and undefined threats) may influence how the search effort in decision making is carried out. Studies show that a search can be carried out with a rational process, a negotiation, an opportunity validation, emergent solution, problem solving, or redevelopment (after a failed opportunity), as shown in Table 1 [36]. The influence of the premise types on the type of search applied suggests another question: Q4: Is decision’s success influenced by joint effect of type of premise used to guide the effort and the type of search (rational, negotiated, opportunity, etc.) employed? Decisions that appear daunting to a decision maker may influence the type of intelligence that is gathered. Perceived difficulty is made up of two components. A decision would seem difficult when decision makers are unsure of their goals making criteria to weigh the merits of an alternative argumentative [37,30]. Difficulty also arises when decisions have political ramifications stemming from conflicts among key players, turf issues, historical animosities, opposition, and the like [38,11]. An indicator of difficulty can be constructed using these factors. Decisions with neither goal ambiguity nor political issues are thought to have “low” difficulty. Such decisions have clear and mutually agreeable aims with seemingly manageable levels of conflict, turf issues, and the like. Decisions with “high” difficulty can be politically difficult, procedurally difficult, or both. The procedural difficulty stems from goal ambiguity and political difficulty from stakeholder disagreements. When goals are absent, criteria that allow one to weight the merits of alternative would be argumentative or missing, making evaluations difficult and subject to challenge, creating procedural difficulty. Political difficulty puts a premium on managing conflict. Either makes a choice difficult. Such decisions may prompt decision makers to take shortcuts and/or discourage them from collecting data. This may explain why decision makers adopt impressionistic intelligence and avoid more diagnostic approaches to intelligence gathering. Q5: Does decision success depend on the joint effect of decision difficulty with the type of gap, premise, and search approach? It is also argued that resource support influences the results realized from a decision [10,9]. Decision with generous support may have a better chance of providing good results, even when best practices are not followed. More people can be involved and thus more ideas are apt

608

Paul C. Nutt / Omega 35 (2007) 604 – 622

Table 1 Explanatory variables Variables

Data collection

Approaches used

Signal coding

1. Informants answered two open-ended questions: “What first captured your attention” and “Why was this important?” 2. Specifics about performance levels and expectations (e.g., norms or performance benchmarks) were inferred from what was said

Signal coding used “performance gaps” that were: 1. Quantitative—both norms and performance were determined factually. 2. Qualitative—both norms and performance were noted, but either the norm or the performance was not factually determined 3. Impressionistic—no norms or performance indicators were cited. Signals were described as an arena of action

Signal interpretation

1. Decision makers described the motivation to act. Determine whether this was performance or action driven 2. Questionnaire data rating the decision’s importance and urgency, on a 1–5 scale (1 = low, 5 = very high) by the two secondary informants

Interpretations: 1. Need—performance driven, calling for better results 2. Opportunity—action driven, calling for a particular action 3. Defined threat—opportunity with urgency and importance both rated very high 4. Undefined threat—need with both urgency and importance rated very high

Search behavior evoked

1. Decision makers were asked to specify the steps undertaken to uncover alternatives that were considered before a course of action was selected 2. Classified according to procedures reported in [54]

Search approaches uncovered 1. Negotiated—stakeholders meet to uncover options 2. Rational—outcome target set and formal protocol followed to find alternatives that can produce expected results 3. Problem solving—a variation of the rational approach in which the target is stated as a problem to be overcome 4. Opportunity—an idea noted in the signal prompting action was adopted 5. Emergent opportunity—the adopted idea emerged before a search could be completed 6. Redevelopment—the idea found in the signal was abandoned and a search undertaken to find a replacement

to be generated [3]. Many studies show that increasing the number of ideas in the mix, as a decision is made, improves the prospect of success [39,40,36]. Q6: Does decision success depend on the joint effect of resources support with the type of gap, premise, and search approach?

3. Methods A database made up of 376 strategic decisions is explored to answer the research questions. This database has several key features. Twenty-two percent of the

decisions are drawn from public-sector organizations, 33% come from private-sector organizations, and 44% from third sector (non-profit) organizations, with a single decision taken from each organization. These organizations are medium to large in size and none are new startups. The decisions involve all of the types of decisions found in the literature (e.g., [8]): technology (18%), controls (14%), products or services (30%), personnel policy (5%), support services (18%), reorganizations (9%), and markets (4%). Half of the decisions failed, suggesting that the database contains both good and less desirable practices [41]. Data were collected using interviews and questionnaires to capture how the decisions were made.

Paul C. Nutt / Omega 35 (2007) 604 – 622

Interviews were carried out to identify decision making practices. There were three informants for each decision. The primary informant (the strategic decision maker) was well placed in the participating organizations—nearly two-thirds were top executives (CEOs, COOs, or CFOs). Two secondary informants provided values for some of the variables by filling out questionnaires. The secondary informants were the line managers, subordinate to the primary informant, in 57% of the cases, a staff person in 35% of the cases, and a task force member in 8% of the cases. 3.1. Soliciting participation People holding key positions in organizations were identified as potential participants by drawing on my prior employers and associates, colleagues, student projects, and executive education. Each was asked to participate in a study seeking to improve our understanding of strategic decision making. The study was presented as a long-term effort to accumulate a sufficient number of decisions to uncover and appreciate the practices used in decision making. A decision was defined as an episode, beginning when the organization first became aware of a motivating concern and ending with an implementation attempt. To ensure interest and first hand knowledge, the contact person was asked to select a strategic decision that had been made in the past year for study. A strategic decision was defined as having considerable importance due to the resources required and its expected consequences [3,8]. The contact person was then asked to identify three people involved in the decision, who could be interviewed, including the person who had primary responsibility for the decision. In most instances, the contact person suggested a strategic decision for which he or she was responsible and became the primary informant. After data had been collected, the contact person made calls to solicit others to participate. Cases were accumulated in this way over a period of several years. 3.1.1. Informant roles The primary informant, the decision maker, provided information about the steps that were followed to make the decision. The secondary informants filled out surveys. One of the secondary informants provided a second listing of decision making steps, as a validation. To separate thinking about outcomes from the recall of how the decision was made, questionnaire data were collected from the two secondary informants before the interviews. The secondary informants rated each decision’s value, duration, urgency, importance, political

609

difficulty, and goal clarity on anchored rating scales, shown in Table 1. 3.1.2. Reconstructing events Retrospective data collected to recall events, such as decision making steps, can be biased by inaccurate recall. Self-justification, memory lapses, and logical inconsistencies can distort what is reported. Miller et al. [42] provides a way around these difficulties. They call for multiple informants and data sources that focus on factual events in interviews and seek convergence in interpretations. “Second chance” reviews of the data collected are used to jog the informant’s memory [43]. Papadakis and Barwise [44] add some procedural criteria. They call for recent decisions to reduce memory failure, informants with first hand knowledge, using archival records and documents, and cross checking the sources used (interviews and documents). Both emphasize validity by attempting to verify events that have occurred. To do this, two informants were interviewed separately to uncover the steps taken to make each decision. An interview procedure was devised to deal with the dual problems of what people can remember and choose to tell in an interview. Drawing on the qualitative research principles of Denzin [45], each informant was asked to recall what first captured his/her attention. Questioning proceeded from this point by asking, “What happened next”. For example, after an informant described what captured his/her attention, he/she was asked why this seemed important and merited attention. Questioning continued in this way, taking cues from the last response to fashion the next one. The information obtained from the secondary informant was used to corroborate what the primary informant said [46]. The study focuses on the early precipitating events in which managers investigate signals. The data in the decision archive allows such an appraisal because informants were encouraged to ruminate about the signal that prompted action at the beginning of the interview. After the informants indicated what had captured their attention, they indicated how people in the organization confirmed that the trend or event merited attention. Two open-ended questions were asked at this point in the interview: “What captured your attention” and “why was this important?” The responses were used to fashion a follow-up question about the trend or event. For example, if a response posed questions about morale the follow up question asked why morale merited attention. If the response cited data, such as customer complaints had increased by 20%, the next query incorporated the data—“How do we know that the 20% increase in complaints were preventable?”

610

Paul C. Nutt / Omega 35 (2007) 604 – 622

3.1.3. Triangulating informant responses The informants’ recall of decision making steps was compared to improve the validity of each decision’s description [47]. The author prepared two documents—a narrative of about 20 pages and a summary of the steps recalled by each informant. The informants individually reviewed their narrative and the listing of steps and made changes they believed were warranted. Documents such as notes, proposals, or files that still existed were also collected. Documents and the listing of steps offered by each informant were compared to find inconsistencies and gaps. The inconsistencies and gaps were investigated in a follow-up interview with the primary informant (the decision maker). In this interview, attempts were made to reconcile differences and fill in gaps [48]. Thus, method and two types of informant triangulation were used to validate each decision description. A clear picture of the decision, agreeable to the informants, was required to include a decision in the database. As data were collected, approximately 20 decisions failed to meet either the clarity or the agreement tests and were discarded. If both tests were met, several kinds of documentation, including the one-page listing of all decision making steps described above, were placed in the data base. All of the 376 decisions were profiled in this way. 3.2. Identifying the explanatory variables The explanatory variables were taken from the questionnaires and from the write up created for each decision. These narratives provided the information to uncover actions taken to create “intelligence” about a decision. Data from the questionnaires were used to refine some of these actions, putting them in more precise categories. Three separate classifications were carried out to specify values for the variables found in the research questions. According to the literature, signals are decoded with a “performance gap”. In a performance gap, performance and norms can be measured with different degrees of precision. The narratives were reviewed to identify the nature of the signal that captured the decision maker’s attention, determining the measurement precision of each. Each decision was examined to determine how concerns about cost, utilization, margin, etc. were measured, such as with a numeric estimate or with a qualitative assessment, such as being “too large”. Applying the definitions found in Table 1 to this information, the gap was identified quantitative, qualitative, or impressionistic. One of these three gap categories was identified in 335 of the 376 decisions.

Eighty-nine percent of the decisions had one of the hypothesized gaps (Table 2). The remaining cases had no information or conflicting information about signal decoding. To identify premises, the next few steps in each decision profile were examined to determine how the motivation to act was characterized. The literature contends that this can occur in one of two ways: decisions can be performance or opportunity driven. Performance indicates a desired outcome such as a lower cost, higher utilization, and more market share. Opportunity cuts to a solution or answer that seems desirable, such as buying a new system being used by a competitor. Performance driven premises were called “needs”, and idea driven ones “opportunities”. This interpretation is qualified by “threats”. The informants identified a threat in a decision when urgency and importance were both rated as very high. A “defined threat” is an opportunity that is very urgent and very important. An “undefined threat” is a need that is very urgent and very important (Table 1). For 337 of the 376, or 89% of the decisions, a need, opportunity, defined threat, or undefined threat was identified. The 39 unclassified cases had multiple interpretations making it impossible to say which was used as a premise. A search approach was identified by working backward from the course of action used in a decision to determine how the solution/remedy was discovered [36]. Several search approaches were uncovered. A “rational approach” used a formal protocol (benchmarking, search, innovation) with a goal specifying expected results as the target. A variation has formal protocols but the protocol was directed by a problem to be overcome. A goal seeks to improve utilization; a problem seeks to illuminate the source of poor utilization. Search was called “opportunistic” when an idea contained in the signal prompted the decision making effort. An “emergent opportunity” was noted when an idea terminated an ongoing search. This occurred when an idea surfaced outside a formal search effort that was ultimately implemented. Another kind of abort was noted when the idea in an opportunity was abandoned and steps were taken to find a better solution. This was called “redevelopment”. Such a search was found to be undirected with no target, beyond finding a replacement idea. Negotiation was used as the search approach when the decision maker put stakeholders in a team or group and asked them to identify alternatives (see Table 1). These six search approaches were noted for 338 of the 376 or 90% of the cases (Table 2). Unclassified cases engaged in multiple search behaviors or lacked the information needed to classify them.

Paul C. Nutt / Omega 35 (2007) 604 – 622

611

Table 2 Intelligence gathering and its effects on success Main effects

N

Freq

Signal recognition

(Performance gap) Measured Quantitatively 140 Qualitatively 135 Impressionistically 60 Sig Premised as a Need 165 Opportunity 118 Undefined threat 22 Defined threat 32 Sig Search behavior provoked Negotiation 50 Rational 93 Problem solving 56 Redevelopment 50 Opportunity 64 Emergent opportunity 25 Sig Totals

338

Value indicators

Adoption indicators

No. of Alts.

Merit

Na

Ratingb

DMRT

Duration DMRT

Time Moc

Initial use DMRT

Full use

Rated

DMRT

Ratee

DMRTf

71% 64% 40% p  .05

A B C

59% 53% 28% p  .008

A A B

42% 40% 18%

2.2 1.8 1.7 ns

3.8 A 3.8 A 3.1 B p  .001

8.9 9.5 9.4 ns

50% 35% 6% 9%

2.0 B 1.7 B 3.2 A 1.6 B p  .05

3.6 3.7 3.5 3.4 ns

9.5 10.4 6.7 6.2 p  .05

B B A A

61% 64% 68% 47% p  .002

A A A B

53% 50% 68% 28% p  .005

B B A C

15% 27% 16% 15% 19% 7%

2.1 2.5 2.3 1.6 1.3 1.0 p  .05

4.0 3.9 3.3 3.3 3.5 3.2 p  .01

7.4 8.5 13.1 9.7 9.1 6.0 p  .05

A/B B D C B A

78% 70% 51% 58% 56% 56% p  .009

A A B/C B B B

74% 57% 41% 48% 37% 40% p  .0002

A B C B C C

100%

2.07

A A A A/B B B

3.6

A A A/B A/B A/B B

9.1

63%

50%

a Number

of alternatives uncovered. 5 = outstanding 4 = good 3 = adequate 2 = disappointing 1 = poor c Time measured in months. d Percent of decisions used from the outset. e Percent of decision in full use after a 2-year period. f Duncan multiple range test (DMRT): letter codes indicate significant differences in the means, p  .05; from most (designated A), to least. b Scale:

To improve intra-rater (intra-person) reliability, the cases were re-classified by the author for each variable until there was agreement. To do this, the cases were grouped into one of the categories identified for each of the three factors (gap, premise, and search approach), other categories, and an unknown category. This was repeated to see if the classifications held up. After several sorts, the previous classifications were being reproduced in the current ones, suggesting that intra-rater reliability had been achieved. Inter-rater (inter-person) reliability was determined by asking a colleague to review the decision summaries and indicate how the performance gap was defined, whether there was a need or opportunity as a premise, and the search behavior evoked. The definitions offered in Table 1 were provided, asking the second rater to match cases with these categories, or to an unclassified category. Reliability was computed as

percentage agreement. This led to a 90% (or more) rating agreement for each of the variables. 3.3. Success measures To identify decision making success one must assume that different decisions (choosing one location or product over another) produce different patterns of success and that these decision outcomes can be identified with sufficient clarity to allow success measurements to be made. These assumptions seem reasonable because Bell et al. [10], Dean and Shafman [17], Boal and Bryson [16], and Nutt [43] were able to trace the postulated linkage, showing that different approaches produce different decisions that have different outcomes and consequences. Studies that concentrate on tracing the impacts of a decision are preferred because attempts to link

612

Paul C. Nutt / Omega 35 (2007) 604 – 622

decisions to overall organizational performance, such as ROI, are not successful [44,40]. Decision results of interest are the scope of a search (number of alternatives), their value, how long it takes, and disposition or use. To measure “number of alternatives”, the decisions were reviewed to count the number of options that were uncovered. Using such a measure can be justified by the creativity literature, which finds results improve as more alternatives are uncovered [62]. Decision making research has drawn a similar conclusion [44]. Merit was estimated from data taken from the questionnaires filled out by the two secondary informants. The informants checked along an anchored rating scale with five anchors. The scale anchors defined a rating of 5 as outstanding, which was assigned to decision that made a decisive contribution by providing exceptional perceived quality. A rating of 1, termed poor, was assigned when a decision had no impact. The remaining scale points for the decision value measure were termed good for ratings of 4, adequate for ratings of 3, and disappointing for ratings of 2. To improve recall and precision, the estimate-reflect-estimate (ERE) procedure was used [49]. First, the informants made an initial rating. The informants then reviewed an average of the two ratings and compared the result to their initial rating. Discussion was encouraged in which the raters were asked to argue for a higher or a lower rating. A second rating was then made. Decision merit was determined by an average of the informants’ final ratings. This approach was followed for three reasons. First, organizations are reluctant to provide objective data about benefits and the like. Alexander [50] found that a manager’s subjective determinations of decision are highly correlated with objective measures, making subjective measures of value a good estimate of the actual value. Second, using the secondary informants to make this assessment increases objectivity by avoiding a self-serving assessment by the decision maker. Third, using the ERE approach has been shown to move a subjective estimate toward a true value. Duration has two time periods. The first measures the time for plan development, from the triggering trend or event to the completion of a plan. The second measures the elapsed time from the end of plan’s development to the end of implementation attempts by the organization. This first measure was used in this study because this time period denotes the duration of a search effort and, thus, could be influenced by it. Informant’s recall was refined by a discussion of the initial time estimates using the ERE procedure, as noted above. The average development time in months from the final estimates

was used in the analysis. Development time seems important because decision makers are often under pressure to produce results. Duration is a useful measure because more timely ways to complete a decision would be valued [44]. Decisions were followed for 2 years to capture changes in use. Several kinds of changes were observed. First, some decisions had a limited scale of use, suggesting a partial adoption. Other decisions experienced substantial delays before adoption. Finally, some decisions were withdrawn, becoming ultimate rejections, and some of the initially rejected decisions were put to use. Using the information, two measures were created called initial and full adoption. “Initial adoption” accounts for whether there was an adoption and a rejection when a plan was first offered. The “full adoption” measure accounts for changes in adoption and treats partial adoptions as failures, making it a downstream indicator of the degree of use. Using such measures seems reasonable because success for an organization stems from putting a decision to use [51,52]. If a decision is not used, it has no value. 3.4. Decision difficulty and resources Questionnaires, filled out by the secondary informants, provided information used to classify decisions according to their difficulty and available resources. The questionnaire asked each secondary informant to characterize the decision’s political difficulty (defined as conflicts and the like among key stakeholders), its goal clarity, and its supporting resources. For each factor, the informants were asked to check along an anchored rating scale that ran from 1 = least to 5 = most, following common practice in the scaling of such factors (e.g., [8,9]). A factor rating of 1 or 2 was characterized as low and a rating of 4 or 5 high. The perceived difficulty of a decision was termed low if both factors were rated 2 or lower. “High” difficulty was assigned to decision when either political or procedural difficulty was rated at 4 or higher. For resources, “high” was assigned to a decision when available resources were rated at 4 or higher and “low” if rated 2 or lower. To provide a sharp distinction between low and high for complexity and resources, decisions with an intermediate rating of 3 were dropped in this analysis. 3.5. Analysis ANOVA is recommended when the explanatory relationship has main effects and interaction effects made up of categorical variables [53]. Five ANOVAs were

Paul C. Nutt / Omega 35 (2007) 604 – 622

carried out, one of each success measure (number of alternatives, value, time, initial adoption, and full adoption) as the dependent variable. In these analyses, the explanatory factors were made up of the types of performance gaps recognized, premise types, and search approaches, their interactions, and the interactions with complexity and resources. The theory indicates gap, premise, and search as the sequence typically called for in these steps, suggesting that an interaction between the performance gap and premise factors and the premise and search factors but not between the performance gap and search factors be investigated. The interaction of difficulty and resources with gap, premise, and search was also explored. This analysis determines whether the findings generalize to high and low difficulty decisions and to decision with high and low resources. A Duncan Multiple Range Test isolated significant differences in the success of each of the actions/tactics found in the three explanatory factors and the combinations of the actions/tactics found in the interactions. Tables 2–7 offer the results of these analyses.

4. Discussion of results Analysis found that the tactics in the explanatory variables appear to capture what is done to gather intelligence in 89% of the cases studied. (The remaining cases were found to have at least one of the explanatory factors that resisted classification with the factors under study.) To answer the research questions, the classified tactics and actions will be discussed and then evaluated. 4.1. Signal recognition Intelligence was extracted from signals to decode the nature and perceived magnitude of concerns. A quantitative performance gap was used to document concerns in 42% of the cases (Table 2). A quantitative gap attempted to measure a performance shortfall in factual terms, such as a “20% cost increase that must be curtailed”. This required a decision maker to gather intelligence to document both norms and current performance. Norms indicate performance expectations, gathered from what other organizations are able to do, and from other sources. The quantitative gap often extracted the amount of performance improvement deemed useful from the signal, such as claims that specify costs in a benchmarked competitor. A qualitative gap identified the type of concern, such as cost or market share, but called them “too high” or “too low”. No numeric indica-

613

tors of either performance or norms were collected. This was observed in 40% of the cases. An impressionistic gap failed to indicate a specific performance shortfall and no factual data was collected. In these cases, the decision maker identified an arena of action; such as “we must deal with our image” or “we need to improve our morale”, without asking for intelligence to indicate the kind of performance expected or ways to measure image or morale. In these cases, the decision maker made an assertion indicating what needed attention without offering any factual justification. Social motivation theories may offer a way to understand why this behavior arises. Decision makers using an impressionistic performance gap many seek to gain control over a situation by allowing them the latitude to act in any way they see fit. This was observed in 18% of the cases. The influence of the performance gap types on success is shown in Table 2. Quantitative gaps produced the best decision results and impressionistic the worst, with qualitative in between. Decisions with a quantitative gap were rated as good with a 71% initial use and a 59% full use. Those with a qualitative gap had an initial use rate of 64%, with comparable quality and a full use rate of 53%. Decisions made with an impressionistic gap were rated as adequate and had a 48% initial use and a 28% full use. The type of performance gap did not affect the number of alternatives uncovered or development time. Quantitative gaps may produce superior results because ambiguities about the motivation for action are swept away. Qualitative gaps are less persuasive and thereby may draw in people who oppose a surmised arena of action, or indeed any action. The merits of taking action are subject to interpretation and debate, and thus can be challenged by those who see themselves as potential losers. Impressionistic gaps imply what should be done. This can raise questions about the motives of a decision maker, even when the decision maker has no personal stakes in the decision, and draw in even more stakeholders who see themselves as losers should a course of action under consideration be adopted. 4.2. Signal interpretation A premise frames the decision making effort. Premise was classified as one of the four types in 90% of the cases. When a premise was given as an idea, as opposed to a need, success was unaffected. However, success declined dramatically for defined threats. When an opportunity is adopted to respond to a threat, produced by a very urgent and important decisions,

614

Paul C. Nutt / Omega 35 (2007) 604 – 622

duration was reduced to 6.2 months, but initial use fell to 47% and complete use to 28% for defined threats. For an undefined threat, coupling very high urgency and importance with a need, the number of alternatives doubled to 3.2, initial and full use rose to 68%, in a comparable time period. As one would expect, threats cut the time to act significantly. However, success dropped precipitously when ideas about how to proceed were used by the decision maker to mobilize action. These findings denote the importance of appearing to be impartial during a threat-driven decision making effort. Threats intensify concerns that limit search when an idea emerges that seems to head off the threat without further delay. The idea makes any additional search seem unwarranted. However, the price to be paid for rapid action is found in a sharply reduced rate of adoption. Attempting to meet a need under conditions of threat with a search increased adoption rates by 20–50%. 4.3. Search behavior evoked Search tactics were identified in 90% of the cases. Some search tactics were successful, others less so. The most successful searches used a negotiation or a rational search approach. For these search approaches, there was an average of 2.1–2.5 alternatives considered producing decisions rated as good with 70% or more initial use and 57% to 74% full use. The other search tactics produced fewer alternatives (1.6–1.0) and decisions that were rated as just a bit above adequate, with 51–58% initial use and 37–48% full use. Problem-directed searches are less successful than goal-directed searches. More alternatives were considered (2.3) but this took much longer (13.1 months) and had no effect on value, which led to low rates of adoption (51% initially and 41% full use). Opportunitydriven decision making was also unsuccessful. When an emergent opportunity was used, value and full use fell to some of the lowest levels noted in the study. An emergent opportunity led to rapid decision making, getting results in just 6.0 months. But there was a price to be paid for being efficient. Opportunities produced little in the way of timely decisions, better value, or more use. A redevelopment search, one in which the opportunity was discarded, failed duplicate the results of a rational (goal-directed) search. Fewer alternatives (1.6) were found that produced decisions valued at between adequate and good, and comparatively little use (48–58%). The longer duration seems linked to the absence of a performance-based goal or target, required by a rational search.

4.4. Intelligence-premise links Table 3 shows how signals recognized as each of the performance gap types interact with the types of premises and how these premises prompted various types of search behavior. The frequency of these links indicate performance gaps that make a premise more or less likely, and premises that seem to prompt various types of search. A chi-square test indicated that the relationships are significant (p < .05). The association of performance gap and premise indicates how one influences the likelihood of the other arising during decision making. Quantitative gaps had a strong affinity for needs. Needs were observed twice as often to establish the premise for acting, compared to an opportunity, whether a threat emerges or not (Table 3). This relationship is missing for the qualitative gap. This type of gap was used about as often for both need and opportunity premises, whether threats were present or not. Impressionistic gaps are more apt to prompt an opportunity premise when a threat is present. Thus, quantitative gaps were associated with a need premise and impressionistic ones when a defined threat arose as a premise. Premise can be linked to search behavior in the same way. Needs were more apt to prompt a rational search and less apt to lead to adopting an opportunity. Premises as an opportunity took shape when the idea in an opportunity seemed compelling and enticed people to step away from starting a formal search. The same pattern emerged for needs and opportunities premised as a threat. An undefined threat prompted more rational searches; defined threats lead to using the idea that prompted action. Defined threats seem to make negotiation infeasible. To explore the impacts of these associations, the interaction of performance gap and premise, and premise and search behavior are considered. These findings are shown in Tables 4 and 5. A quantitative performance gap was likely to bring about a need premise, which lead to somewhat more successful decisions from those with an opportunity premise (first two rows, Table 4). Value outcomes were comparable, but use declined. Differences between need and opportunity premises were much more apparent under a threat. Undefined threats (need based) produced nearly twice the alternatives, better value, and huge increases in use. The price to be paid arose in timeliness. Without an idea to cling to under a threat, duration more than doubled. A qualitative performance gap (nonnumeric measures of performance or norms) had a similar, but a less pronounced effect. With an undefined

Paul C. Nutt / Omega 35 (2007) 604 – 622

615

Table 3 Linking intelligence with premises and search The intelligence

Number of cases

Premises as a Need Opportunity Unidentified threat (UT) Defined threat (DT)

Frequency

QT

QL

IMP.

QT

QL

IMP.

82 40 12 6

57 53 9 16

25 24 1 10

58% 29% 8% 4%

42% 39% 8% 12%

42% 40% 2% 16%

The premise*

Need

Opportunity

UT

DT

Need

Opportunity

UT

DT

Provoking searches Negotiation Rational Problem solving Redevelopment Opportunity Emergent opportunity

30 54 37 20 13 11

16 24 13 21 37 7

4 8 4 2 2 2

0 7 2 6 12 5

18% 33% 22% 12% 8% 7%

13% 20% 11% 18% 31% 6%

18% 35% 18% 9% 9% 9%

0 22% 6% 19% 38% 16%

QT, quantitatively; QL, qualitatively; IMP, impressionistically. ∗ Percent of column total.

Table 4 Intelligence and premise interactions N

Joint effects of Intelligence

Quantitative

Qualitative

Impressionistic

Totals a Number

Freq

Premise

Need Opportunity UT DT Need Opportunity UT DT Need Opportunity UT DT

82 40 12 6 57 53 9 16 25 24 1 10 335

24% 12% 4% 2% 17% 16% 3% 5% 7% 7% < 1% 3% 100%

Value indicators

Adoption indicators

No. of Alts.

Merit

Na

DMRT

Ratingb

1.1 1.7 4.5 2.6 2.1 1.9 1.3 1.4 2.1 1.5

B/C B/C A B A A B B

3.7 3.9 3.7 3.1 3.7 3.8 3.7 3.9 3.1 3.3

Duration DMRT

Time 9.0 9.9 8.0 2.6 8.7 12.2 5.4 6.2 11.0 8.7

*

*

*

1.6

3.0

8.3

Moc

Initial use

Full use

DMRT

Rated

DMRT

Ratee

DMRTf

B/C C B A B C A A

73% 67% 75% 50% 63% 68% 78% 44% 32% 46%

A A/B A B A A A B

62% 52% 75% 17% 53% 53% 67% 44% 24% 33%

B C A D B B A C

*

*

50%

30%

of alternatives uncovered. 5 = outstanding 4 = good 3 = adequate 2 = disappointing 1 = poor c Time measured in months. d Percent of decisions used from the outset. e Percent of decision in full use after a 2-year period. f Duncan multiple range test (DMRT): letter codes indicate significant differences in the means, p  .05 for each block of data. ∗ Insufficient observations. b Scale:

616

Paul C. Nutt / Omega 35 (2007) 604 – 622

Table 5 Premise and search interactions N

Joint effects of Premise

Search

Need

Opportunity

Undefined Threat

Defined Threat

a DMRT:

Freq

Negotiation Rational Problem solv. Redevelopment Opportunity Emerg. oppor. Negotiation Rational Problem solv. Redevelopment Opportunity Emerg. oppor. Negotiation Rational Problem solv. Redevelopment Opportunity Emerg. oppor. Negotiation Rational Problem solv. Redevelopment Opportunity Emerg. oppor.

31 54 37 20 13 11 16 24 13 21 37 7 6 8 4 2 2 2 0 6 2 6 12 5

9% 16% 11% 6% 4% 3% 5% 7% 4% 6% 11% 2% 2% 2% 1% < 1% < 1% < 1% 0% 2% < 1% 2% 3% 1%

Value indicators

Adoption indicators

No. of Alts.

Merit

N

DMRTa

Rating

DMRT

Time Mo

A A/B A B B B B A B

4.1 3.9 3.4 2.9 3.1 2.8 3.9 3.9 3.5 3.7 3.6 4.3 4.3 4.2 2.7

A A A/B B B B A/B A/B B B B A A A B

7.0 7.1 11.8 10.4 13.8 10.7 9.6 11.1 20.4 11.9 7.4 3.5 6.6 10.4 5.0

2.3 2.1 2.1 1.9 1.6 1.3 2.7 2.1 2.8 1.5 1.2 1.0 1.8 8.0 2.2

Duration

Initial use

Full use

DMRT

Rate

DMRT

Rate

DMRT

A A B B C B B/C C D C B A A B A

80% 67% 65% 45% 54% 45% 76% 63% 46% 71% 57% 86% 83% 100% 50%

A B B C C C A B C A C A B A C

77% 56% 43% 35% 46% 35% 70% 46% 38% 62% 38% 57% 83% 100% 50%

A B C C C C A C D B D B/C B A C

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

2.9

A

*

1.0 1.4 1.2

4.2

A

*

B B B

3.4 3.7 2.6

5.2

B

*

B A/B C

4.2 10.3 2.8

86%

*

A

*

B C A

67% 42% 0%

50%

A

*

B C

50% 33% 0%

A B

Duncan multiple range test for data in each block. observations to conduct a statistical test. Comparisons made with remaining mean values.

∗ Insufficient

threat, use increased, but value and time were unaffected. Impressionistic gaps had a negative effect on success, regardless of the premise types (needs or opportunity). This effect was much greater for a need premise. Value and use declined when a need premise followed an impressionistic gap. This effect could not be explored for threats because of insufficient observations. One interpretation might be that an impressionistic performance understanding makes it difficult for decision makers to call for action using a need premise, when faced with a threat. It may be essential to have what appears to be an answer or a remedy when a threat arises under such conditions. As noted in Table 5, some search approaches produce superior results and others inferior ones, no matter what premise is used. Negotiation always works well, but may be infeasible for defined threats. Given the pressure to act being confronted, a compromise solution would seem superfluous if a seemingly workable idea had emerged early-on. Problem solving is associ-

ated with undesirable outcomes no matter how a decision is premised. Value and use outcomes are always below that of other search approaches. A rational search approach is carried out more frequently under a need premise. This leads to superior results with better value and more use, compared to most of the other search options. A rational approach works even better when there is a threat, producing many more alternatives that prompt high value and greater use. Opportunity premises made it a bit awkward for one to use a rational approach. An idea must be set aside to permit the decision maker to search for a better one. When this occurs it reduces the success of a rational approach somewhat. Opportunities that crop up during a decision making effort tend to be unsuccessful. An opportunity premise makes this even more likely. Better results are obtained with several of the other search approaches. An exception arises for the “emergent opportunity”, an idea that emerges to displace an ongoing search. This produced decisions that had quite good value with high

Paul C. Nutt / Omega 35 (2007) 604 – 622

initial use in a short time period. However, the effect was short-term. The full use rate for emergent opportunities was just 57%. Several other search approaches had comparable value and better use, but were not as timely (e.g., bargaining and redevelopment). Redevelopment is undertaken to replace a failed idea. Unlike an “emergent opportunity”, search is mounted because the opportunity lost favor and requires a replacement. This works very well, producing good value and considerable use in a reasonable time period. Negotiation is another good way to search for solutions when an opportunity premise has emerged. Value and use are high and the time commitment reasonable (9.6 months) for opportunity premised decisions made by bargaining. Problem-directed searches are to be avoided. Lots of options are found, but the time period becomes excessive (20.4 months), and value and use are poor. Problemdirected searches were unaffected by the premise; results are poor regardless of the premise that was used to push the action forward. Undefined threats, a need premise used under conditions of very high urgency and decision importance, favors a rational approach. Rational approaches produce even better results than negotiation, with its “cooptative” advantage, when such a premise is used. Also, an opportunity search for an undefined threat is very unlikely, as logic would suggest. A need drives the decision process toward a way to improve results, such as finding a way to cut costs. The idea in an opportunity must meet such a test before it would be considered. This makes it difficult to sidetrack things with dubious ideas that may have other particularistic virtues that serve a stakeholder’s interest, but not the organization’s. Problemdriven searches under conditions of an undefined threat should be avoided, because such a search produces little value and low use (Table 5). Defined threats—opportunities offered to deal with an urgent and important decision—are handled best by ignoring the idea and mounting a rational search. This produces more alternatives that had high value and use. Similarly, a belated rational search, one that waits until the opportunity premise has been abandoned, is more successful than clinging to a dubious opportunity. Such an opportunity has little long-term use and grabbing onto an emergent one has even less success. Such approaches were found to have low value and no use. 4.5. Decision difficulty Table 6 summarizes the impact of difficulty on intelligence gathering. The table locates perceived diffi-

617

culty on the far left of the table to show how degrees of difficulty interact with each of the three intelligence factors in the columns. The data show that perceived difficulty alters the success profile for premise, gapping, and searching but not to the extent that any of the prescriptions uncovered must be changed. Still there were some interesting effects that merit discussion. Frequencies that indicate how often low and high difficulty arises with premise, gap, and search types are found in the adjacent row. The frequencies suggest whether some tactics are limited to the less difficult decision. The data suggest not. There was no shift in the number of instances in which the tactics applied for the high difficulty decision was different from the low difficult ones. Thus, a claim that low difficulty decisions lend themselves to the easily quantifiable decision is not supported. Both high and low difficulty decisions where attempted by setting need premises, stating quantitative gaps, and using rational searches. Quantitative gaps work well for both the high and low difficulty decision and better than qualitative or unknown gapping under conditions of high difficulty. Thus, quantitative gaps appear to be the best way to capture a summative notion about the intelligence gathered. The biggest effect of difficulty can be found in the increase in duration and decline of full use when an unknown gap is used for decisions with low difficulty. This causes duration to double and full use rates to fall by nearly half. The impact of decision difficulty on premising is similar to that found for gapping. The success record for premises stated as needs or as opportunities is similar for decisions with or without difficulty. Big differences arise when a threat is noted. Leaving a threat undefined (as a need) has a much higher success rate than offering a solution (an opportunity). Opportunities offered in the face of a decision with low difficulty had a disastrous effect on full used, falling to 26%. In all other cases, the success indicators were higher when decisions were treated as an undefined threat, no matter what the level of difficulty. This suggests that opportunity premises offer no advantage for either a threat-ridden or a difficult decision and that a statement of needs offers the best way to premise decision making action. Finally, high difficulty decisions do not alter the findings uncovered for the search approaches. Negotiated searches had a decline in success as decisions went from low to high difficulty. Rational searches were unaffected by difficulty. Both continued to outperform other ways to search by a significant margin, no matter what the difficulty of the decision. Opportunities work much

618

Paul C. Nutt / Omega 35 (2007) 604 – 622

Table 6 Complexity interactions with gap premise and search N

Joint effects of Complexity

Low High Low High Low High Low High Significance Totals Low High Low High Low High Significance Totals Low High Low High Low High Low High Low High Low High Significance Totals

Freq

Factor

Premise Need Need UT UT Opportunity Opportunity DT DT

Perf. gap QT QT QL QL UNK UNK

Search RAT RAT NEG NEG OPPOR OPPOR EO EO PS PS REDEV. REDEV.

61 74 10 6 54 53 19 12

21% 26% 3% 2% 17% 17% 6% 4%

289

86%

68 46 56 68 20 33

23% 16% 19% 23% 7% 11%

291

87%

46 37 16 31 26 29 4 3 29 32 23 14

16% 13% 6% 11% 9% 10% 1% 1% 10% 11% 8% 5%

290

86%

Value indicators

Adoption indicators

No. of Alts.

Merit

Na

t-test

Ratingb

t-test

Time Moc

t-test

Rated

t-test

Ratee

t-testf

ns

3.7 3.4 3.2 4.1 3.7 3.8 3.7 3.0 ns

ns

9.5 10.8 5.6 10.8 10.3 11.4 4.4 9.0 ns

ns

63% 56% 60% 71% 58% 69% 68% 50% ns

ns

56% 43% 60% 71% 48% 49% 26% 50% ns

.05

2.2 1.9 2.2 5.1 1.7 1.8 1.8 1.3 .01

1.9 2.5 2.1 1.7 1.6 1.8 ns

2.2 2.9 1.8 2.3 1.3 1.3 1.3 1.0 2.3 1.8 1.9 1.3 .10

.01 ns .05

.07 ns ns

.07 .06 ns ns .08 ns

Duration

.04 ns .05

3.7 3.9 3.8 3.6 3.3 2.9 .05

ns

3.9 3.8 3.9 4.0 3.5 3.6 3.7 2.6 3.4 3.0 3.2 3.5 .04

ns

ns .06

ns ns .10 ns ns

Initial use

.05 ns .05

8.9 10.1 8.8 11.1 9.0 4.4 ns

ns

7.0 10.9 8.3 7.6 11.1 9.4 6.0 13.0 8.8 15.2 10.6 10.7 ns

.07

ns .05

ns ns ns .05 ns

69% 78% 62% 63% 40% 33% .01

65% 70% 81% 74% 48% 64% 75% 33% 68% 34% 48% 71% .10

.07 .06 .05

ns ns .05

ns ns .05 .01 .01 .03

Full use

57% 58% 48% 51% 25% 50% .03

54% 54% 81% 64% 27% 41% 25% 33% 48% 31% 48% 43% .006

.07 ns .01

ns ns .01

ns .05 .05 .10 .10 ns

a Number

of alternatives uncovered. 5 = outstanding 4 = good 3 = adequate 2 = disappointing 1 = poor c Time measured in months. d Percent of decisions used from the outset. e Percent of decision in full use after a 2-year period. f t-test: significant differences in the means for complexity high and low. b Scale:

better for the high difficulty decisions and emergent opportunities are much worse for the high difficulty decision. But, in both cases, the best-case success rates remain far below that realized by a rational or a ne-

gotiated search. The same results are noted for problem solving as a means to search. The outcomes were no better and most were not a good as those noted for rational and negotiated searches, and duration doubles

Paul C. Nutt / Omega 35 (2007) 604 – 622

619

and use rates fall dramatically when difficult decisions are encountered. A redevelopment search (in which an opportunity failed) did quite well for high difficulty decisions. Use rates approach that noted for rational and negotiated searches. Here goals are ambiguous or politics present, making a search without clear aims a bit more defensible, which may explain this result.

proaches. Both negotiated and rational searches continued to outperform the other search approaches by a significant margin, no matter what the level of resources provided to make the decision.

4.6. Decision resources

This study uncovered a number of intelligence gathering tactics. The success of these tactics should be of interest to managers and management. The way a manager documents a triggering signal proved to be quite important. A factual measure of the performance shortfall found in a trend or event, such as current cost or margin, and expected results, such as documentation of competitor’s cost or margin, leads to the best results. Decisions made with a clear-cut justification, formed by such a performance gap, are far more successful than decisions in which performance gaps were set qualitatively or impressionistically. Premises follow the performance gap and are used to motivate action. The premise can take shape as documentation of the performance shortfall, a need, or an action to take, an opportunity. Both may be used under a “threat” (very high urgency and importance), creating an undefined or need-based threat, or a defined one with an opportunity to push the action forward. When faced with a threat, quantitative performance gaps were more apt to prompt a need-based premise and impressionistic ones an opportunity premise. Quantitative performance gaps easily translate into needs and led to successful decisions, whether faced with a threat or not. A need premise has less impact when signals are documented qualitatively or impressionistically. Premises evoke a variety of search behaviors—some are more successful than others. When opportunities (ready-made ideas) slide through and displace a search, success declines. This was noted for decisions made under a threat and for decisions with less urgency and importance. Needs were found to facilitate a rational search and increase the prospect of being successful. This was noted for threats as well. Negotiated decisions appear to be infeasible for defined threats. An available solution and pressure to act seem to make it impossible to use compromise and bargaining to uncover options in such decisions. Negotiation produced superior results when used under conditions of undefined threats. Threats do not invalidate negotiation, but the possibility of a quick fix seems to. A quick fix solution and its variations appear to be a recipe for failure. The prospect of success declines sharply when a ready-made solution was adopted at the outset. If a

Table 7 summarizes the impact of resources on intelligence gathering. As with the difficulty factor, the table locates resources on the far left of the table to show how it interacts with each of the three intelligence factors in the columns. The data show that available resources fail to alter the success profile for premise, gapping, and searching. The prescriptions stand, as with difficulty. Still there were some interesting effects that merit discussion. Frequencies that indicate how often low and high resources arise with premise, gap, and search types are found adjacent rows. The frequencies suggest when a tactic demands more resources. The data suggest that high resources are much less likely to be allocated no matter what tactic is applied. Thus, a claim that the more effective tactics call for high resources is not supported. Need premises, quantitative gaps, and rational searches were used for both high and low resource decisions. Quantitative gaps work better than qualitative ones, regardless of the level of resource support. Thus, quantitative gaps appear to be the best way to capture a summative notion about the intelligence gathered. The biggest effect of resources can be found with decision with an unknown gap. Here, resources seemed counter productive. There was an increase in the number of alternatives with high resources. However, duration doubled, quality fell, as did adoption rates. Thus, resources increase the number of ideas but not the success of a decision. Resources had little impact on premising. The success record for premises stated as needs or as opportunities is similar for decisions with high and low resources. Big differences arise when an undefined threat is noted. Pumping in resources when the threat is left undefined (as a need) more than doubles the number of alternatives considered and increases duration. However, quality fell and adoption rates declined. Again, more resources were unable to influence the success of a decision. A statement of needs remains the best way to premise decision making action. Finally, increasing resource support for a decision failed to alter the findings uncovered for the search ap-

5. Conclusions

620

Paul C. Nutt / Omega 35 (2007) 604 – 622

Table 7 Resource interactions with gap premise and search N

Joint effects of Resource

Low High Low High Low High Low High Significance Totals Low High Low High Low High Significance Totals Low High Low High Low High Low High Low High Low High Significance Totals

Freq

Factor

Premise Need Need UT UT Opportunity Opportunity DT DT

Perf. gap QT QT QL QL UNK UNK

Search RAT RAT NEG NEG OPPOR OPPOR EO EO PS PS REDEV. REDEV.

72 37 9 7 55 29 9 12

31% 16% 4% 3% 24% 13% 4% 5%

230

61%

58 36 60 29 25 25

25% 16% 26% 13% 11% 11%

228

61%

42 27 39 18 23 15 1 3 22 14 16 8

20% 13% 6% 19% 11% 7% 0% 1% 11% 7% 8% 4%

205

54%

Value indicators

Adoption indicators

No. of Alts.

Merit

Na

t-test

Ratingb

t-test

Time Moc

t-test

Rated

t-test

Ratee

t-testf

ns

3.7 4.2 4.4 3.4 3.7 3.8 3.5 3.6 ns

.06

9.0 11.1 8.0 5.3 11.7 10.7 2.9 7.0 ns

ns

70% 62% 100% 57% 58% 62% 67% 58% ns

ns

65% 58% 100% 57% 58% 55% 33% 42% .13

ns

2.1 2.4 1.5 3.5 1.8 2.0 1.5 2.1 .06

.01 ns ns

2.2 2.5 2.0 1.9 1.6 2.4 ns

ns

2.6 2.9 2.1 2.1 1.4 1.4 — — 1.7 2.2 1.5 2.7 ns

ns

.07

ns — ns .05

.03 ns ns

3.7 3.9 3.8 3.6 3.3 2.9 .05

ns

ns

Duration

ns ns .08

3.9 3.8 3.9 4.0 3.5 3.6 — — 3.4 3.0 3.2 3.5 ns

ns ns ns — ns ns

Initial use

.05 ns .04

10.4 9.1 9.8 8.5 6.6 12.0 ns

ns

7.9 11.1 6.9 8.5 7.8 12.6 — — 19.1 8.7 8.9 6.2 ns

.07

ns .03

ns .05 — .02 .07

.02 ns .07

81% 72% 70% 65% 48% 35% .003

ns

73% 74% 87% 67% 57% 53% — — 54% 36% 63% 50% .08

ns

ns .05

.03 ns — .07 .05

Full use

74% 67% 65% 58% 32% 25% .009

62% 67% 79% 61% 53% 43% — — 54% 28% 56% 50% ns

.02 ns .06

ns ns .07

ns .05 .05 — .01 ns

a Number

of alternatives uncovered. 5 = outstanding 4 = good 3 = adequate 2 = disappointing 1 = poor c Time measured in months. d Percent of decisions used from the outset. e Percent of decision in full use after a 2-year period. f t-test: significant differences in the means for complexity high and low. b Scale:

ready-made solution is derailed, things do not improve. When the ready-made solution fails, a rational search to find a replacement proved to be hard to carry out and lead to reduced chance of decision success. These find-

ings generalize to decisions with high and low difficulty and to decisions with high and low resource support. The findings suggest a number of guidelines for decision makers and the management of decision making.

Paul C. Nutt / Omega 35 (2007) 604 – 622

Quantitative performance gaps offer an effective way to extract intelligence from a signal, even when a decision has procedural and/or political difficulty and regardless of the resources available to support the effort. Also desirable is premising actions with need statements followed by rational or negotiation tactics to search for alternatives. This finding also holds for decisions with both high or low difficulty and resource support. The decisions in the database suggest that a rational search approach appear to be feasible for most, if not all decisions. Collecting data to document performance can be time consuming and costly, but has considerable payoff. First, it makes a need-based premise easy to articulate, which facilitates carrying out rational search approaches as well as negotiation. A reading of the cases in the database, which failed to apply the best approach, was used to test feasibility. The cases show that quantitative data could have been extracted from available intelligence to replace qualitative or impressionistic gap information. With this information, need statements could have been fashioned to premise a search. When a quantitative gap is translated into a need the more effective search approaches are evoked, which improves the prospect of success. References [1] Eisenhardt K, Zbaracki M. Strategic decision making. Strategic Management Journal 1992;13:17–37. [2] Harrison M, Phillips B. Strategic decision making: an integrative explanation. Research in the sociology of organizations, vol. 9, JAI Press; 1991. p. 319–58. [3] Mintzberg H, Raisinghani D, Theoret A. The structure of unstructured decisions. Administrative Science Quarterly 1976;21(2):246–75. [4] Nutt PC. Calling out and calling off the dogs: managerial diagnoses in organizations. Academy of Management Review 1979;4(2):203–14. [5] Eisenhardt K. Decision making and all that jazz. In: Papadakis V, Barwise P, editors. Strategic decisions. Boston, MA: Kluwer; 1998. [6] Downs A. Inside bureaucracy. Boston: Little Brown; 1969. [7] Pounds W. The process of problem finding. Industrial Management Review 1969; Fall: 1969; 1–19. [8] Hickson D, Butler R, Gray D, Mallory G, Wilson D. Top decisions: strategic decision making in organizations. San Francisco, CA: Jossey-Bass; 1986. [9] Bryson JM, Bromiley P, Jung VS. The influences of context and process on project planning success. Journal of Planning Education 1990;9(3):183–95. [10] Bell G, Bromley P, Bryson J. Spinning a complex web: links between strategic decision making context, content, process and outcome. In: Papadakis V, Barwise P, editors. Strategic Decisions. Boston, MA: Kluwer; 1998. [11] Starbuck WO. Organizations as action generators. American Sociological Review 1983;48:91–102.

621

[12] Cyert RM, March JG. A behavioral theory of the firm. Englewood Cliff, NJ: Prentice-Hall; 1963. [13] Simon HA. The new science of management decision. (revised edition), Englewood Cliffs, NJ: Prentice-Hall; 1977. [14] Witte E. Field research on complex decision making process—the phase theory. International Studies of Management and Organization 1972;56:156–82. [15] Soelberg PO. Unprogrammed decision making. Industrial Management Review 1967; Spring: 19–29. [16] Boal KB, Bryson JM. Representing and testing policy implications of planning processes. Strategic Management Journal 1987;8:211–31. [17] Dean J, Sharfman M. Does decision making matter? A study of strategic decision making effectiveness. Academy of Management Journal 1996;39(2):368–96. [18] Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases. Science 1974;185:1124–31. [19] Dutton JE, Dukerich JM, Harquail CV. Organizational images and member identification. Administrative Science Quarterly 1994;39:239–63. [20] Dutton JE, Jackson SE. Categorizing strategic issues: links to organizational action. Academy of Management Review 1987;17:76–90. [21] Dutton JE. The processing of crisis and non-crisis strategic issues. Journal of Management Studies 1986;23(5):501–17. [22] Cowan DA. Developing a process model of problem recognition. Academy of Management Review 1986;11(4):763 –76. [23] Locke EA, Latham GP, Erenz M. The determinants of goal attainment. Academy of Management Review 1988;13:23–39. [24] March JG, Simon HA. Organizations. New York: McGraw-Hill; 1958. [25] Cowan DA. Developing a classification structure of organizational problems: an empirical investigation. Academy of Management Journal 1990;33(2):366–90. [26] Kiesler S, Sproull L. Managerial responses to changing environments: perspectives on problem sensing from social cognition. Administrative Science Quarterly 1982;27:548–70. [27] Festinger L. A theory of cognitive dissonancem. Evanston, IL: Peterson; 1957. [28] Lerner MJ. The desire for justice and reaction to victims. In: McCauley JR, Berkowitz L, editors. Altruism and helping behavior. New York: Academic Press; 1970. p. 205–29. [29] Cray D, Mallory G, Butler R, Hickson D, Wilson D. Explaining decision processes. Journal of Management Science 1991;28(3):227–51. [30] Nutt PC. Why decisions fail: avoiding the blunders and traps that lead to debacles. San Francisco, CA: Berrett-Koehler; 2002. [31] Quinn JB. Strategies for change: logical incrementalism. Homewood, IL: Dow Jones-Irwin; 1980. [32] Cohen MD, March JP, Olsen JP. A garbage can model of organizational choice. Administrative Science Quarterly 1976;17:1–25. [33] March JG, Olsen JP. Garbage cans of decision making in organizations. In: March JG, Weissinger-Baylon R, editors. Ambiguity and command. Marshfield, MA: Putnam; 1986. [34] Mausch M, LaPotin P. Beyond garbage can: an AI model of organizational choice. Administrative Science Quarterly 1989;34:38–67. [35] Dutton JE, Duncan RB. The creation of momentum for change through the process of strategic issue diagnosis. Strategic Management Journal 1987;8:279–95.

622

Paul C. Nutt / Omega 35 (2007) 604 – 622

[36] Anonymous. Search and decision making. European Journal of Operational Research 2005;160(3):851–76. [37] Thompson JD. Organization in action. New York: McGrawHill; 1967. [38] Brunsson N. The irrationality of action and action rationality: decisions, ideologies, and organization action. Journal of Management Studies 1982;19:29–44. [39] Genmunde H, Hauschildt J. Number of alternatives and efficiency in different types of management decisions. The European Journal of Operational Research 1985;22(2):178–90. [40] Ragagopalan N, Rasheed A, Datta D, Spreitzer G. A multitheoretic model of strategic decision process. In: Papadakis V, Barwise P, editors. Strategic decisions. Boston, MA: Kluwer; 1998. [41] Nutt PC. Surprising but true: half of organizational decisions fail. Academy of Management Executive 1999;13(4):75–90. [42] Miller C, Cardinal L, Glick W. Retrospective reports in organizational research: a re-examination of recent evidence. The Academy of Management Journal 1997;40(1):189–204. [43] Nutt PC. The tactics of implementation. Academy of Management Journal 1986;29(2):230–61. [44] Papadakis V, Barwise P. Strategic decisions. Norwell, MA: Kluewer; 1998. [45] Denzin NK. The research act. Englewood Cliffs, NJ: PrenticeHall; 1989. [46] Yin RK. Applied case study research. Hollywood, CA: Sage; 1993. [47] Lincoln Y, Guba. Naturalistic inquiry. Beverly Hills, CA: Sage; 1984. [48] Huber G, Power DJ. Retrospective reports of strategic-level managers: guidelines for increasing their accuracy. Strategic Management Journal 1985;6:171–80.

[49] Nutt PC. Managing planned change. New York: Macmillan; 1992. [50] Alexander L. Successfully implementing strategic decisions. In: Mayon-White P, editor. Planning and making change. London: Harper Row; 1986. [51] Pelz DC. Some expanded perspectives on use of social science in public policy. In: Yinger M, Cutler SJ, editors. Major social issues: a multidisciplinary view. New York: Free Press; 1978. p. 346–57. [52] Beyer JM, Trice HM. The utilization process: a conceptual framework and synthesis of empirical findings. Administrative Science Quarterly 1982;27(4/5):591–622. [53] Box G, Hunter S. Response surface methodology. New York: Wiley; 1987. [54] Nutt PC. Search and decision making. European Journal of Operational Research 2005;160(3):851–76. [55] Janis I. Crucial decisions, New york: Free Press; 1989. [56] Kolb DA, Problem management: learning from experience, In: Srivastra S, editor. The executive mind, San Francisco, CA: Jossey Bass, 1983 p. 109–43. [57] Pfeffer J. Managing with power: politics and influence in organizations. Boston, MA: Harvard University Press, 1992. [58] Hogarth R. Judgment and choice, New York: Wiley; 1980. [59] Nisbett R, Ross L. Human inferences: strategies and shortcomings of human judgments (revised edition). New York: Wiley; 1989. [60] Hammond K, McCelland G, Mumpower J. Human judgment and decision making. New York: Praeger; 1980. [61] Covey S. Principled central leadership. New York: Summit; 1995. [62] Arnabile T. Creativity in context: update to social psychology of creativity, Boulder, Colorado: Westview; 1996.