Decision Making Through the Lens of Ignorance* Eileen Gambrill, Ph.D. School of Social Welfare University of California, Berkeley USA**

*Decisions, Assessment, Risk, and Evidence (DARE) Conference, Belfast Ireland, July 1-2, 2014. **Hutto Patterson Professor of Child and Family Studies, School of Social Welfare, University of California, Berkeley, Berkeley, CA 94720-7400, USA. [email protected]

1

Need to Make Decisions About: • • • •

Problems to focus on How to frame problems (what are causes) How to estimate risk/prognosis What assessment (diagnostic) data to gather and how to do so • Whether to intervene (threshold level) • What intervention methods to select (e. g., investigate, take a child into care, etc.) • How to measure outcome 2

Need to Make Decisions (Cont’d) • How to allocate scarce resources • What to do when things go wrong • How and/or whether to plan for maintenance of positive outcomes • How to resolve differences of opinion • What criteria to rely on when making decisions 3

Naturalistic Decision Making • • • • •

Ill-structured problems Uncertain, changing circumstances Time pressures High stakes Multiple players with competing goals

Ref: Gigerenzer, 2014; Klein, 1998; Salas, Rosen & DiazGranados, 2010; Woods & Cook, 1999

4

Uncertainties (Examples) • Natural history of behavior • Causes related to behaviors of interest • Effectiveness of a given method for an individual • Access to needed resources

5

Wolf (2000, p. 136)

6

Current Problems • Failure to use methods that have been critically tested and found to be effective • Use of methods that have been found to be ineffective • Use of methods that have been found to be harmful • Use of unnecessary procedures • Promoting use of untested methods • Failure to involved clients as informed participants 7

Disease Mongering/Medicalization* • Stretching diagnostic boundaries • Non medical concerns are transformed into medical ones (e. g., misbehavior of children) • Portraying mild concerns as serious illnesses • Making healthy people think they are sick to sell products • Creating “mental illnesses” – Origin of the diagnosis of “social anxiety disorder” (Moynihan and Cassels, 2005)

• Making healthy people think they are sick to sell more products Ref: Conrad, 2007; Payer, 1992; Szasz, 2007

8

Making the Well Sick • • • • •

Watch out! You may have… You may catch… Do you have …? Be sure to get screened for …

9

Disease Mongering/Medicalization (Cont’d) Can manuscript reviewers detect problem framing that reflects disease mongering? Does a propaganda index facilitate detection (Gambrill & Reiman, 2011)?

10

Ignorance as a Key Reason • Has a fundamental role in our lives • Is socially constructed – Vested interests in creating and maintaining ignorance; for example: Adverse effects of prescribed drugs About ineffectiveness of services • May be avoidable or unavoidable. • Its distribution is unequal. - “Who knows not and why not?”(Proctor and Schiebinger, 2008). - Like knowledge, ignorance can be power. 11

Kinds of Ignorance (Kerwin and Witte (1983) • Known unknowns: all the things we know we do not know • Unknown unknowns: all the things we do not know we do not know • Unknown knowns: all the things we do not know we know 12

Kinds of Ignorance (Cont’d) • Errors: things we think we know but do not • Taboos: “Socially enforced irrelevance” • Denials: things too painful to know

13

About Ignorance • Is not always negative - Spurs investigation regarding problems - Encourages caution by avoiding premature judgment –It encourages productive humility (Darwin, 1871). • Closely related to uncertainty; decreasing ignorance may increase uncertainty 14

Questions About Ignorance • What are sources? Who are the purveyors of ignorance and what are their reasons? • What strategies are used to promote ignorance? • Do researchers and/or educators contribute to avoidable ignorance? • Is it always good to reveal ignorance? • How can we minimize avoidable ignorance that has harmful consequences? • How is avoidable ignorance related to well-being and clinical performance? 15

Propaganda as a Source of Ignorance • “..propaganda seeks to induce action, adherence, and participation-with as little thought as possible” (Ellul, 1965). • Different levels of propaganda analysis Societal: The central role of integrative propaganda in advanced technological societies: technology presses for ever greater efficiency and standardization (Ellul, 1965). – Technology as magic (Stivers, 2001) • The media, advertising, public relations - Promotion of a way of life and related products 16

Schneider WH, Danger: Men Talking, 1965

17

Device-Based Views of Propaganda • Censorship – Hide well-argued alternative views – Hide lack of evidence for claims

• Distortion – Misrepresent evidence-based practice

• Diversion – Create fear – Encourage ridicule

• Fabrication – Make up data

• Appeal to prejudices and emotions 18

Sources of Ignorance Vague, misleading government and agency reports The biomedical industrial complex (e. g., the medicalization of human suffering) One’s self (e. g., arrogance, statistical innumeracy) 19

Schneider WH, Danger: Men Talking, 1965

20

Sources of Ignorance • The peer reviewed literature is deeply flawed. Ioannidis (2005; 2008) argues that most published research findings are false. – Bogus citations – Failure to replicate – Frequency of retractions (e. g., see retractionwatch.com)

21

Benefits of a Lens of Ignorance • Avoid a justification approach to knowledge in which we seek support for our views (Popper, 1992, 1994). • Minimize egocentric and sociocentric influences (Paul, 2003; Paul & Elder, 2014).

• Encourage use of critical thinking knowledge, skills, and values. • Highlight opportunities to decrease avoidable ignorance by drawing on available knowledge.

22

Need for a Systemic View: Long Advocated, Still Languishing • Characteristics of social workers • Characteristics and circumstances of clients including their neighborhoods and communities • Available resources • Quality of professional education and training

23

Need for Systemic View • Service system and related policies, practices, and values • Organizational characteristics • Research priorities and frameworks

24

Dysfunctional Organizational Cultures • A blame culture - Blame rather than learn • An authoritarian culture - Taboos: It is not safe to discuss certain topics • A defensive culture - Hide what is done to what effect

25

Risks from Referrals • Purchasing services from agencies which provide harmful and/or ineffective services In what percentage do clients receive high quality services? In what percentage do clients receive ineffective and or harmful services? Are conflicts of interests involved? 26

Risks from Failing to Arrange Timely Feedback • About outcomes of decisions • About quality of common factors offered following every session • About client complaints and compliments • About errors and their consequences

27

Dysfunctional Organizational Arrangements • Dysfunctional contingencies, e.g. lack of timely positive feedback for desired behaviors. • Limited or poor supervision • Excessive work loads • Dysfunctional recording requirements. - They do not contribute to problem solving. - They drain time from face to face contact with clients. 28

Organizational Arrangements • Poor match between staff competencies and what is needed to help clients: e.g., provision of irrelevant or misleading continuing education programs • Lack of transparency about what is done to what effect including the evidentiary status of services offered • No one is responsible for overall service coordination and quality (lack of service integration). 29

Organizational Arrangements • • • •

Poor communication among staff Groupthink Lack of support for developing effective teams Ineffective audit systems

30

Organizational Arrangements • No effective harvesting of complaints and compliments and corrective action based on this • No effective tracking of errors and acting on what is found • Lack of needed tools to make timely decisions (e. g., checklists, computers, Apps) • A failure to involve clients as informed participants 31

Risks From Relying on Misleading Criteria to Evaluate Claims • • • • •

Unfounded authority Tradition Popularity Testimonials Case examples

• • • •

Newness Anecdotal experience Consensus Post hoc ergo propter hoc 32

Risks from Ignoring Misleading Effects of Language • Use of “weasel” words and phrases. - “It is well known that …” - “It is agreed that …” • Conviction through repetition • Use of vague terms (e.g. the Barnum Effect) • Reification • Misleading metaphors • Pseudotechnical jargon 33

Risks from Ignoring Common Fallacies • Begging the question: assuming what should be argued • Ad hominem arguments (at the person) • Strawperson: misrepresenting positions • Diversion/red herring • Is-ought fallacy • Ecological fallacy: assuming what is true of a group is true of an individual • Assuming a weak argument is false 34

35

Implications of Walton’s Approach to Fallacy • Conversations have different goals. – Decide if the goal of a discussion is to arrive at the most accurate answer. – Decide if you are really interested in pursuit of an accurate answer. – If yes to both of the above, related questions are never out of order and fallacies used as illicit attempts to block the goal of discovering what is most accurate, are always out of order. 36

37

38

Risks from Ignoring the Influence of Social Psychological Persuasion Ploys • • • • • •

A confident manner Imitate the majority Persuasion by affect Reciprocity rule Principle of liking Contrast effect

• Appeal to consistency • Promoting false scarcity • Making it personal • Appeal to authority

39

Risks from Lack of Knowledge About Research on Decision Making • We use both intuitive and analytic thinking styles. • Intuition may be uninformed or informed (by extensive experience providing corrective feedback, Hogarth, 2001).

40

Research on Decision Making • Less may be more; satisfying using simple rules (Gigerenzer, 2002, 2014) – Ecological rationality: fast and frugal heuristics – Heuristic is a simple decision strategy that focuses on the few relevant predictors (Mareweski & Gigerenzer, 2012) • Situational awareness is vital • Use of debiasing strategies is important 41

Use Frequencies to Estimate Risk 1000 families

4 abuse

3 pos. result

1 neg. result

996 not

259 737 pos. result neg. result

See: Munro (2005). 42

Recognize Errors as Learning Opportunities • Errors are inevitable. • A cascade of errors may occur as in diagnostic momentum. • They are typically due to systemic causes such as lack of timely corrective feedback.

43

Examples of Errors • • • • • •

Assuming that form equals function Confusing naming and explaining Treating continuous variables as discrete Treating dynamic events as static Confusing correlation with causation Confusing form and function

44

Avoidable Errors in Risk Assessment • Using relative rather than absolute risk • Using misleading surrogate measures • Confusing risk and uncertainty

45

What About Cognitive Biases? • Availability (e. g., vivid case examples, anchoring effect) • Confirmation biases; cherry picking • Belief perseverance • Premature closure • Fundamental attribution error: overemphasizing personality-based explanations for behaviors of others, while underestimating the role of situational influences regarding ourselves • Framing effects 46

Framing Effects: An Example Doctor: If you have orthoscopic surgery on your knee, you will have a good chance for full use of your knee. Patient: How good a chance? Doctor: In about 75% of such cases the operation is a complete success. Patient: And what about the Cortisone treatment? Doctor: About a quarter of those who get Cortisone do not improve to full use of the knee. Patient: Let’s do the knee surgery. 47

Cognitive Biases • • • • •

Overconfidence Reliance on representativeness (similarity) Wishful thinking Affect heuristic: influence by feelings False consensus effect (we overestimate the extent to which others agree with us) • The validity effect • Stereotyping • Hindsight bias 48

Cognitive Biases • Status quo bias • Omission bias: judging harmful actions as worse than equally harmful inactions • Sunk cost error • Overlooking regression effects • Gambler’s fallacy • Base rate neglect: ignoring true prevalence 49

Ourselves as a Source of Avoidable Ignorance • Ignorance concerning the role of integrative propaganda in our society • A justification approach to knowledge • Lack of critical thinking, values, knowledge and skills including knowledge concerning cognitive biases and common fallacies • Confusing skepticism and cynicism

50

Ourselves as a Source • Motivational influences - Lack of caring about and empathy for clients; value status and/or funding more than helping clients and avoiding harm - Value winning rather than understanding - Lack of curiosity - A preference for entertainment - A reluctance to assume responsibility for decisions - A reluctance to hold others responsible for their actions and decisions 51

Ourselves as a Source • Emotional influences - Low tolerance for uncertainty - Fear of rejection: need for social approval - Fear of appearing ignorant (difficulty in saying “I don’t know”) - Too much (or too little) fear of making mistakes and fear of failure - Vulnerability to emotional appeals - Mood changes - Lack of courage 52

Ourselves as a Source • Arrogance: inflated self-assessments; incorrect appraisals of the gap between one’s knowledge and what is available • Lack of self-management skills (e. g., time planning, emotion regulation) • Lack of assertive skills

53

Ourselves as a Source • Unrealistic Expectations • Reluctance to assume responsibility for decisions • Reluctance to hold other responsible for their actions and decisions

54

Benefits of a Lens of Ignorance • Keep the uncertainty in making decisions in view • Encourage productive disagreement • Contribute to life long learning • Highlight opportunities to decrease avoidable ignorance

55

Benefits of a Lens of Ignorance • Recognize criticism as vital for making informed decisions because we are all equal in our vast ignorance (Popper, 1992, 1994) • Antidote to burnout (e. g., from unrealistic expectations)

56

Take Advantage of Valuable Websites • The Cochrane Database of Reviews • The Campbell Database of Reviews • Database of Uncertainties about the Effectiveness of Treatment (DUETS) • Fallacyfiles.com • Pharmedout.org

57

Cultivate Critical Thinking Values • Courage: Critically appraise claims regardless of negative reactions. • Curiosity: An interest in deep understanding and learning. • Intellectual empathy: Accurately understanding and presenting the views of others. • Humility: Awareness of the limits of knowledge including our own lack of arrogance (e. g., promoting bogus claims of effectiveness). • Integrity: Honoring the same standards of evidence to which we hold others. • Persistence: Willingness to struggle with confusion and uncertainty. _______________________________________________

Based on R. Paul (1993). Critical thinking: What every person needs to survive in a rapidly changing world (3rd Ed.). Foundation for Critical Thinking www.criticalthinking.org

58

Downsides of a Focus on Ignorance • • • •

Potential loss of placebo effects Potential discouragement Loss of funding if inflated claims are not made. Rejections of manuscripts because of honest description of findings

59

Ethical Obligations as a Guide • • • •

Beneficence Non-malfeasance Self-determination Justice (equitable distribution of scarce resources)

60

59