W hat Does This Picture Look Like, And W hat Does Your Answer Say About You? Sour c e : L i l i e n f i e l d , S. O. , Wo o d , J. M . , & G a r b, H . N. ( 2 0 0 0 ) . T h e s c i e n t i f i c

C I RC

status of projective techniques. Psychological Science in the Public Interest, 1, 27-66.

Goals: Te a c h t h e i m p o r t a n c e o f v a l i d i t y. Te a c h t h e i m p o r t a n c e o f r e l i a b i l i t y. Te a c h t h e i m p o r t a n c e of independent agreement.

Basic Idea: This lesson discusses the scientific use of p r o j e c t i v e t e s t s. Projective tests are used by psychologists to predict a variety of personality characteristics based solely on how a person responds to a few q u e s t i o n s. C a n scientists really d i s t i n g u i s h p e o p l e ’s personalities with just a few responses to a drawing?

Gain Attention/Interest: Activity

Show the class the sample inkblot included at the end of this lesson and have the students write a few thoughts a b o u t w ha t t h e y s e e. A s k t h e m t o c o m p a r e r e a c t i o n s. This will illustrate the variety of interpretations resulting from the ambiguous inkblot. Responding to an inkblot will help students better understand the rest of the lesson.

Alternate Activity

Show the class a picture (any picture can work, although one they are not familiar with would be best) and have each student write a brief story about the picture. Comparing a few volunteered stories will illustrate the number of different ways people can interpret a single stimulus (in this case, the picture). Have you ever looked up at the sky and seen a cloud that reminded you of something? A horse, a clown, or a car? What would you think if some scientists told you that what you saw in that cloud told them something about your personality? Wo u l d y o u b e l i e ve i t ? What if someone asked you to draw a picture of a certain p e r s o n ? Wo u l d yo u f e e l c o m f or ta b l e h av i n g h e r l o o k a t t h e picture and decide whether or not you are depressed, a g g r e s s ive , o r p o o r l y a d j u s t e d ? I m a g i n e b e i n g a c c u s e d o f a crime and having a scientist ask you to draw a picture. What if how that scientist interpreted your drawing would have a large impact on whether or not a guilty verdict was returned by a jury during your trial?

www.circ.cornell.edu

C o p y r i g h t © 2 0 0 5 M a t t h e w C . M a k e l & We n d y M . W i l l i a m s. P r i n c i p a l I n v e s t i g a t o r : We n d y M . W i l l i a m s . C o n t e n t : M a t t h e w C . M a k e l & We n d y M . W i l l i a m s . L a y o u t & D e s i g n : P a u l B. P a p i e r n o.

T h e s e a r e e x a m p l e s o f p r o j e c t i v e t e s t s. T h e y a r e c a l l e d p r o j e c t i ve t e s t s b e c a u s e t h e y a t t e m p t t o t a k e a s m a l l p a r t o f o n e ’s p e r s o n a l i t y a n d p r o j e c t i t f u r t h e r. T h i s i s s i m i l a r t o a p r o j e c t i l e s u c h a s a b a s e b a l l o r s p a c e s h i p. They are projectiles because they are thrown/launched/propelled.

T h i n k & Wr i t e # 1 Have students write their thoughts about p r o j e c t i v e t e s t s. D o t h e s e t e s t s wo r k ? A r e t h e y wo r t h w h i l e ? A r e t h e y s c a r y ? There are many types of projective tests; some show seemingly random inkblots (like a stain from spilling a glass of punch on a white tablecloth), while other projective tests ask people to perfor m a specific task (such as drawing a picture) from an open-ended question. An open-ended question h a s n o s e t n u m b e r o f a n s w e r s ( e . g. , “ H o w d o y o u f e e l ? ” ) . A q u e s t i o n t h a t i s n o t o p e n - e n d e d h a s o n l y s p e c i f i c a n s w e r s ( e . g. , “A r e y o u h a p p y ? ” ) . Based on the answers given, psychologists deter mine personality traits and t e n d e n c i e s. A p o p u l a r e x a m p l e o f a p r o j e c t i v e t e s t i s t h e Ro r s c h a c h ( p r o n o u n c e d R OA R - s h o c k ) t e s t , w h i c h i s a d m i n i s t e r e d a b o u t s i x m i l l i o n t i m e s a y e a r. How can we understand projective tests and how they work? Through science.

Ask: What is Science?

S c i e n c e i s t h e s e a r c h f o r a n s w e r s t o q u e s t i o n s. I t i s not limited to questions about space or chemical m o l e c u l e s. S c i e n c e c a n b e a b o u t a n y t h i n g. S o m e people may want only to find the answer that shows that they are right, but scientists take careful steps to m a k e s u r e t h e y f i n d t h e b e s t a n s w e r. To d o t h i s, scientists use proof by disproof when solving p r o b l e m s. I f a l l b u t o n e o f t h e p o t e n t i a l a n s w e r s i s eliminated, then scientists conclude they have p r o b a b l y f o u n d t h e c o r r e c t a n s w e r. B y e l i m i n a t i n g a l l o t h e r p o s s i b l e a n s w e r s, t h e y h a v e f o u n d a p o s s i b l e c o r r e c t o n e b y d i s p r ov i n g a l l o t h e r s. O n l y a f t e r d o i n g t h i s d o s c i e n t i s t s b e l i e v e t h e y m i g h t h a ve a r i g h t a n s w e r.

1

Ask: What is Science?

2 Define the Problem: See Many Sides.

Define the Problem: See Many Sides Before we go any further, we need to complete an important step that all scientists must complete w h e n t h e y a n s w e r q u e s t i o n s. We h a ve t o c l e a r l y define our problem. Can scientists really predict a large part of your personality based solely on how you interpret an inkblot or on how you draw a p i c t u r e ? D o n’t y o u r a n s w e r s d e p e n d a t l e a s t i n p a r t on what kind of mood you are in? Or does our tr ue personality influence our behavior so much that it can be perceived even though we have only answered a f e w q u e s t i o n s ? O b v i o u s l y, m a n y q u e s t i o n s c o u l d b e a s k e d a b o u t t h i s t o p i c . To d a y w e w i l l d i s c u s s o n e of the most general questions: Do projective tests work?

T h i n k & Wr i t e # 2 Hypothesis for mation Have students for m a hypothesis reg arding whether or not they think projective tests work. Urge them either to give support for their answer or to give an example of what they could do to find support. I n o u r f i r s t a c t i v i t y, w e s a w s o m e t h i n g s i m i l a r t o a r e a l p r o j e c t i v e t e s t . What do you think scientists would have to do to be convinced that the tests actually work? As with any controversial topic, different people have va r y i n g o p i n i o n s o n t h e u s e o f p r o j e c t i v e t e s t s. S o m e t h i n k p r o j e c t i v e t e s t s h e l p s c i e n t i s t s l e a r n a c c u r a t e i n f o r m a t i o n a b o u t a n i n d i v i d u a l ’s p e r s o n a l i t y. O t h e r s f e e l t h e s e t e s t s c a n n o t p o s s i b l y p r ov i d e p r e c i s e infor mation about all people. What are some g roups of people who might be interested in whether or not projective tests should be used?

A f e w s a m p l e g r o u p s / p r o f e s s i o n s, a l o n g w i t h t h e i r e d u c a t i o n a l r e q u i r e m e n t s, a r e l i s t e d b e l o w Clinical Psychologist. Clinical Psychologists help patients with psychological problems and often use projective tests to help learn about t h e p e r s o n a l i t i e s o f t h e i r p a t i e n t s. C l i n i c a l p s y c h o l o g i s t s a t t e n d f o u r - y e a r c o l l e g e s a n d t h e n g o o n t o e a r n g r a d u a t e d e g r e e s. Ju d g e . Ju d g e s p r e s i d e o v e r c o u r t c a s e s i n w h i c h p r o j e c t i ve t e s t s a r e u s e d as evidence. Judges attend four-year colleges and then go on to law school. L a w y e r. L a w y e r s c a n u s e p r o j e c t i ve t e s t s t o h e l p t h e i r c l i e n t s. L a w y e r s c a n a l s o t r y t o d i s c r e d i t p e o p l e w h o u s e p r o j e c t i ve t e s t s. L a w y e r s a t t e n d four-year colleges and law school. S o c i a l Wo r k e r. S o c i a l wo r k e r s u s e t h e r e s u l t s o f p r o j e c t i v e t e s t s t o e v a l u a t e c l i e n t s. S o c i a l w o r k e r s a t t e n d a t l e a s t f o u r y e a r s o f c o l l e g e a n d m a y a t t e n d g r a d u a t e s c h o o l f o r t wo a d d i t i o n a l y e a r s a n d e a r n a M a s t e r ’s o f C l i n i c a l S o c i a l Wo r k ( M C S W ) d e g r e e . What about you? Do you think scientists should use projective tests? ( A s k f o r s h o w o f h a n d s, Ye s o r N o. )

3 Fact Versus Opinion: What Constitutes Evidence?

Distinguish Fact From Opinion: Learn What Constitutes Scientific Evidence Opinions are interesting, but scientists have to c h e c k t h e f a c t s. T h e y d o n o t r e l y o n o p i n i o n s o r the thoughts of other people. They need facts to convince them that an answer is right or w r o n g. S c i e n t i s t s u s e s e v e r a l c r i t e r i a t o e v a l u a t e whether or not projective tests actually succeed a t a s s e s s i n g p e r s o n a l i t y. T h e s e c r i t e r i a a r e : Va l i d i t y : Va l i d i t y m e a n s t h a t t h e t e s t s a r e actually measuring what they claim to be m e a s u r i n g. A t e s t t h a t i s s u p p o s e d t o m e a s u r e y o u r p e r s o n a l i t y, b u t c o n s i s t s o n l y o f m a t h p r o b l e m s, i s n o t v a l i d . A p r o j e c t i ve t e s t w o u l d need to measure personality accurately to have h i g h v a l i d i t y. I f i t d i d n o t m e a s u r e p e r s o n a l i t y, then it would not be valid.

Reliability: A test is reliable (or has reliability) if it measures consistently from time to time. A test is reliable if a person taking the test more than once receives similar results each time the test is taken. If a person takes a projective test five times and is labeled depressed only once, then the test is not reliable. Independent agreement: A test shows independent agreement if multiple scientists using the test draw the same conclusions independently of each o t h e r. I f o n l y o n e p e r s o n c o n t i n u a l l y g e t s t h e s a m e r e s u l t s, t h e n i t m i g h t just be a result of how that one person conducts the test. If multiple people find the same result, we can have greater confidence that the test is accurate. It is only when scientists answer a question in a way that is valid, reliable, and with independent agreement, that they can be said to have solid evidence to make a decision. Ju s t t o m a k e s u r e t h i n g s a r e c l e a r , l e t ’s g o o v e r a f e w e x a m p l e s. W h i c h o f these is an example of meaningful evidence? Marcia takes a projective test for fun and finds out she gave the same answer for one of the questions as did her friend who s u f f e r s f r o m d e p r e s s i o n . M a r c i a t h e n b e g i n s t o wo r r y t h a t s h e might be depressed as well. A scientist would not consider Marcia depressed, because depression was n o t m e a s u r e d i n a v a l i d w ay, g i ve n t h e a b s e n c e o f o t h e r s y m p t o m s o f depression. Becca has been seeing a therapist for several months and, after taking a well-respected projective test given to her by her t h e r a p i s t , s h e w a s d i a g n o s e d w i t h b i p o l a r d i s o r d e r. A f t e r s h e started taking medication for bipolar disorder, Becca started f e e l i n g s i g n i f i c a n t l y b e t t e r. The example with Becca represents a valid and reliable test. Shawn and his friends wrote a bunch of questions about p e r s o n a l i t y a n d g a ve t h e m o u t t o f r i e n d s. A f t e r s c o r i n g h i s b e s t f r i e n d ’s t e s t , S h a w n s a i d h e s h o u l d s e e a p s y c h o l o g i s t b e c a u s e something was wrong with him. S h a w n’s c o n c e r n m a y e ve n t u a l l y t u r n o u t t o b e va l i d . H o w e ve r , i t i s n o t b a s e d o n s c i e n c e . H i s m e a s u r e s h o w s n e i t h e r r e l i a b i l i t y n o r v a l i d i t y.

Weigh Evidence and Make Decisions Back to our main question: How well do projective t e s t s wo r k ? To c h e c k , l e t ’s t e s t t h e m o n t h e s c i e n t i f i c r e q u i r e m e n t s w e j u s t d i s c u s s e d : v a l i d i t y, r e l i a b i l i t y, and independent agreement. Yo u m a y w a n t t o b r i e f l y r e v i e w t h e d e f i n i t i o n o f e a c h t o e n s u r e s t u d e n t u n d e r s t a n d i n g.

4 Weigh Evidence and Make Decisions.

The Results: Research on projective tests shows little v a l i d i t y o r r e l i a b i l i t y f o r m o s t p e r s o n a l i t y t r a i t s. T h i s means that the tests did not accurately predict the personality traits they were tr ying to measure (thus they were not valid), or that they did not make the same predictions across different times (thus they were not reliable). A non-scientist might read this i n f o r m a t i o n a n d c l a i m t h a t p r o j e c t i ve t e s t s a r e w i t h o u t v a l u e a n d s h o u l d never be used. However, research has shown that projective tests may be va l i d a n d r e l i a b l e m e a s u r e s f o r s o m e p s y c h o l o g i c a l d i s o r d e r s. D i s o r d e r s such as schizophrenia, bipolar disorder, and dependency appear to be r e g u l a r l y i d e n t i f i e d u s i n g p r o j e c t i ve t e c h n i q u e s. F. Y. I This means that scientists have found that Schizophrenia is a disease of the brain with a variety of symptoms projective tests can predict the presence, with i n c l u d i n g , b u t n o t l i m i t e d t o, s o m e a c c u r a c y, o f d i s o r d e r s s u c h a s hearing voices and believing schizophrenia, bipolar disorder, and people are reading/controlling d e p e n d e n c y. T h i s i s n o t a l a r g e n u m b e r o f o n e ’s m i n d o r p l o t t i n g t o h u r t o n e . d i s o r d e r s, b u t i t i s s o m e . R e g a r d l e s s o f w h a t he/she thought beforehand, a scientist would Bipolar Disorder causes unusual t a k e t h i s i n f o r m a t i o n a s a s i g n o f s u p p o r t t h a t c h a n g e s i n a p e r s o n’s b e h a v i o r a n d mood. It is called “bipolar” p r o j e c t i v e t e s t s a r e n o t u s e l e s s, t h o u g h t h e because people with it can have evidence suggests that their use should be drastic mood swings, ranging from limited. a person being extremely “up” to a

T h i n k & Wr i t e # 3 How about now? Have students record their thoughts now that they have learned what scientists have found regarding pr o j e c t ive t e s t s. Wer e t h e i r hy p o t h e s e s c o r r e c t ? Do they feel the same as they did during Think & Wr i t e 1 ?

person being extremely “down”. Dependency is the need to rely on an external source (frequently an illicit drug or alcohol) in order to function.

Move From Science to Society

5 Move From Science To Society.

Should projective tests be banned and never used a g a i n ? S o m e m a y b e l i e v e s o, b u t t h i s i s a n e x t r e m e position because projective tests can be useful tools in h e l p i n g p s y c h o l o g i s t s p r e d i c t s o m e p r o b l e m s. However, research has found that projective tests tend t o b e v a l i d a n d r e l i a b l e i n o n l y a f e w d i a g n o s e s, a n d thus over use is a risk. What if a projective test had the potential to help one o f y o u r f a m i l y m e m b e r s ? Wo u l d y o u w a n t t o s e e t h e test banned, or used? What about the question we discussed earlier: Whether the results of a projective test could help or hurt your defense in a court case? Wo u l d y o u b e w i l l i n g t o r e l y o n a s c i e n t i s t w h o c l a i m e d to be able to use a projective test to learn about your personality? Scientists are forced to answer questions s u c h a s t h e s e f r e q u e n t l y.

Revisit, Review, Reflect, and Re-evaluate Current findings have led researchers to believe that p r o j e c t i v e t e s t s m a y b e h e l p f u l i n s o m e a s p e c t s, b u t n o t i n a l l . T h u s, p r o j e c t i v e t e s t s c a n b e s t b e u s e d i n a l i m i t e d w a y. H o w e v e r, t h e l a s t w o r d h a s n o t b e e n s p o k e n a b o u t p r o j e c t i v e t e s t s. S c i e n t i s t s c o n s t a n t l y r e v i s i t p r e v i o u s l y - a n s w e r e d q u e s t i o n s. Pe r h a p s a n e w method of testing could make projective tests more a c c u r a t e f o r a w i d e r v a r i e t y o f p e r s o n a l i t y t r a i t s. However, these new findings can only be uncovered if scientists review current and past research. By r e f l e c t i n g o n t h e s e t h i n g s, f u t u r e s c i e n t i s t s m a y r e evaluate past answers and discover that projective t e s t s s h o u l d n o t b e u s e d , o r c o n v e r s e l y, t h e y m a y l e a r n that projective tests can be used for a variety of phenomena. By revisiting, reviewing, reflecting, and re-evaluating, scientists gather new evidence and continue to make contributions to what we know a b o u t p r o j e c t i v e t e s t s.

6 Revisit, Review, Reflect, and Re-evaluate.

K e y TTe h i nrkm &sWr i t e # 4 What's next? B a s e d o n w h a t t h e y k n o w n o w, a s k s t u d e n t s : W h a t is the future of projective tests? Should clinical psychologists continue to use them? What other things should scientists check regarding projective tests? Why?

Discussion Questions 1 . Wo u l d y o u w a n t p r o j e c t i v e t e s t s u s e d o n y o u ? W hy o r w hy n o t ? 2 . C a n y o u t h i n k o f a n y e x a m p l e s w h e n r e l i a b i l i t y, v a l i d i t y, o r independent agreement would not be important?

H o m e wo r k Q u e s t i o n s 1. Find another situation in which reliability is important. 2. Find another situation in which validity is important. 3. Find another situation in which independent agreement is important.

Cor nell Institute for Research on Children

January 3, 2005

Quiz Questions Ve r s i o n A

1. What kinds of criteria do scientists need in order to decide that a test can be used? a. b. c. d.

reliability validity independent agreement all of the above

2. Reliability means: a. b. c. d. e.

it is consistent from time to time goodness of fit different people reach the same conclusion best intentions it measures what it claims to measure.

3 . Va l i d i t y m e a n s : a. b. c. d. e.

it is consistent from time to time goodness of fit different people reach the same conclusion best intentions it measures what it claims to measure.

4. Independent agreement means: a. b. c. d. e.

it is consistent from time to time goodness of fit different people reach the same conclusion best intentions it measures what it claims to measure.

Quiz Questions Ve r s i o n B

1 . H o w i s i t p o s s i b l e t h a t p e o p l e ’s i n t e r p r e t a t i o n s o f i n k b l o t s c a n b e u s e d to decide diagnoses for mental health purposes? What kinds of criteria do you need in order to decide that such uses are possible? a. b. c. d. e.

goodness of fit independent agreement reliability validity best intentions

2.

means a test is consistent from time to time

3.

means different people reach the same conclusion

4.

means it measures what it claims to measure.

Quiz Questions Ve r s i o n C

1 . H o w i s i t p o s s i b l e t h a t p e o p l e ’s i n t e r p r e t a t i o n s o f i n k b l o t s c a n b e u s e d to decide diagnoses for mental health purposes? What kinds of criteria do you need in order to decide that such uses are possible and appropriate?

2. What is reliability? Explain and give an example.

3. What is validity? Explain and give an example.

4. What is independent agreement? Explain and give an example.