Introduction to Survey Research Methods

Introduction to Survey Research Methods Micah Gell-Redman∗ This version, November 18, 2014 Lecture Notes Prepared for Students of Field Survey Resear...
27 downloads 0 Views 306KB Size
Introduction to Survey Research Methods Micah Gell-Redman∗ This version, November 18, 2014

Lecture Notes Prepared for Students of Field Survey Research Methods, UCSD Fall 2014 ∗ UCSD

Political Science

Contents 1

Introduction to the Research Process

1

2

Research Questions

5

3

Research Design

8

4

Life cycle of a survey question

13

4.1

Defining the Concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4.2

Choosing a measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

A The Literature Review

18

A.1 Frequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . . 22 B In-Class Presentations

23

C Steps for producing questionnaire items

25

2

1

Introduction to the Research Process Many different goals can be pursued through survey research. These various

goals are each important, but they are separate from the one goal that almost all survey-based research shares - to answer important questions in the social and behavioral sciences. It may not sound so fancy, but this common goal is what brings together the professors and graduate students participating as your instructors. It is also at the core of the commitment we made to the organizations that provide funding to support the program. In short, all of the other things we want to accomplish depend on it. Your first job in this course is to gain a basic understanding of the research process in the social and behavioral sciences, so that you understand how a survey needs to be planned and administered in order to be a valid instrument for accomplishing our shared goal. Our introduction to the research process in the social and behavioral sciences will be divided into four sections: 1. Overview 2. Research questions and hypotheses 3. Research design and sampling 4. Variables and measurement In less formal language, the four sections will answer the following four questions. What is the point of learning survey research methods? What kind of questions do social and behavioral scientists ask? How can these questions be answered adequately? How can the concepts we are studying be made mathematically precise? 1

We begin now by explaining, in general terms, what social and behavioral scientists do. Social and behavioral scientists simplify and formalize the complex world in order to uncover systematic relationships between concepts. Let’s take this definition apart piece by piece. Simplify - In order to offer an explanation that can be applied to more than one particular case, it is necessary to set aside some detail. Consider this excerpt from a text by the Argentine author Jorge Luis Borges: En aquel imperio, el arte de la cartografía logró tal perfección que el mapa de una sola provincia ocupaba toda una ciudad, y el mapa del imperio, toda una provincia. Con el tiempo, estos mapas desmesurados no satisficieron y los colegios de cartógrafos levantaron un mapa del imperio, que tenía el tamaño del imperio y coincidía puntualmente con él. . . ’1 A map is useless unless it provides some way of condensing and simplifying information about the world around it. If the map is too detailed, it cannot be easily read by someone trying to navigate the terrain.2 As our maps get smaller, they become easier to use, but less detailed. An enduring challenge for researchers is to decide how much detail to leave out, and how much to retain. Formalize - Researchers must provide clear definitions of the concepts they are 1 Excerpted

from “Del rigor en la ciencia”. Translation: In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast map was Useless, and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and Winters. In the Deserts of the West, still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in all the Land there is no other Relic of the Disciplines of Geography. 2 Consider a physician who, observing a patient, can remark enough detail to fill many pages of notes, but only a few words are required for diagnosis

2

studying, and precise methods for measuring these concepts. As we’ll see when we cover research design, it is almost impossible to build a body of systematic knowledge without precise definitions and clear measurements. Definitions may come from previous research, from mathematical models or by convention among scholars. Ignore the systematic relationships part for a moment, we’ll come back to that. For now, focus on the term concept. To understand what that term means in this context, we need to define a few more terms. Variable - a concept that has been clearly defined and precisely measured at a given point in time and for a particular population. Independent variable - the concept which the researcher believes is leading to or causing a particular outcome, or dependent variable. The researcher may have very strong reasons for designating one variable as independent, or the designation may be arbitrary. Dependent variable - the concept which the researcher believes is being affected or caused by a particular factor, or independent variable. The term “variable” is used in place of “concept” for two reasons. First, we are interested in studying populations made up of individuals. Each concept that we study can take a variety of values (for example, if we measure the heights of 100 different people, the concept height measured in centimeters can take as many as 100 different values). In the jargon, the concept is said to vary across individuals. The other reason we use the term variable is to distinguish between a concept and its measurement. A researcher may be interested in studying the concept of social stigma. In order to study the concept, the researcher may propose many different ways of measuring stigma. Each of these different measurements will be associated with a specific variable, but all are derived from the same concept. 3

Systematic relationship - we discuss two types of systematic relationship between variables. The first is an association. Two variables are said to be associated, when the level of one variable increases along with the level of another. A classic example is age and height. If we were to line up all of the girls in a nursery school ages 3 to 5, and record each of their ages (in months) and heights (in centimeters), we would see something like the pattern depicted in Figure 1.

3

80 60 20

40

height in centimeters

100

age and height among girls, 3-5

40

45

50

55

60

age in months

Figure 1: Simulated data showing the relationship between age and height among girls aged 3 to 5 years. Another important type of relationship between concepts is a causal relationship. The easiest way to think about causal relationships is to imagine an experiment. The researcher wants to know whether a certain type of fertilizer helps sunflowers grow taller. The researcher places sunflower seeds from the same 3 To

generate a similar plot in R, use the following code: age