13 Collecting Statistical Data 13.1 The Population 13.2 Sampling 13.3 Random Sampling

1.1 - 1

• Polls, studies, surveys and other data collecting tools collect data from a small part of a larger group so that we can learn something about the larger group. • This is a common and important goal of statistics: Learn about a large group by examining data from some of its members. 1.1 - 2

 Data collections of observations (such as measurements, genders, survey responses)

1.1 - 3

 Statistics is the science of planning studies and experiments, obtaining data, and then organizing, summarizing, presenting, analyzing, interpreting, and drawing conclusions based on the data

1.1 - 4

 Population the complete collection of all individuals (scores, people, measurements, and so on) to be studied; the collection is complete in the sense that it includes all of the individuals to be studied

1.1 - 5



Census Collection of data from every member of a population



Sample Subcollection of members selected from a population

1.1 - 6

A Survey •

• •

The practical alternative to a census is to collect data only from some members of the population and use that data to draw conclusions and make inferences about the entire population. Statisticians call this approach a survey (or a poll when the data collection is done by asking questions). The subgroup chosen to provide the data is called the sample, and the act of selecting a sample is called sampling.

1.1 - 7

A Survey •

•

The first important step in a survey is to distinguish the population for which the survey applies (the target population) and the actual subset of the population from which the sample will be drawn, called the sampling frame. The ideal scenario is when the sampling frame is the same as the target population–that would mean that every member of the target population is a candidate for the sample. When this is impossible (or impractical), an appropriate sampling frame must be chosen.

1.1 - 8

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

The U.S. presidential election of 1936 pitted Alfred Landon, the Republican governor of Kansas, against the incumbent Democratic President, Franklin D. Roosevelt. At the time of the election, the nation had not yet emerged from the Great Depression, and economic issues such as unemployment and government spending were the dominant themes of the campaign. 1.1 - 9

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

The Literary Digest had used polls to accurately predict the results of every presidential election since 1916, and their 1936 poll was the largest and most ambitious poll ever. The sampling frame for the Literary Digest poll consisted of an enormous list of names that included: 1.1 - 10

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

(1) every person listed in a telephone directory anywhere in the United States, (2) every person on a magazine subscription list, and (3) every person listed on the roster of a club or professional association.

1.1 - 11

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

From this sampling frame a list of about 10 million names was created, and every name on this list was mailed a mock ballot and asked to mark it and return it to the magazine.

1.1 - 12

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

• Based on the poll results, the Literary Digest predicted a landslide victory for Landon with 57% of the vote, against Roosevelt’s 43%. • the election turned out to be a landslide victory for Roosevelt with 62% of the vote, against 38% for Landon.

1.1 - 13

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

• The difference between the poll’s prediction and the actual election results was 19%, the largest error ever in a major public opinion poll.

1.1 - 14

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

• For the same election, a young pollster named George Gallup was able to predict accurately a victory for Roosevelt using a sample of “only” 50,000 people. • What went wrong with the Literary Digest poll and why was Gallup able to do so much better?

1.1 - 15

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

• The first thing seriously wrong with the Literary Digest poll was the sampling frame, consisting of names taken from telephone directories, lists of magazine subscribers, rosters of club members, and so on. Telephones in 1936 were something of a luxury, and magazine subscriptions and club memberships even more so, at a time when 9 million people were unemployed. 1.1 - 16

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

• When it came to economic status the Literary Digest sample was far from being a representative cross section of the voters. This was a critical problem, because voters often vote on economic issues, and given the economic conditions of the time, this was especially true in 1936.

1.1 - 17

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

• When the choice of the sample has a built-in tendency (whether intentional or not) to exclude a particular group or characteristic within the population, we say that a survey suffers from selection bias. • Selection bias must be avoided, but it is not always easy to detect it ahead of time. Even the most scrupulous attempts to eliminate selection bias can fall short. 1.1 - 18

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

• The second serious problem with the Literary Digest poll was the issue of nonresponse bias. • In a typical survey it is understood that not every individual is willing to respond to the survey request (and in a democracy we cannot force them to do so).

1.1 - 19

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

• Those individuals who do not respond to the survey request are called nonrespondents, and those who do are called respondents. • The percentage of respondents out of the total sample is called the response rate.

1.1 - 20

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

• For the Literary Digest poll, out of a sample of 10 million people who were mailed a mock ballot only about 2.4 million mailed a ballot back, resulting in a 24% response rate. • When the response rate to a survey is low, the survey is said to suffer from nonresponse bias.

1.1 - 21

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

• One of the significant problems with the Literary Digest poll was that the poll was conducted by mail. This approach is the most likely to magnify nonresponse bias, because people often consider a mailed questionnaire just another form of junk mail.

1.1 - 22

CASE STUDY 2

THE 1936 LITERARY DIGEST POLL

The Literary Digest story has two morals: (1) You’ll do better with a well-chosen small sample than with a badly chosen large one, and (2) watch out for selection bias and nonresponse bias.

1.1 - 23

Examples •

Page 516, problems 17,18,19 (Solutions on following slides)

NOTE:

students should omit problem 28 from homework exercises

1.1 - 24

Examples Solutions 17(a) the citizens of Cleansburg

17(b) the sampling frame is limited to that part of the target population that passes by a city street corner between 4:00 pm and 6:00 pm

1.1 - 25

Examples Solutions 18(a) 475

18(b) yes, this survey is subject to nonresponse bias. The response rate is

475 0.266 1313 475

1.1 - 26

Examples Solutions 19(a) the choice of street corner could make a difference in responses collected

19(b) interviewer D. We are assuming that people who live or work downtown are more likely to answer yes than people in other parts of town.

1.1 - 27

Examples Solutions 19(c) yes. People on street between 4 pm and 6 pm are not representative of the population at large. Also, the five street corners were chosen by the interviewers and the passers-by are unlikely to represent a cross-section of the target population. 19(d) omit

1.1 - 28

Convenience Sampling •

•

One commonly used short-cut in sampling is known as convenience sampling. In convenience sampling the selection of which individuals are in the sample is dictated by what is easiest or cheapest for the data collector, never mind trying to get a representative sample. A classic example of convenience sampling is when interviewers set up at a fixed location such as a mall or outside a supermarket and ask passersby to be part of a public opinion poll.

1.1 - 29

Convenience Sampling •

•

A different type of convenience sampling occurs when the sample is based on self-selection–the sample consists of those individuals who volunteer to be in it. Self-selection is the reason why many Area Code 800 polls are not to be trusted. Convenience sampling is not always bad–at times there is no other choice or the alternatives are so expensive that they have to be ruled out.

1.1 - 30

Quota Sampling •

•

Quota sampling is a systematic effort to force the sample to be representative of a given population through the use of quotas–the sample should have so many women, so many men, so many blacks, so many whites, so many people living in urban areas, so many people living in rural areas, and so on. The proportions in each category in the sample should be the same as those in the population. If we can assume that every important characteristic of the population is taken into account when the quotas are set up, it is reasonable to expect that the sample will be representative of the population and produce reliable data.

1.1 - 31

Random Sampling •

•

The best alternative to human selection is to let the laws of chance determine the selection of a sample. Sampling methods that use randomness as part of their de- sign are known as random sampling methods, and any sample obtained through random sampling is called a random sample (or a probability sample).

1.1 - 32

Simple Random Sampling •

•

The most basic form of random sampling is called simple random sampling. It is based on the same principle a lottery is. Any set of numbers of a given size has an equal chance of being chosen as any other set of numbers of that size. In theory, simple random sampling is easy to implement. We put the name of each individual in the population in “a hat,” mix the names well, and then draw as many names as we need for our sample. Of course “a hat” is just a metaphor.

1.1 - 33

Simple Random Sampling •

•

These days, the “hat” is a computer database containing a list of members of the population. A computer program then randomly selects the names. This is a fine idea for small, compact populations, but a hopeless one when it comes to national surveys and public opinion polls. For most public opinion polls–especially those done on a regular basis” the time and money needed to do this are simply not available.

1.1 - 34

13 Collecting Statistical Data 13.1 The Population 13.2 Sampling 13.3 Random Sampling 13.4 Sampling: Terminology and Key Concepts 13.5 The Capture-Recapture Method 13.6 Clinical Studies

1.1 - 35

Survey In a survey, we use a subset of the population, called a sample, as the source of our information, and from this sample, we try to generalize and draw conclusions about the entire population.

1.1 - 36

 Parameter a numerical measurement describing some characteristic of a

population.

population parameter

1.1 - 37

 Statistic a numerical measurement describing some characteristic of a sample.

sample statistic 1.1 - 38

Sampling Error • •

We will use the term sampling error to describe the difference between a parameter and a statistic used to estimate that parameter. In other words, the sampling error measures how much the data from a survey differs from the data that would have been obtained if a census had been used.

1.1 - 39

Sampling Error •

Sampling error can be attributed to two factors: chance error and sampling bias.

1.1 - 40

Chance Error • •

Chance error is the result of the basic fact that a sample can only give us approximate information about the population. Different samples are likely to produce different statistics for the same population, even when the samples are chosen in exactly the same way–a phenomenon known as sampling variability.

1.1 - 41

Chance Error •

While sampling variability, and thus chance error, are unavoidable, with careful selection of the sample and the right choice of sample size they can be kept to a minimum.

1.1 - 42

Sampling Bias • •

Sample bias is the result of choosing a bad sample and is a much more serious problem than chance error. Getting a sample that is representative of the entire population can be very difficult and can be affected by many subtle factors.

1.1 - 43

Sampling Bias •

As opposed to chance error, sample bias can be eliminated by using proper methods of sample selection.

1.1 - 44

Sampling Proportion • • •

The size of the sample is denoted by the letter n The size of the population is denoted by the letter N The ratio n/N is called the sampling proportion.

1.1 - 45

Examples •

Page 515, problems 5,6,7,8

1.1 - 46

Examples 5(a) 680/8325 5(b) 45%

1.1 - 47

Examples 6(a) the registered voters in Cleansburg 6(b) the 680 registered voters polled by telephone 6(c) simple random sampling

1.1 - 48

Examples 7. Smith 3%, Jones 3%, and Brown 0%

1.1 - 49

Examples 8. Chance, the sample was chosen randomly to eliminate selection bias and there was a 100% response rate to eliminate non-response bias.

1.1 - 50

13 Collecting Statistical Data 13.1 The Population 13.2 Sampling 13.3 Random Sampling 13.4 Sampling: Terminology and Key Concepts 13.5 The Capture-Recapture Method 13.6 Clinical Studies

1.1 - 51

Finding the N-value • • •

Finding the exact N-value of a large and elusive population can be extremely difficult and sometimes impossible. In many cases, a good estimate is all we really need, and such estimates are possible through sampling methods. The simplest sampling method for estimating the N-value of a population is called the capturerecapture method.

1.1 - 52

THE CAPTURERECAPTURE METHOD ■ Step 1. Capture (sample): Capture (choose) a sample of size n1, tag (mark, identify) the animals (objects, people), and release them back into the general population.

1.1 - 53

THE CAPTURERECAPTURE METHOD ■ Step 2. Recapture (resample): After a certain period of time, capture a new sample of size n2 and take an exact head count of the tagged individuals (i.e., those that were also in the first sample). Call this number k.

1.1 - 54

THE CAPTURERECAPTURE METHOD ■ Step 3. Estimate: The N-value of the population can be estimated to be approximately (n1• n2)/k.

1.1 - 55

Capture-Recapture Method The capture-recapture method is based on the assumption that both the captured and recaptured samples are representative of the entire population.

1.1 - 56

Capture-Recapture Method •

•

Under these assumptions, the proportion of tagged individuals in the recaptured sample is approximately equal to the proportion of the tagged individuals in the population. In other words, the ratio k/n2 is approximately equal to the ratio n1/N. From this we can solve for N and get N ≈ (n1• n2)/k

1.1 - 57

Example 13.6 Small Fish in a Big Pond A large pond is stocked with catfish. As part of a research project we need to estimate the number of catfish in the pond. An actual head count is out of the question (short of draining the pond), so our best bet is the capture-recapture method.

1.1 - 58

Example 13.6 Small Fish in a Big Pond Step 1. For our first sample we capture a predetermined number n1 of catfish, say n1 = 200. The fish are tagged and released unharmed back in the pond.

1.1 - 59

Example 13.6 Small Fish in a Big Pond Step 2. After giving enough time for the released fish to mingle and disperse throughout the pond, we capture a second sample of n2 catfish. While n2 does not have to equal n1, it is a good idea for the two samples to be of approximately the same order of magnitude. Let’s say that n2 = 150. Of the 150 catfish in the second sample, 21 have tags (were part of the original sample).

1.1 - 60

Example 13.6 Small Fish in a Big Pond Assuming the second sample is representative of the catfish population in the pond, the ratio of tagged fish in the second sample (21/150) is approximately the same as the ratio of tagged fish in the pond (200/N). This gives the approximate proportion 21/150 ≈ 200/N which in turn gives N ≈ 200 150/21 ≈ 1428.57

1.1 - 61

Example 13.6 Small Fish in a Big Pond Obviously, the value N = 1428.57 cannot be taken literally, since N must be a whole number. Besides, even in the best of cases, the computation is only an estimate. A sensible conclusion is that there are approximately N = 1400 catfish in the pond.

1.1 - 62