Biostatistics Xinhai Li

Hypothesis Testing • What is hypothesis testing • Standard procedures • Examples

1

Definition

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

What the data refer to? • If we flip a coin 100 times, and 45 come up heads this could easily occur by chance. There is not sufficient evidence to suggest that the coin is unfair.

• If we flip a coin 100 times, and 25 come up heads this would be an rare event if the coin was fair. The low probability is evidence that the coin may not be fair.

The coin is definitely unfair? Although rare, 25 heads is still possible by chance from a fair coin.

2

Definition

Biostatistics Xinhai Li

Lecture 3. Hypothesis testing (1/2)

Use probability density

0.06 0.04

qbinom(0.05, 100, .5) 42

0.02

pbinom(25, 100, .5) 2.82e-7

0.00

Probability density

0.08

hist(rbinom(100000, 100, .5), freq = F, main = “”, xlab = 'Number of head', ylab = 'Probability density')

30

40

50 Number of head

60

70 3

Definition

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Hypothesis testing • A hypothesis is a claim or statement about a property of a population (e.g. the mean or a proportion of the population) • A hypothesis test (or test of significance) is a standard procedure for testing a claim or statement about a property of a population. It is extremely important to realize that we are not making definitive conclusions. We are giving probabilistic conclusions. We are either concluding that the results we get are likely due to chance, or unlikely. (Zar 1999) 4

History

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Origins •

Hypothesis testing is the product of Ronald Fisher, Jerzy Neyman, Karl Pearson and Egon Pearson.

•

Fisher emphasized rigorous experimental design and methods to extract a result from few samples assuming Gaussian distributions.

•

Neyman and E. Pearson emphasized mathematical rigor and methods to obtain more results from many samples and a wider range of distributions (Neyman and Pearson 1933).

•

Modern hypothesis testing is a hybrid of the Fisher vs. Neyman/Pearson formulation, methods and terminology developed in the early 20th century.

Neyman, Jerzy; Pearson, Egon S. (1933). “On the problem of the most efficient tests of statistical hypotheses". Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 231: 289–337.

5

Parallel approaches

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Other options rather than frequentist hypothesis testing • Confidence interval (CI) is a particular kind of interval estimate of a population parameter and is used to indicate the reliability of an estimate • The Bayesian approach to hypothesis testing is to base rejection of the hypothesis on the posterior probability • Other approaches to reaching a decision based on data are available via decision theory and optimal decisions 6

R script

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

t-test using R # t-test daily energy intake in kJ for 11 women (Altman, 1991, p. 183) daily.intake = c(5260,5470,5640,6180,6390,6515, 6805,7515,7515,8230,8770) mean(daily.intake) One Sample t-test sd(daily.intake) data: daily.intake quantile(daily.intake) t = -2.8208, df = 10, p-value = 0.01814 t.test(daily.intake, mu = 7725) # Nonparametric wilcox.test(daily.intake, mu = 7725) # Two samlpes x1 = rnorm(300, 0, 1) x2 = sample(0:100, 300, rep = T) t.test(x1, x2) # Check normality plot(x1); hist(x1); qqnorm(x1) shapiro.test(x1) D.G. Altman (1991), Practical Statistics for Medical Research, Chapman & Hall.

alternative hypothesis: true mean is not equal to 7725 95 percent confidence interval: 5986.348 7520.925 sample estimates: mean of x 6753.636 Wilcoxon signed rank test data: daily.intake V = 8, p-value = 0.0293 alternative hypothesis: true location is not equal to 7725

7

Example

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Example sample mean - population mean

• Black Americans were paid less? (Equalizing opportunity in the labor market?)

8

Example

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Research proposition American blacks still have an earnings disadvantage •

Our operational hypothesis: mean income for blacks is lower than for the population as a whole

•

We draw a random sample of 166 blacks and calculate their mean income for 1990 ($18037)

•

Based on census information, we know the mean income for all Americans in 1990 ($23766)

•

By comparing the means, we are asking whether it is reasonable to consider the sample of black Americans a random sample that is representative of the population of full-time workers in the U.S. 9

Example

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Sample and population Black

All

x $ 18 ,037 n 166

$ 23 ,766 $ 16 ,101

The sample mean is $5,729 below the population mean

10

Example

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

What are the possibilities? • The average earnings of the black population are about the same as the national average and this sample happens to show a really low mean difference = sampling variability • The average earnings of the black population are indeed lower than average earnings nationally difference = real

11

Example

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

How do we decide which explanation makes more sense? • The traditional way: hypothesis tests – Five steps

12

Example

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

One sample hypothesis test • A random sample: 166 blacks earned an average of $18,037 in 1990. • Population: The national average was $23,766 with a standard deviation of $16,101. Do blacks make significantly less than the population?

13

Standard procedure

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Step one: checking assumptions Hypothesis testing involves several assumptions that must be met for the results of the test to be valid For the one sample hypothesis test, we assume: • random sampling • the level of measurement is interval-ratio • the sampling distribution is normal 14

Standard procedure

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Step two: stating hypotheses • Null hypothesis (H0): a statement of ‘no difference’ • Alternative hypothesis (Ha): a statement that reflects the research question • Both are expressed in terms of population parameters 15

Standard procedure

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Step two: stating hypotheses

H o : blacks $ 23 , 766 H 1 : blacks $ 23 , 766 16

Standard procedure

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Step three: select the sampling distribution and establish the critical region (1/3) • Select sampling distribution: use Z statistic and normal distribution when the sample is large • Zcritical: the score associated with a particular level and marking the beginning of the critical region • Critical region: area under the sampling distribution that includes all unlikely sample results z

17

Standard procedure

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Sampling distribution of x bar assuming H0 is true (2/3) • Sampling Distribution of X

z

$ 23 , 766 $ 16 ,101

• P-value: the “chance” of getting the observed sample mean further away from the hypothesized population mean? P-value small: reject Ho P-value large: fail to reject Ho 18

Standard procedure

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Step three: selecting sampling distribution and establish the critical region (3/3) • Confidence level: = .05 • Zcritical = - 1.65

-1.65 19

Standard procedure

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Step four: compute the test statistic

Z

x

N 18,037 23,766 Z 16,101 166 Z 4.58 20

Standard procedure

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Step five: make a decision • Plot the test statistic on the sampling distribution • If the test statistic is in the critical region, our decision is: reject the null (“statistically significant”) • If the test statistic is not in the critical region, our decision is: fail to reject the null

-4.58 -1.65

0

21

Standard procedure

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Step five: make a decision • We reject the null hypothesis. • We are 95% confident that blacks have significantly lower annual earnings than the population as a whole.

22

Standard procedure

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

Formal hypothesis testing 1. Making/checking assumptions 2. Convert your claim into a symbolic null and alternative hypothesis 3. Select the sampling distribution and establish the critical region 4. Calculate a test statistic 5. Compare the test statistic to critical values OR a probability, write a conclusion 23

Assumptions

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

For the one sample hypothesis test, we assume • Random sampling • The level of measurement is interval-ratio • The sampling distribution is normal

24

Assumptions

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

When can you assume the shape of the sampling distribution is normal? • When the population distribution is normal • When you check sample mean (Based on the Central Limit Theorem) – shapiro.test(x) – qqnorm(x)

p>0.05 straight line

25

Proof of Central Limit Theorem

Biostatistics Xinhai Li

Lecture 3. Hypothesis testing (1/2)

Central limit theorem • Sampling distribution of means becomes normal as N increases, regardless of shape of original distribution. 40 20

y 0 1.96 pnorm(z) t.test(x, mu=0, var.equal = T)

n

or X < 0 1.96

n 33

Test statistic

Lecture 3. Hypothesis testing (1/2)

Biostatistics Xinhai Li

T test statistic • Test continuous outcome • Unknown variance • Under H0

X 0 ~ t( n 1) s/ n

• Critical values: depend on degree of freedom, from computer or tables in statistics books • t-distribution approximately normal for degrees of freedom (df) >30 t.test(x, mu=0, var.equal = F) 34

Biostatistics Xinhai Li

Lecture 3. Hypothesis testing (1/2)

Test statistic

Comparing the student-t distribution to the normal distribution 0.5 Student t with 4 degrees of freedom Student t with 15 degrees of freedom Normal distribution

0.45 0.4 0.35

Probability

0.3 0.25 0.2 0.15 0.1 0.05 0 -4

-3

-2

-1

0 Standard Deviations

1

2

3

4

35

Biostatistics Xinhai Li

Lecture 3. Hypothesis testing (1/2)

0.1

P

Plot

0.2

0.3

0.4

R script

# Plot a normal distribution and t distributions 0.0

x