STAT511 — Spring 2014

Lecture Notes

1

Chapter 4 February 10, 2014

Continuous Random Variables & Probability Distributions 4.1

Continuous Random Variables

Chapter Overview • Continuous random variables • Probability density function (pdf) – Definition and interpretation • Cumulative distribution function (cdf) – Definition and interpretation – Relationship between cdf and pdf • Expectation, variance and percentile for continuous rv • Some continuous distributions – Uniform and exponential – Normal ∗ Pdf ∗ Using normal table • Other continuous distributions − Gamma, Chi-squared, Weibull, Lognormal, Beta etc.

Continuous rv and the Probability Density Function

• Continuous random variables – Definitions – Examples • Probability density functions (pdf) – Definitions – Interpretations – Examples Continuous rv Definition 1. A random variable X is said to be continuous if its set of possible value includes an entire interval of numbers on the real line. • Example 4.1.1 (Example 4.1 in textbook) Make depth measurements at a randomly selected location in a specific lake. Let X = the depth at this location. X can be any value between 0 and maximum depth M . • Example 4.1.2 (Example 4.2 in textbook) A chemical compound is randomly selected and let X = the pH value. X can be any value between 0 and 14. Purdue University

Chapter4˙print.tex; Last Modified: February 10, 2014 (W. Sharabati)

STAT511 — Spring 2014

Lecture Notes

2

Probability Density Function (PDF) Definition 2. Let X be a continuous rv. Then a probability distribution or probability density function (pdf) of X is a function f (x) such that for any two numbers a and b with a ≤ b, Z b f (x)dx. P (a ≤ X ≤ b) = a

The graph of f is the density curve. i.e., the probability that X falls in [a, b] is the area under the function f (x) above this interval. f (x) must satisfies the following: 1. f (x) ≥ 0 for all x. R∞ 2. −∞ f (x)dx = 1 Probability Density Function (PDF)

Interpretations of f (x) The density function f (x) gives us an idea about the distribution of probability density instead of probability itself. 1. For any c P (X = c) = 0, i.e., the probability that X takes any specific value is 0. 2. We can only look at the probability that X falls on a specific interval. This is given by the integration of f (x). 3. For any two numbers a and b with a < b, P (a ≤ X ≤ b) = P (a < X ≤ b) = P (a ≤ X < b) Rb = P (a < X < b) = a f (x)dx. Pdf Examples • Example 4.1.3 Uniform Bus comes every 20 minutes, let X = waiting time till a bus comes. The pdf of X is: 1 f (x) = , 0 ≤ x ≤ 20. 20 What is the probability that waiting time is longer than 5 minutes? What is the probability that the waiting time is between 5 and 10 minutes? In general Given b > a, X with pdf: 1 f (x) = b−a , a ≤ x ≤ b is said to have uniform distribution.

Purdue University

Chapter4˙print.tex; Last Modified: February 10, 2014 (W. Sharabati)

STAT511 — Spring 2014

Lecture Notes

3

• Example 4.1.4 Exponential Let X = the life span of some bacteria (in hours). X is a continuous rv, the pdf is give as: f (x) = 2e−2x , x ≥ 0 What is the probability that the bacteria lives over 2 hours? What is the probability that the bacteria dies within an hour? In general given λ > 0, X with pdf f (x) = λe−λx , x ≥ 0 is an rv with exponential distribution.

4.2

Cumulative Distribution Functions and Expected Values

Cumulative Distribution Function, Expectation, Variance and Percentile • Cumulative distribution function (cdf) for continuous rv – Definition and interpretation – Relationship between pdf and cdf – Examples • Expectation and variance of continuous rv – Definition – Examples • Percentile – Definition and interpretation – Examples Cumulative Distribution Function (CDF) Definition 3. The cumulative distribution function F (x) for a continuous rv X is defined for every number x by: Z x F (x) = P (X ≤ x) = f (y)dy −∞

i.e., F (x) is the area under f (x) to the left of x. We have: 1. 0 ≤ F (x) ≤ 1 2. F (x) is non-decreasing. F (x) and f (x)

From the definition of cdf, we can easily derive: Rx • P (X ≤ x) = F (x) = −∞ f (y)dy • f (x) = F 0 (x), for which the derivative F 0 (x) exists. Rb • For a < b, P (a < X < b) = a f (x)dx = F (b) − F (a) R∞ • P (X > a) = a f (x)dx = 1 − F (a) Purdue University

Chapter4˙print.tex; Last Modified: February 10, 2014 (W. Sharabati)

STAT511 — Spring 2014

Lecture Notes

4

Finding F (x) and use F (x) to Compute Probabilities • Example 4.2.1 Uniform cdf Find the cdf F (x) for the uniform distribution:  f (x) =

1 10 ,

0,

2 ≤ x ≤ 12; otherwise.

Sketch F (x). What is P (x < 6)? What is P (x > 3)? In general, for uniform

 f (x) =

1 b−a ,

0,

a ≤ x ≤ b; otherwise.

The cdf is given by: F (x) =

  0,

x−a b−a ,



1,

x < a; a ≤ x < b; x ≥ b.

Example Continued • Example 4.2.2 Exponential cdf Find the cdf F (x) for the exponential distribution f (x) = λe−λx , λ > 0, x ≥ 0. Sketch F (x). What is P (X > a)? What is P (a < X < b)? The general form of an exponential cdf is:

 F (x) =

0, x < 0; 1 − e−λx , x ≥ 0.

Expectation of Continuous rv Definition 4 (Expectation). The expectation or mean value of a continuous rv X with pdf f (x) is defined as: Z ∞ E(X) = µX = x · f (x)dx −∞

Expectation for continuous rv is an integration instead of a summation, it is a measure of the center of the distribution. Properties of Expectation for Continuous rv

1. E(aX + b) = aE(x) + b 2. E(a1 X1 + a2 X2 + ... + an Xn ) = a1 E(X1 ) + a2 E(X2 ) + ... + an E(Xn ) 3. Expectation of function of X: if h(X) is any function of X, expectation of h(X) is: Z ∞ E[h(X)] = µh(X) = h(x) · f (x)dx −∞

Purdue University

Chapter4˙print.tex; Last Modified: February 10, 2014 (W. Sharabati)

STAT511 — Spring 2014

5

Lecture Notes

Examples of Expectations • Example 4.2.3 Uniform expectation Find the expectation of the uniform rv with pdf:  1 b−a , a ≤ x ≤ b; f (x) = 0, otherwise.

Answer: E(X) =

a+b 2

• Example 4.2.4 Exponential expectation Find the expectation of the exponential rv with parameter λ.

Answer: E(X) =

1 λ

Examples Continued... • Example 4.2.5 Find E(X 2 ) for uniform distribution with parameters a, b.

E(X 2 ) =

a2 + ab + b2 3

• Example 4.2.6 Find E(X 2 ) for exponential distribution with parameter λ.

E(X 2 ) =

2 λ2

Variance of Continuous rv Definition 5 (Variance). The variance of a continuous rv X with pdf f (x) and expectation E(X) is: Z ∞ V ar(x) = (x − E(X))2 · f (x)dx = E[(X − E(X))2 ] −∞

Standard deviation of X is:

p

V ar(X)

Variance of continuous rv is an integration instead of a summation, it is a measure of the spreadness of the distribution. Properties of Variance:

1. V ar(aX + b) = a2 V ar(X) 2. V ar(X) = E(X 2 ) − (E(X))2 = E(X 2 ) − µ2X

Purdue University

Chapter4˙print.tex; Last Modified: February 10, 2014 (W. Sharabati)

STAT511 — Spring 2014

6

Lecture Notes

Examples of Variances • Example 4.2.7 Variance of uniform Find the variance of uniform:  1 b−a , a ≤ x ≤ b; f (x) = 0, otherwise.

V ar(X) =

(b − a)2 12

• Example 4.2.8 Variance of exponential Find the variance of exponential with parameter λ. V ar(X) =

1 λ2

Percentiles of a Continuous Distribution Definition 6. Let p be a number between 0 and 1. The (100p)th percentile of the distribution of a continuous rv X, denoted η(p), is defined by: Z η(p) p = F (η(p)) = f (y)dy −∞

• Example 4.2.9 (Example 4.8 in textbook) Find the 80th percentile of the pdf given below:  3 2 2 (1 − x ), 0 ≤ x ≤ 1; f (x) = 0, otherwise. Median of a Continuous rv: 50th Percentile Definition 7. The median of a continuous distribution (denoted µ ˜, is the 50th percentile. That is: Z µ˜ 0.5 = F (˜ µ) = f (y)dy −∞

i.e., median divides the pdf into two halves with equal area. • Example 4.2.9 Find the median of the pdf.

5

The Normal Distribution

Normal Distribution • Normal pdf • Standard Normal, pdf and cdf • Normal table • zα notation • Non-standard Normal • Examples Purdue University

Chapter4˙print.tex; Last Modified: February 10, 2014 (W. Sharabati)

STAT511 — Spring 2014

7

Lecture Notes

Normal Distributions Definition 8. A continuous rv X is said to have a normal distribution with parameters µ and σ, where −∞ < µ < ∞ and σ > 0, if the pdf of X is (x−µ)2 1 f (x) = √ e− 2σ2 , −∞ < x < ∞. σ 2π

• E(X) = µ. • V ar(X) = σ 2 and thus std dev= σ. Standard Normal Distribution Definition 9. The normal distribution with parameter values µ = 0 and σ = 1 is called a standard normal distribution. The standard normal rv is denoted by Z. pdf is: z2 1 f (z) = √ e− 2 , −∞ < z < ∞ 2π

The cdf, denoted by Φ(z) (instead of F (z)) is: Z

z

Φ(z) = P (Z ≤ z) =

f (y)dy −∞

• The standard normal density curve is called z curve. z curve is bell shaped, symmetric wrt y axis. • Φ(z) gives the area under the normal density curve from −∞ to the number z. Standard Normal Distribution

Standard Normal Table There is no closed form for Φ(z), so standard normal cdf values have been tabulated using numeric methods. Example 4.3.1 Let Z be a standard normal rv, find the following using the standard normal table: 1. P (Z ≤ 0.85) P (Z ≤ 0.85) = Φ(0.85) 2. P (Z > 1.32) P (Z > 1.32) = 1 − P (Z < 1.32) = Φ(1.32) 3. P (−2.1 < Z < 1.78) P (−2.1 < Z < 1.78) = P (Z < 1.78) − P (Z < −2.1) = Φ(1.78) − Φ(−2.1) Purdue University

Chapter4˙print.tex; Last Modified: February 10, 2014 (W. Sharabati)

STAT511 — Spring 2014

Lecture Notes

8

Another Example Example 4.3.2 Let Z be a standard normal rv, find z when: 1. P (Z < z) = 0.9278 P (Z < z) = Φ(z) = 0.9278, look for 0.9278 in table, and find z accordingly. 2. P (|Z| < z) = 0.8132 P (−z < Z < z) = P (−z < Z < 0) + P (0 < Z < z) = 2P (0 < Z < z) = 2(Φ(z) − Φ(0)) = 2(Φ(z) − 21 ) = 2Φ(z) − 1 = 0.8132, thus Φ(z) = 0.9066 zα N otation Later when we discuss inferential statistics, we will need values on the measurement axis that capture small tail areas under the normal curve, this is denoted zα : zα denote the value on the measurement axis for which α of the area under the z curve lies to the right of zα . • 1 − α is the area lies to the left of zα under the z curve. i.e., zα is the 100(1 − α)th percentile of the standard normal dist. • z curve is symmetric wrt y axis, so area to the left of −zα is also α. • z is usually referred to as z critical values. • Example 4.3.3 What is z0.05 ? It is the ?-th percentile? Non-standard Normal Distributions Proposition. When X ∼ N (µ, σ 2 ), then Z=

X −µ σ

has a standard normal distribution, thus       b−µ a−µ b−µ a−µ P (a ≤ X ≤ b) = P ≤Z≤ =Φ −Φ σ σ σ σ     a−µ b−µ , P (X ≥ b) = 1 − Φ P (X ≤ a) = Φ σ σ Non-standard normal curve: 1. Roughly 68% of the values are within σ of the mean. 2. Roughly 95% of the values are within 2σ of the mean. 3. Roughly 99.7% of the values are within 3σ of the mean.

Purdue University

Chapter4˙print.tex; Last Modified: February 10, 2014 (W. Sharabati)

STAT511 — Spring 2014

Lecture Notes

9

Empirical Rule

Example of Non-standard Normal Example 4.3.4 Reaction time for an in-traffic response to a brake signal from standard brake lights can be modelled with a normal with mean 1.25 sec and std dev 0.46 sec. What is the probability that reaction time is between 1.00 and 1.75 sec? X ∼ N (1.25, 0.462 ), and want to find P (1.00 ≤ X ≤ 1.75). Exercise A particular rash shown up at an elementary school. It has been determined that the length of time that the rash will last is normally distributed with µ = 6 days, and σ = 1.5 days. Find the probability that for a student selected at random, the rash will last for between 3.75 and 9 days. Using Normal Table and Some More Distributions

• Using Normal table – Many Examples • The Normal approximation to the Binomial • Several continuous distributions – Gamma – χ2 – More in section 4.5 (read if interested) Using Normal Table • Example 4.4.1: HW4 4.27bcde • Example 4.4.2: HW4 4.29 • Example 4.4.3: HW4 4.30f • Example 4.4.4: HW4 4.35ab

Purdue University

Chapter4˙print.tex; Last Modified: February 10, 2014 (W. Sharabati)

STAT511 — Spring 2014

Lecture Notes

10

Percentiles of an Arbitrary Normal The (100p)th percentile of a normal distribution with mean µ and standard deviation σ can be easily transformed from the percentile of a standard normal. (100p)th percentile of N (µ, σ 2 ) = µ + (100p)th percenile of N (0, 1) · σ Example 4.4.5 What is the 95th percentile of N (µ = 2, σ = 4.5)? Normal Approximation to Binomial Let X be a binomial rv based on n trials, each with probability of success p. Check the binomial pmf (histogram)p is not too skewed, X has approximately a normal distribution, with µ = np and σ = np(1 − p).. ! x + 0.5 − np P (X ≤ x) = Φ p np(1 − p) In practice, the approximation is adequate provided that both np ≥ 10 and n(1−p) ≥ 10 Normal Approximation Example Example 4.4.6 72% women marry before 35 years old. For 500 women, what is the probability that at least 375 get married before 35? µ = np = 500 · 72 = 360, p p σ = np(1 − p) = 500 · 0.72(1 − 0.72) ≈ 10 We may use Normal approximation:   375.5 − 360 P (X ≤ 375) ≈ Φ = Φ(1.55) = 0.9394 10

5.1

Gamma Distribution and Its Relatives

Gamma Distribution • Gamma Distribution: X has a gamma if the pdf is: ( 1 α−1 e−x/β , x ≥ 0; β α Γ(α) x f (x) = 0, otherwise. Here Γ(α) is the gamma function, and α > 0, β > 0.. • Gamma function:

Z

∞

Γ(α) =

xα−1 e−x dx

0

• Expectation and variance of gamma dist E(X) = αβ V ar(X) = αβ 2 • Exponential λ is gamma with α = 1, β = λ1 . Purdue University

Chapter4˙print.tex; Last Modified: February 10, 2014 (W. Sharabati)

STAT511 — Spring 2014

Lecture Notes

11

Chi-square Distribution • For a gamma distribution, let α = ν/2, β = 2,we end up with a χ2 dist: ( 1 x(ν/2−1) e−x/2 , x ≥ 0; 2ν/2 Γ(ν/2) f (x) = 0, otherwise. • ν here is called the degree of freedom of a χ2 distribution. Applications of the Exponential Distribution Suppose that the number of events occurring in any time interval of length t has a Poisson distribution with parameter and that the numbers of occurrences in nonoverlapping intervals are independent of one another. Then the distribution of elapsed time between the occurrences of two successive events is exponential with parameter λ = α.

Purdue University

Chapter4˙print.tex; Last Modified: February 10, 2014 (W. Sharabati)