Probability Models Weather example: Venn diagram for all combinations of 3 binary (true/false) events.
Ø
Ø
Ø
● ● ●
Raining or not Sunny or not Hot or not
Sample spaces: When a random event happens, what is the set of all possible outcomes? May be discrete or continuous. Conditioning: Suppose I observe some data. How does my probability model change? Independence: Is there any relationship between pairs of variables in my model? Would data provide knowledge?
Defining a Probabilistic Model flip a coin, roll a die, receive an email, take a picture, …
The Axioms of Probability event A
Ø
Ø
Ø
Ø
event B
Valid probabilities defined by any function mapping subsets of to [0,1] that satisfies these axioms (assumptions) The nonnegativity and additivity axioms are fundamental to our intuitive understanding of probability and uncertainty Unit normalization is just a convention, and other options could also be used (e.g., probability between 0% and 100%) The additivity axiom guarantees that the probabilities of any finite set of disjoint events are additive (induction)
Conditional Probability
Bayes’ Rule
Independence of Two Events
Computing a PMF
Several Random Variables
y z
May compute marginal of any subset of variables, possibly conditioned on values of any other variables.
Expected Values of Functions
Ø
Consider a non-random (deterministic) function of a random variable:
Ø
What is the expected value of random variable Y?
Ø
Correct approach #1:
Ø
Correct approach #2:
Ø
Incorrect approach:
(except in special cases)
Linearity of Expectation Ø
Consider a linear function:
Ø Ø
Example: Change of units (temperature, length, mass, currency, …)
Ø Ø
In this special case, mean of Y is the linear function applied to E[X]:
Expectation of Multiple Variables
Ø
The expectation or expected value of a function of two discrete variables:
Ø
A similar formula applies to functions of 3 or more variables
Ø
Expectations of sums of functions are sums of expectations:
Ø Ø
This is always true, whether or not X and Y are independent Specializing to linear functions, this implies that:
Variance and Standard Deviation
Ø
Ø
The variance is the expected squared deviation of a random variable from its mean (these definitions are equivalent):
By definition, the standard deviation is the square root of the variance:
Variance and Moments
Ø
The variance is the expected squared deviation of a random variable from its mean (these definitions are equivalent):
Terminology: Moments of random variables
first moment or mean of X second moment of X pth moment of X
Continuous Random Variables CDF: cumulative distribution function PDF: probability density function
Model processes or data which are encoded as real numbers: temperature, commodity price, DNA expression level, light on camera sensor, …
Continuous Random Variables
Ø
Ø
Ø
For any discrete random variable, the CDF is discontinuous and piecewise constant If the CDF is monotonically increasing and continuous, have a continuous random variable:
The probability that continuous random variable X lies in the interval (x1,x2] is then
1
0
Probability Density Function (PDF)
Ø
Ø
If the CDF is differentiable, its first derivative is called the probability density function (PDF):
1
By the fundamental theorem of calculus: 0
Ø
For any valid PDF:
0
Expectations of Continuous Variables
Ø
The expectation or expected value of a continuous random variable is:
Ø
The expected value of a function of a continuous random variable:
Ø
The variance of a continuous random variable:
Ø
Intuition: Create a discrete variable by quantizing X, and compute discrete expectation. As number of discrete values grows, sum approaches integral.
Joint Probability Distributions
Marginal Distributions
Example: Uniform Distributions
Independence
Distributions ●
Bernoulli Distribution
●
Gaussian Distribution
●
Exponential Distribution
●
Geometric Distribution
●
Uniform Distribution
●
Formulas will be provided, but you should be familiar with their properties
Things to Prepare ●
Lecture Notes
●
Homework Questions
●
(Textbook Problems)
●
(Recitation Material)
