Lessons 11,12 - Marginal Distributions and Conditional Distributions

Properties of Discrete Conditional Distributions

1. f(x | y) must be non-negative 2. sum of f(x | y) for fixed y and all x must = 1

Properties of Continuous Conditional Distributions

1. f_X|Y(x | y) >= 0 for all x *non-negative 2. integral over all x in R of f(x | y)dx = 1

Determining Independence of X and Y

For a pair of random variables to be independent, their joint support must be rectangular. This means there cannot exist any (x,y) with f_X(x) > 0 and f_Y(y) > 0 but f(x,y) = 0

Finding Discrete Conditional Distributions

Here we need to find the conditional PMF for X (# red balls) given there is 1 white ball drawn (Y=1). We find this by summing the conditional joint pmf values where y=1. This can be generalized to a formula where y is a fixed variable. In this case the conditional joint pmf is the pmf f(x,y) divided by the marginal pmf f_Y(y).

Finding Marginal PMF of X

Here we only consider the values of X for the pmf f(x)=P(X=x). We can use combinatorials and plug in values of x or we can add up the probabilities in the rows.

Finding Marginal PMFs from Joint PMFs

When (X,Y) is a discrete bivariate random vector with PMF f_XY(x,y), we can find the marginal PMFs of X and Y by summing their respective probabilities. f_X(x) = P(X=x) = Σf(x,y) for all y in R *fixing the value of x and adding up all values of y f_Y(y) = P(Y=y) = Σf(x,y) for all x in R *fixing value of y and adding up all values of y

Independent Conditional Distributions

When X and Y are independent, f(x | y) = f(x) and f(y | x) = f(y)

Independent Bivariate Random Variables X and Y

X and Y are independent iff there exist functions g(x) and h(y) where f(x,y) = g(x) • h(y) for all x,y *g(x) and h(y) could also be f_X(x) and f_Y(x)

Independent Conditional Distributions cont.

f(x,y) = f_X(x) • f_Y|X(y | x) = f_Y(y) • f_X|Y(x | y) This works even when X is discrete and Y is continuous though we must be careful when describing f(x,y) because it is neither a pdf or pmf

Conditional Probability of Discrete Joint RVs

Consider a bivariate discrete random variable (X,Y). We can think about events A = [X=x} and B=[Y=y]. Then, P(X=x | Y=y) = P(A | B) = P(A∩B) / P(B) = f_XY(x,y) / f_Y(y) = f_X|Y(x | y) We can represent the conditional probability of X | Y as the bivariate distribution of X and Y divided by the marginal distribution of Y. It is important that f_Y(y) = P(Y=y) > 0

Conditional Probability of Continuous RVs

Consider bivariate RV (X,Y) with typical joint PDF and marginal PDFs. For any y where f_Y(y) > 0, f_XY(x | y) = f_XY(x,y)/f_y(y) for all x

Bayes' Theorem for Random Variables

Extension of Bayes' to RVs. g(y | x) = g(x | y)•f_y(y) / f_X(x)

Law of Total Probability for Random Variables

Extension of the law of total prob. to RVs. If discrete, the marginal dist. f_X(x) = the sum of g(x | y)•f_Y(y) for all values of y. If continuous, the marginal dist. f_X(x) = the integral over all reals of g(x | y)•f_Y(y)dy

Finding Joint CDFs from Joint PDFs

We find this by integrating the joint PDF across both the x and y bounds where x_not and y_not are the upper bounds during the calculation. F_X(x) = P(X<=x_0) = integral of f_X(x) over the range 0 to x_0

Independent Random Variables

In the previous two examples we found that the marginal CDFs were independent. F_X(x) • F_Y(y) = F_XY(x,y) and P(X<=x, Y<=y) = P(X<=x) • P(Y<=y) equate to P(A∩B) = P(A) • P(B)

Finding Marginal PDFs and CDFs

We find the marginal PDF of X by integrating y out of the equation over the y bounds in the support. We find the marginal CDF of X by integrating the marginal PDF of X across the x bounds (0 to x_not) and replacing x_not with a proper x afterward for the sake of clarity.

Multiplication Rule for Distributions

The multiplication rule P(A∩B) = P(A)•P(B) can be extended to distributions. f(x,y) = f(x | y)•f_Y(y) = f_X(x)•f(y | x)

Finding Continuous Conditional Distributions

To find continuous conditional distribution, we need to find the joint pdf and relevant marginal pdf. Consider f(x | y). Here we are given the pdf f(x,y), but we need to find the marginal pdf f_Y(y) by integrating out the x variable. We find that f(x | y) = f(x,y) / f_Y(y)

Marginal Distributions Intro

Used to determine behavior of only one of the RVs in a joint distribution. i.e. distribution of X marginally over Y is where we only care about X • If (X,Y) is discrete, we need to find f(x)=P(X=x) from the joint PMF f(x,y) • If (X,Y) is continuous, we need to find f(x)=P(X=x) from the joint PDF f(x,y)

Finding Probabilities from Conditional PDFs

We can do this directly and find a single probability by plugging a static number into the variable that is conditioned on, or we can do it generally to create a pdf by leaving the variable as is.

Finding Marginal PDFs from Joint PDFs

We can find the marginal PDFs by integrating the joint pdf for the fixed variable across all values of the other variable. f_X(x) = integral bounded by y across R for all x in R f_Y(y) = integral bounded by x across R for all y in R