stats 67 - lecture 7 & 8 video
cumulative dsitribution function for binomial random variable X
F(x) =P(X<=x) =ΣP(X=x̄) =ΣC(n,x̄)*p^x̄*(1-p)^(n-x̄)
Properties of variance If c is a constant, then var(c) = __1__ If c is a constant, then var(cX) = __2__ If X and Y are independent random variables then var (X + Y) = __3__ As a result, if Xi's are independent random variables and Ci's are constants, then var(∑CiXi) = __4__
1 - 0 (what is the variance of 5 is like saying how much does it jump around -- which is 0) 2 - c^2 * var(X) (var(5x) = 25*var(c) -- the standard deviation then removes the power of 2) 3 - var(X) + var(Y) 4 - ∑Ci^2var(Xi)
The cumulative distribution function (cdf) of X is not just P(X = x) but __1__ We denote cdf as F(x) (where the pmf is f(x)). The cumulative distribution function is the __2__ Note that F(x) = P(X ≤ x) = 1 − P(X > x).
1 - P(X ≤ x) 2 - sum of several probability mass functions. The cumulative distribution function (cdf) of X is not just P(X = x) but P(X ≤ x). We denote cdf as F(x) (where the pmf is f(x)). The cumulative distribution function is the sum of several probability mass functions. Note that F(x) = P(X ≤ x) = 1 − P(X > x).
The expectation of X is denoted as E(X). E(X) = ∑xP(X = x) = ∑ xf(x) where ∑ lower bound is x∈Sx Can be viewed as __1__. For ease of notation, we let µ be the population expected value of X. That is to say E(X) = __2__
1 - averaging over all possible X values while weighting each possible value by its probability 2 - µ The expectation of X is denoted as E(X). E(X) = ∑xP(X = x) = ∑ xf(x) where ∑ lower bound is x∈Sx Can be viewed as averaging over all possible X values while weighting each possible value by its probability
Bernoulli distribution handles ___1___ Set Sx = {0, 1}. Let P(X = 0) = 1 − p and P(X = 1) = p, where p is some number between 0 and 1 (a probability). Then we say X follows a Bernoulli distribution with parameter p. Note that f (x) = P(X = x) = p^x (1 − p)^(1−x) for x = {0, 1}. -- important -- Because of the fact that P(X = 0) = 1 − p and P(X = 1) = p, we get E(X) = 0 ∗ (1 − p) + 1 ∗ p = p. And thus, var(X) = E(X^2 ) − (E(X))^2 = p − p^2 = p(1 − p).
1 - binary outcomes. P(X = 0) = 1 − p and P(X = 1) = p means that it can be an unfair outcome, 0 shows up with probability 1 - p and 1 shows up with probability p, so it depends on what p is p^x (1 − p)^(1−x) plug in x=1 you get p
expectation of x general formula and expectation of x as a binomial distribution
1 - general: Σxf(x) 2 - x*C(n,k)a^k*b^n-k
In general, expectations of X are noted as the __1__ of X. Thus E(X^k ) is noted as the k-th __2__ of X, for k = 1, 2, 3, 4, ....
1 - moments 2 - moment
Each value of x(x=0,1,2,...n) has C(n,x) ways of occurring. Each of these events has probability p^x(1-p)^n-x ways of occurring (x many successes and n-x nonsuccesses) We can now create the ___1___ of X using functions. f(x) = P(X=x)=C(n,x)p^x(1-p)^n-x for x=0,1,2,...,n You can verify that f(x) is a valid probability mass function because f(x) = P(X=x)=C(n,x)p^x(1-p)^n-x >= 0, f(x) is only defined for 0<=x<=n and because Σf(x)=1 which can be proven with __2__ n and p are paramters -> n is known, p is unknown
1 - probability mass function 2 - the binomial theorem, which states that (a+b)^n = ΣC(n,k)a^k*b^n-k, setting a=p and b=1-p
f (x) is the __1__ of a random variable X. The input is __2__, which is a specified value of X from its support. The output is __3__ the probability that X is equal to what we specified, x. Thus f (x) = P(X = x). Since it is a probability, 0 ≤ f (x) ≤ 1. And all the outputs must sum to 1.
1 - probability mass function 2 - x 3 - P(X = x) f (x) is the probability mass function of a random variable X. The input is x, which is a specified value of X from its support. The output is P(X = x), the probability that X is equal to what we specified, x. Thus f (x) = P(X = x). Since it is a probability, 0 ≤ f (x) ≤ 1.
binomial distribution We say X follows a binomial dsitribution with parameters n and p. X is the number of ___1___ in n many independent trials Each trial has the probability ___2___ of having a success, and so it has the probability of __3__ of not succeeding This is like counting the number of heads that occur in n many independent flips (this specific case of flipping a coin to heads has p=0.5)
1 - successes, yesses, or heads 2 - p 3 - 1-p
Can derive the expectation and variance in an easier fashion when we construct the binomial random variable as the sum of n many independent and identical Bernoulli random variables. The binomial random variable X can be seen as the __1__ That is to say, if Zi are independent and identical Bernoulli random variables with p, then X=ΣZi E(X)=E(ΣZi)= ΣE(Zi)=Σp=np And variance: var(X) = var(ΣZi) = Σvar(Zi) = Σp(1-p)=np(1-p) What is f(x) and E(x)?
1 - sum of n independent and identical Bernoulli random variables 2 - f(x) = P(X=x) = C(n,x)p^x(1-p)^n-x for x=0,1,2,....,n 3 - E(x) =np and var(X) = np(1-p)
The the variance of x is defined to be __1__ var(X) = __2__ We denote the variance of X as __3__ Since it is the expected value of a squared random variable, σ^2 > 0. σ = sqrt(var(X)) is known as the __4__
1 - the average squared deviation from the mean/expected value/average (which is E(X)). 2 - var(X) =E[(X − µ)^2 ] =E[(X − E(X))^2 ] = E(X^2)-[E(X)]^2 3 - σ^2 4 - standard deviation of X
Note with the properties of variance, it is straightforwards to compute things of the form var(5 + 4X) once we have computed var(X). var(5 + 4X)=4^2*var(X). In general, for constants a and b: ___1___
1 - var(a+bX)=b^2 * var(x).
Let h(X) be a function of X, such as h(X) = 1/X E(h(X)) __1__ h(E(X)). Thus E(1/X) != 1/E(x) As a result (remember the formula for variance depends on expectations), var(h(X)) __2__ h(var(X))
1.) !=, does not equal 2.) !=, does not equal
Can now create the probability mass function of X, f (x), to define its probability distribution. What formula would you use to find f(x) = P(X=x) for flipping a coin n times and keeping track of the heads.
C(n,x)*(1/2)^n
variance of X
E(X^2)-[E(X)]^2
Say X is a random variable that follows a binomial distribution with n and p (for example n = 100 and p = 1 / 4). What is the expectation of x and the variance of x? What is P(x=5)? P(x=55)?
E(x = np=100*1/4) var(x)=np(1-p)=100*1/4*3/4 P(x=5)=C(100,5)*p^5*(1-p)^100-5 =(100!/(5!95!))*(1/4)^5*(3/4)^95 P(X=55)=C(100,55)*p^55*(1-p)^(100-55)=(100!/(55!55!))*(1/4)^55*(3/4)^45
Say n=5 and outcome is Y (yes or 1) and N (no or 0). Elements in this sample space are the outcome of 5 trials, in order. Now say x=2. Thus x=2 and n-x=3. An element from the sample space that has x=2 is YYNNN or NYNYN. P(YYNNN): P(Y and Y and N and N and N) =P(Y)P(Y)P(N)P(N)P(N)=pp(1-p)(1-p)(1-p)=p 2 (1 − p) 3 . Also P(NYNYN): P(N and Y and N and Y and N) = ??
P(N)P(Y)P(N)P(Y)P(N) =(1-p)p(1-p)p(1-p) =p^2(1 − p)^3
Say X is a random variable that follows a binomial distribution with n and p (for example n = 100 and p = 1 4 ). Compute the probability that we see less than 99 successes out of the 100 attempts. This is to say P(X < 99) = P(X ≤ 98). There are two ways to do this.
Remember we can use the complement rule. Say event A is X < 99 and so A^c is X ≥ 99. With a binomial random variable X, then F(98) = P(X ≤ 98) = ΣP(X=x̄) =Σ(100, x̄)*(1/4)^x̄*(3/4)^n-x̄ where x̄ <= 98 = C(100, x̄)(1/4)^ x̄(3/4)^n- x̄ So we compute this for 99 pieces C(100,0)(1/4)^0(3/4)^100-0 + C(100,1)(1/4)^1(3.4)^100-97 + ....+ C(100, 97)(1/4)^97(3/4)^100-97+C(100,98)(1/4)^98(3/4)^100-98 *BUT using the complement rule*: P(X ≤ 98) = 1 − P(X > 98) = 1 − P(X ≥ 99) To compute this, will have 2 pieces to calculate. P(X ≥ 99) = P(X = 99) + P(X = 100) 1 - [C(100,99)(1/4)^99(3/4)^100-99 + C(100,100)(1/4)^100(3/4)^100-100] = 1 - C(100,99)(1/4)^99(3/4)^100-99 + C(100,100)(1/4)^100(3/4)^100-100
Now extend the flipping a coin 3 times example to flipping a coin a large number of times, say 100 times. Sx = {0, 1, 2, 3, ..., 99, 100} The sample space will contain 2^100 elements, each a 100 character long string of HHHHH..... or HTHHTHT..... For each value of x in Sx, can think of it as arranging x many heads among 100 spots The probability of each event in the sample space is (1/2)^100 Each value of x will need to determine how many of the 2^100 events in the sample space will result in that x value. Each value of x will need to compute the number of ways of selecting x many objects from a total of n = 100 of them to pick from. This is the C(n, k) = n!/k!(n-k)! formula from before. Each value of x = 0,1,2,...,100 will have the probability computes as multiplying the number of ways x can occur with (1/2)^100 Sx = {0,1,2,3,...,99,100} What is P(X=20)
Take x = 20 Would need the number of elements in S (the sample space) that have 20 H's arranged amoing the 100 spots (and the other 80 by T's) Using the formula, we get there are 100!/20!(100-20)! total ways of having 20 heads show up in 100 flips. The probability of each event (20 H's and 80 T's) is (1/2)^100 Then P(x=20) = 100!/20!(100-20)!*(1/2)^100
For example, P(10 < X ≤ 14) = P(11 ≤ X ≤ 14) means
This is to say the probability of X being less than or equal to 14 and greater than 10. P(11 ≤ X ≤ 14) = P(X = 11) + P(X = 12) + P(X = 13) + P(X = 14)
Suppose you play a game where you can either win a dollar or lose a dollar. It's as equally likely that you'll win a dollar as it is that you'll lose a dollar. Say Sx= {−1, 1} where P(X = 1) = 0.5 and P(X = −1) = 0.5 What is the variance of X? What is the standard deviation?
We know variance of X is E(X^2)-[E(X)]^2 We first find E(X) = -1 *.5 + 1*.5 = 0 E(X^2) = -1^2 *.5 + 1^2*.5 = 1 We plug into our variance formula var(X) = 1 - 0^2 = 1 The standard deviation is sqrt(1) = 1
Suppose you are in a scenario where you either lose 1,000 dollars or win 1,000 dollars. Each is equally likely. Say Sx = {−1000, 1000} where P(X = 1000) = 0.5 and P(X = −1000) = 0.5 What is the variance of X?
We know variance of X is E(X^2)-[E(X)]^2 We first find E(X) = -1000 * .5 + 1000 * .5 = 0 E(X^2) = -1000^2 * .5 + 1000^2 * .5 = 1000^2 We plug into our variance formula var(X) = 1000^2 - 0^2 = 1000^2 The standard deviation is sqrt(1000^2) = 1000
Example is rolling a die once. Our pmf table will have 1/6 for x = 1,2,3...6 and f(X=x). What is the variance of X?
We know variance of X is E(X^2)-[E(X)]^2 We first find E(X) = 1 * 1/6 + 2* 1/6 + 3 * 1/6 + 4 * 1/6 + 5 * 1/6 + 6 * 1/6 = 3.5 E(X^2) = 1^2 * 1/6 + 2^2* 1/6 + 3^2 * 1/6 + 4^2 * 1/6 + 5^2 * 1/6 + 6^2 * 1/6 = ~ 15.1 E(X^2)-[E(X)]^2 = 15.1 - 3.5^2 = ~2.91
Suppose we are using the example of flipping a coin 100 times and keeping track of when it lands on heads. Sx={0,1,2,3,...99,100} Can do this for all values of x =0,1,2,...100 What is P(X=0)?
We use the formula of (choose/combinartoric) * probability of the event C(100,0) = 1 total way of having 0 heads show up in 100 flips The probability of this event (100 tails in a row, TTTT....) is (1/2)^100. Then P(X=0) = 1*(1/2)^100
Can now create the probability mass function of X, f (x), to define its probability distribution of a binomial random variable. Counting the number of events is the same, but the probability of each event in the sample space (arrange x many success among n many trials, for x = 0, 1, 2, ..., n). Each event in the sample space will have probability involving p and (1 − p). Specifically, each event that has x many success among the n many trials is p^x * (1 − p)^n−x . This is the product of n many independent and identically distributed Bernoulli random variables. Why is each element in the sample space of the form p^x * (1 − p)^n−x?
Where x is the number of yes' (or 1's or success') and n is the number of no's. Note that all n many trials are independent, where the yes trials have probability p and the no trials have probability (1-p). To get a specific element in the sample space (n trials, a combination of yes' and no's), there are x many yes' and n-x many no's. Thus the probability of a specific element in the sample space will be p^x * (1 − p)^n−x , where the p*x are from the yes' and the (1 − p)^n−x are from the no's.
Say X is a random variable that follows a binomial distribution with n and p (for example n = 100 and p = 1/4 ). Compute the probability that we see less than 4 success's out of the 100 attempts. This is to say P(X < 4) = P(X ≤ 3) (X = 0, 1, 2, or 3)
c(100,0)*(1/4)^0*(3/4)^100-0 + c(100,1)*(1/4)^1*(3/4)^100-1 + c(100,2)*(1/4)^2*(3/4)^100-2 + c(100,3)*(1/4)^3*(3/4)^100-3
Flipping coin 3 times example: example of flipping a coin 3 times and the event of landing on heads for each flip. The support of x, Sx = {0,1,2,3} But the sample space had 8 elements, S = {HHH, HHT, HTH,THH, HTT,THT,TTH,TTT}. For each value of x in Sx , can think of it as arranging x many heads among 3 spots. When x = 1 for example, it is the same as placing a single H among the 3 spots, such as HTT, THT, and TTH. C(3,1) which is 3. So the probability of x = 1 is 3*(1/2)^3 When x = 2 it was placing two H's among 3 spots. HHT, HTH, THH, C(3,2) which is 3. So the probability of x = 2 is 3*(1/2)^3. This is (1/2)^3+(1/2)^3+(1/2)^3 since we're using the mutually exclusive rule. Can see it as selecting x many objects out of 3. How many of the spots will be given an H designation. We have 3 spots, and need to choose x many of them (for x = 0, 1, 2, 3).
example
