Probability: Midterm 2
Tail sum formula for expectation of a counting variable
For a random variable X with possible values {0, 1, ..., n}: E[X] = ∑(j = 1 to n) P(X ≥ j)
Which is typically better: Markov's inequality or Chebyshev's inequality?
Generally speaking, Chebyshev's inequality will produce tighter bounds than Markov's inequality, since it uses the additional information of the standard deviation.
Expectation of a function of a discrete random variable
IF X is a discrete random variable and g(X) is a function of X, then E[g(X)] = ∑ₓ g(x)P(X = x)
Expectation of an indicator
If Iₐ is an indicator for event A, then E[Iₐ] = P(A)
Multiplication rule for expectation
If X and Y are independent random variables, then E[XY] = E[X]E[Y]
Linearity of variance
If X and Y are independent random variables, then Var(X + Y) = Var(X) + Var(Y)
Binomial random variable
If X follows a binomial distribution with probability p of success for any given trial, then the probability that there are k successes in n trials is given by: P(X = k) = C(n, k)⋅pᵏ(1-p)ⁿ⁻ᵏ
Population parameters of a binomial distribution
If X follows a binomial distribution with probability p of success, probability q = 1 - p of failure, and n total trials, then: E[X] = µ = np Var(X) = σ² = npq SD(X) = σ = √(npq)
Population parameters of the Poisson distribution
If X is a Poisson random variable with parameter λ, then: E[X] = µ = λ Var(X) = σ² = λ SD(X) = σ = √(λ)
Expectation of a continuous random variable
If X is a continuous random variable and probability density function f(x), then E[X] = ∫(-∞ to ∞) x⋅f(x)dx
Variance of a continuous random variable
If X is a continuous random variable with mean µ and probability density function f(x), then Var(X) = ∫(-∞ to ∞) (x - µ)⋅f(x)dx
Population parameters of a geometric distribution
If X is a geometric random variable with probability p of success and probability q = 1 - p of failure, then: E[X] = µ = 1/p Var(X) = σ² = q / p² SD(X) = σ = √(q) / p
P(X > n) and P(X ≥ n) for a geometric random variable
If X is a geometric random variable with probability p of success, the chance that the first success happens after the nth trial is equal to the chance that the first n trials were all failures: P(X > n) = (1 - p)ⁿ P(X ≥ n) = (1 - p)ⁿ⁻¹
Population parameters of a negative binomial distribution
If X is a negative binomial random variable with probability p of success, probability q = 1 - p of failure, and r successes, then: E[X] = µ = r/p Var(X) = σ² = rq / p² SD(X) = σ = √(rq) / p
Corollary to Markov's inequality
If X is a nonnegative random variable (X ≥ 0), then P(X < a) ≥ 1 - (E[X] / a) for every a > 0
Expectation of a linear function of a random variable
If X is a random variable and a, b are constants, then E[aX + b] = aE[X] + b, so for any linear function g(X) it follows that E[g(X)] = g(E[X])
Normal random variable
A continuous random variable X is said to be normal if it has a probability density function of the form f(x) = 1/(σ√(2π))⋅e^(-0.5⋅((x - µ) / σ)²) where µ and σ are two scalar parameters characterizing the PDF, with σ assumed nonnegative. µ specifies the mean of the normal distribution, and σ specifies the standard deviation of the normal distribution
Probability density function
A function f(x) satisfying: 1) f(x) ≥ 0 2) f(x) is continuous 3) ∫(-∞ to ∞) f(x)dx = 1 (sum of the area under f(x) is 1) is called a probability density function if, for some random variable X, P(a ≤ X ≤ b) is given by ∫(a to b) f(x)dx
Relationship between geometric random variables and negative binomial random variables
A geometric random variable is just a negative binomial random variable with parameters (r=1, p=p).
Negative binomial experiment
A negative binomial experiment satisfies: 1) The experiment consists of a sequence of independent trials. 2) Each trial can result in either a success (S) or a failure (F). 3) The probability of success is constant from trial to trial, so for i = 1, 2, 3, ... 4) The experiment continues (trials are performed) until a total of r successes have been observed, where r is a specified positive integer. 5) The random variable of interest is X = the number of trials required to achieve r successes (the number of successes is fixed and the number of trials is random)
Indicator random variable
A random variable Iₐ is an indicator random variable for event A if Iₐ = 1 if A occurs Iₐ = 0 if A does not occur (if Aᶜ occurs)
Negative binomial random variable
A random variable X is a negative binomial RV if it represents the number of trials required to achieve r successes. If p is the probability of success for any trial, then the probability that it takes n trials to achieve r successes is given by: P(X = n) = C(n - 1, r - 1)⋅pʳ(1-p)ⁿ⁻ʳ
Continuous random variable
A random variable X is continuous if there is a function f(x) such that for any a ≤ b we have P(a ≤ X ≤ b) = ∫(a to b) f(x)dx
Geometric random variable
A random variable X is geometric if it represents the number of trials it takes to get a success in a geometric setting (possible values are 1, 2, 3, ...) If p is the probability of success for any trial, then the probability that it takes n trials to achieve the first success is given by: P(X = n) = (1 - p)ⁿ⁻¹p
Uniform random variable
A random variable X is said to be uniformly distributed over the interval (α, β) if its probability density function is given by f(x) = 1 / (β - α) if α < x < β f(x) = 0 otherwise
Method of indicators
A random variable X that counts the number of events of some kind that occur can be represented as the sum of the indicators of these events. Then, E[X] = E[∑ₓIₓ] = ∑ₓE[Iₓ] = ∑ₓP(X = x)
Poisson random variable
A random variable X that takes on one of the values 0, 1, 2, ... is said to be a Poisson random variable with parameter λ if, for some λ > 0: P(X = k) = e^(-λ)⋅λᵏ/k! λ represents the average number of events that can occur (if approximating a binomial distribution, λ = np) P(X = k) represents the probability of k events occurring in a fixed interval of time.
Discrete random variable (informal definition)
A random variable that can take on at most a countable number of possible values
Random variable (informal definition)
A variable that assumes numerical values associated with the random outcome of an experiment, where one (and only one) numerical value is assigned to each sample point.
Relationship between binomial and normal distributions
As the number of observations n becomes larger, the binomial distribution gets close to a normal distribution.
Law of large numbers
As the number of trials in a probability experiment increases, the difference between the theoretical probability of an event and the relative frequency approaches zero
Boole's inequality for counting random variables
If X is a counting random variable, then P(X ≥ 1) ≤ E[X]
Expectation of a discrete random variable
If X is a discrete random variable, the expectation of X, denoted E[X], is the mean of the distribution of X. E[X] = ∑ₓ x⋅P(X = x)
Markov's inequality
If X is a nonnegative random variable (X ≥ 0), then P(X ≥ a) ≤ E[X] / a for every a > 0
Population parameters of the normal distribution
If X is a normal random variable with parameters µ and σ, then: E[X] = µ Var(X) = σ² SD(X) = σ
Standard deviation of scaling/shifting
If X is a random variable and a, b are constants, then SD(aX + b) = |a|SD(X)
Variance of scaling/shifting
If X is a random variable and a, b are constants, then Var(aX + b) = a²Var(X)
Bound on one tail (Chebyshev's inequality)
If X is a random variable with expectation µ and standard deviation σ, then for all k > 0: P(X - µ ≥ k) ≤ P(|X - µ| ≥ k) ≤ σ² / k²
Chebyshev's inequality
If X is a random variable with expectation µ and standard deviation σ, then for all k > 0: P(|X - µ| ≥ k) ≤ σ² / k² = Var(X) / k² Equivalently, P(|X - µ| ≥ kσ) ≤ 1 / k² This inequality finds the bound on both tails in X's distribution
Variance
If X is a random variable, the variance of X, denoted Var(X) or σ², is the mean squared deviation of X from its expected value µ = E(X): Var(X) = σ² = E[(X - µ)²] = ∑ₓ (x - µ)²P(X = x) Alternatively, Var(X) = E[X²] - E[X]² = E[X²] - µ² = ∑ₓ(x²P(X = x)) - µ²
Standard deviation
If X is a random variable, then the standard deviation of X, denoted SD(X), is the square root of the variance of X: SD(X) = σ = ƒ√(Var(X))
Population parameters of the uniform distribution
If X is a uniform random variable over the interval (α, β), then: E[X] = µ = (β + α) / 2 Var(X) = σ² = (β - α)² / 12 SD(X) = σ = (β - α) / √(12)
Standardization of a normal random variable
If X is normally distributed with mean µ and SD σ, then Z = (X - µ) / σ has the standard normal distribution
Rule of three sigmas for a normal random variable
If X is normally distributed, then P(µ - 3σ < X < µ + 3σ) ≈ 1
Discrete random variable (formal definition)
If Ω is an outcome space, then a discrete random variable is a function X : Ω → ℝ that takes a discrete set of values and returns the probability that each value is chosen.
Weak law of large numbers
Let X₁, ..., Xₙ be a sequence of independent and identically distributed random variables, each having finite mean E[Xᵢ] = μ. Then, for any ε > 0, P(|(X₁ + ... + Xₙ)/n - µ| ≥ ε) → 0 as n → ∞
P(X = a) if X is a continuous random variable
P(X = a) = ∫(a to a) f(x)dx = 0
Geometric setting
The goal is to repeat a chance behavior until a success occurs. A setting is geometric if: B (binary?): The possible outcomes of each trial are binary (success/failure) I (independent?): Trials are independent T (trials?): The goal is to count the number of trials until the first success occurs S (success?): On each trial, the probability of success is the same
Standard normal distribution
The standard normal distribution is the normal distribution with mean 0 and standard deviation 1.
Infinite series expansion for e^a
e^a = ∑(k = 0 to ∞) a^k / k!
Binomial experiment
A binomial experiment is an experiment that satisfies: 1) Each trial results in one of two mutually exclusive outcomes (success/failure). 2) There are a fixed number of trials. 3) Outcomes of different trials are independent. 4) The probability that a trial results in success is the same for all trials.
Examples of non-binomial experiments
1) Deal 10 cards from a shuffled deck and count the number of red cards (not a binomial experiment since the probability does not remain constant for each trial) 2) Two parents with genes for O and A blood types and count the number of children with blood type O (not a binomial experiment since there is no fixed number of kids/trials) 3) You roll a die 10 times and note the number the die lands on (not a binomial experiment since there are more than 2 possible outcomes)
Properties of the normal distribution
1) The mean, median, and mode are equal. 2) The normal curve is bell-shaped and symmetric about the mean. 3) The total area under the curve is equal to one. 4) The normal curve approaches, but never touches the x-axis as it extends further and further away from the mean. 5) The inflection points are μ − σ and μ + σ (concave down between these points, and concave up outside of this range)
Differences between binomial and geometric distributions
1) There is NOT a fixed number of trials in geometric distributions! 2) Binomial random variables start with 0 while geometric random variables start with 1 3) Binomial distributions are finite, while geometric distributions are infinite.
Examples of binomial experiments
1) Toss a coin 10 times and count the number of heads. 2) You randomly select a card from a deck of cards, and note if the card is an Ace. You then put the card back and repeat this process 8 times.
Calculating normal distribution probabilities using the standard normal cumulative distribution function
If X is normally distributed, then: P(a ≤ X ≤ b) = Φ((b - µ) / σ) - Φ((a - µ) / σ) where Φ is the standard normal cumulative distribution function, which gives the cumulative area to the left of a specified point for the standard normal distribution Values of Φ are typically given in a table.
Expected number of events that occur
If X is the number of events that occur among some collection of events A₁, ..., Aₙ, then E[X] = P(A₁) + ... + P(Aₙ)
Expectation/variance/standard deviation of the average of independent random variables
If X₁, ..., Xₙ are independent random variables, each with the same expectation µ and variance σ², and if we denote the average of these variables as X̄ = (X₁ + ... + Xₙ) / n, then: E[X̄] = µ Var(X̄) = σ² / n SD(X̄) = σ / √(n)
Sums of independent Poisson random variables
If X₁, ..., Xⱼ are independent Poisson random variables with parameters λ₁, ..., λⱼ, then X₁ + ... + Xⱼ is a Poisson random variable with parameter λ₁ + ... + λⱼ
Expectation of a constant
If c is a constant, then E[c] = c Notably, this implies that E[cX] = cE[X]
Variance of a constant
If c is a constant, then Var(c) = 0
When to use the method of indicators
Prerequisite: X must be a counting variable among some collection of events A₁, ..., Aₙ. 1) The probabilities P(X = x) are known, but given by a formula that makes the expression for E[X] hard to simplify. 2) The nature of the dependence between the events is either unknown, or known but so complicated that it is difficult to obtain a formula for P(X = x).
Standard normal cumulative distribution function symmetry
Since the standard normal cumulative distribution function is symmetric: Φ(-z) = 1 - Φ(z)
When is the sum of indicators itself an indicator?
Suppose A₁, ..., Aₙ are events with indicators I₁, ..., Iₙ. Then ∑ⱼIⱼ is an indicator iff all events Aⱼ are mutually exclusive.
Boole's inequality
Suppose A₁, ..., Aₙ are events. The probability of the union of all events Aⱼ is less than or equal to the sum of the probabilities of all Aⱼ
When can the Poisson distribution be utilized to approximate the binomial distribution?
When n is large and p is small. A good rule of thumb is that if np < 10, you can use the Poisson distribution as an approximation
Linearity of expectation
For any two random variables X and Y, E[X + Y] = E[X] + E[Y] no matter whether X and Y are independent or not
Effect of varying µ for a normal distribution
µ controls the location of the mean, so the "hump" will shift to be centered at µ.
Effect of varying σ for a normal distribution
σ controls the spread of the distribution, since it is exactly the standard deviation for the normal distribution.
Sum of infinite geometric series rule
∑(n = 0 to ∞) arⁿ = a / (1 - r)
