Section 3: Probability and Random discrete Variables
Variance Scale
- Distributions that are compact will have a smaller distance from the mean. ie, smaller value - Distributions that have a further distance from the mean will have a larger value.
Discrete Probability Distributions Pre requisites
- Every numeric value is a probability and will therefore be between 0 and 1. - The sum of the probabilities associated with our random variables have to be 1.
Section 3 Summary: 2 Probability distributions
- Expected values, variances, standard deviations - Laws of expected values - Laws of variances
Variance
- Measures spread/dispersion of distribution - Let X be a discrete random variable with values xi that occur with probability p(xi), and E(X) = μ.
Section 3 Summary: 4 Binomial Distributions
- binomial experiment - bernoulli trial - binomial formula - mean and variance of a binomial variable - binomial tables.
Bivariate Distributions
1. Distribution of a single variable - univariate 2. Distribution of two variables together - bivariate 3. So, if X and Y are discrete random variables, then we say p(x,y) = P(X=x and Y=y) is the joint probability that X=x and Y=y.
Correlation
1. if X and Y have a correlation of 0 it does not mean that they are independent! 2. If 2 variables are independent then they will always have a correlation of 0.
Variance, need to know
1.) Var(X)=σ²(X)=σX² 2.) Variance is always non negative (positive or zero): notice that it is a sum of squared things times probabilities - always non-negative 3.) The positive square root of the variance is the standard deviation.
Compound Events
An event is a collection of one or more simple (individual) outcomes or events. E.g. roll a die; event A = odd number comes up. Then A={1, 3, 5}. In general, use sample space S={E1, E2,..., En} where there are n possible outcomes. Probability of an event Ei occurring on a single trial is written as P(Ei)
Joint Event
An event that has two or more chracteristics
Positive Covariance
As X increases Y Increases
Negative Covariance
As X increases Y decreases
Section 3 Summary: 3 Bivariate Distributions
Bivariate distributions: - Marginal distributions - Independence - Sum of two random variables - Covariance and Correlation - Linear Combinations of Variables
Probabilities of Combined Events (unions)
Consider two events, A and B. P(A or B) = P(A U B) = P(A union with B) = P(A occurs, or B occurs, or both occur) P(A or B)=P(A)+P(B)-P(A and B) = entire shaded area. minus the intersection of B (so that it is not double counted)
The sum of two random variables
Consider two real estate agents. X = the number of houses sold by Albert in a week Y = the number of houses sold by Beatrice in a week Bivariate distribution of X and Y shown on next slide
Covariance and Correlation
Covariance gives you the direction Correlation gives you the strength
Binomial Random Variable Example
Define a random variable, X as the number of successes in n trials Want to be able to calculate P(X=k) for k=0,1,2....,n Start with n=1 P(X=0) = P(failure) = 1-p P(X=1) = P(success) = p
Event
Each possible outcome of a variable
Centre of a Probability Distribution
Expected Value (mean)
A Bernoulli Process example
For example, ask a voter which of the two major parties they are most likely to vote for in an approaching election - Two possible outcomes for each trial, "ALP" or "Lib/Nat" - Probability of "ALP" is p, probability of "Lib/Nat" is (1-p) - Trials are independent - the response of one voter should not affect any other voters' responses
Complement Rule
Given an event A and its complement, Ā, so that A+ Ā=S; Know that P(S)=1; so P(A)+P(Ā)=1; therefore P(Ā)=1-P(A) Interpretation: either A happens, or it doesn't
Rules for Expectations - 5. E(XY)=E(X)*E(Y) only if X and Y are independent
Only holds if variables are Independent either be - told the random variables are independent - prove they are independent NEVER ASSUME
Variance Law - 4. V(X+Y)=V(X)+V(Y) if X and Y are independent
Only works if independent!
Variance Law - 5. V(X-Y)=V(X)+V(Y) if X and Y are independent
Only works if independent!
Probabilities of Combined Events (Intersections)
P(A and B) = P(A ∩ B) = P(A intersection with B) = P(A and B both occur)
Probability of A (come back to)
P(A) = P(A∩B) + P(A ∩Bc) Probability of A happening and B happening or A happens and B doesn't (what is the area of the pink circle? A + the intersection also covered by B)
Marginal Probabilities Cont
P(A1)=P(A1 and B1)+P(A1 and B2)=0.11+0.29=0.40 P(A2)=P(A2 and B1)+P(A2 and B2)=0.06+0.54=0.60 P(B1)=P(B1 and A1)+P(B1 and A2)=0.11+0.06=0.17 P(B2)=P(B2 and A1)+P(B2 and A2)=0.29+0.54=0.83
Probabilities of Combined events (conditional)
P(B|A)=P(B occurs given that A has occurred)
Probability Facts
P(S)=1 - Something has to happen 0≤P(A) ≤1 - All probabilities lie between 0 and 1
Probabilities of Combined Events (complements)
P(Ā)=P(Ac)= P(A complement) = P(A does not occur) (what does this look like?)
Section 3 Summary: Probability Concepts
Probability concepts - rules Random Variables; discrete r.v.s
Multiplication Rule
Read text Chapter 4, Sections 1 and 2 in text; read section 4 on counting rules (need later!)
Frequentists Probabilities
Relative frequency of occurrences can be used to estimate probability.
Probability Rule 1
Rule = the probability of what you are looking for exists between the possibility of it not happening at all (0) and it happening every time (1)
Probability Rule 2
Rule = the sum of all probabilities of events occurring in the sample space is equal to 1 Something always happens
4 Successes in 10 Trials
So, 4 successes can be distributed among 10 trials in how many distinct ways?
Binomial Random Variable
The number of (X) successes in (n) trials - For example, you ask 1500 voters which of the two major parties they plan to vote for in an approaching election - We define a choice of "ALP" as a "success" - Possible values of the random variable are any whole number from 0 to 1500, inclusive.
Rules for Expectations - 1.E(c)=c
The probability of a constant will always equal the constant - The centre of a constant is the constant
Variance Law - 1. V(c)=0
There is no variation on a constant, will always be constant
Mutually Exclusive
Two events are mutually exclusive if they cannot occur at the same time. An example is tossing a coin once, which can result in either heads or tails, but not both. Pr(P(A ∩ B) = 0, the two events do not intersect they have nothing to do with each other If A happens B can't happen
Variance Calculation Formula
Use this if you are given a probability distribution, not n, values for p instead. Do not mix up!
Notation Cont
When, X~Bin(n,p) P(X=k) Means probability that we will have k successes and (n-k) failures We know how many ways we arrange those successes and failures.
Describing Probability Distributions
Where is it centred? Where is it spread?
Binomial Random Variable Example (2)
X = number of successes in 2 trials P(X=0) = P(fail, fail) = P(fail)*P(fail) = (1-p)2(squared) P(X=1) = P(success, fail OR fail, success) = P(SF)+P(FS) =2p(1-p) P(X=2) = P(success,success) = P(success)*P(success) = p2(squared)
Binomial Random Variable Example (4)
X = number of successes in 4 trials P(X=0) = P(f,f,f,f) = P(f)*P(f)* P(f)*P(f) = (1-p)4 P(X=1) = P(sfff OR fsff OR ffsf OR fffs) =4p(1-p)3 P(X=2) = P(ssff OR sfsf OR sffs OR fssf OR ffss OR fsfs) =6p2(1-p)2 and so on.....
Random Variable Notation
X and Y use UPPER CASE
Binomial Random Variable Example (10)
X is the number of successes in 10 trials. P(X=0) = P(ffffffffff) = (1-p)10 P(X=10) = P(ssssssssss) = p10 What about X=4? How do we arrange 4 successes among 10 trials? Think of having 4 balls to split between 10 boxes: - 10 choices for the first ball, then 9 choices for the second ball, 8 choices for the third ball and 7 for the last ball. - But, balls are all identical, so can swap balls around within those with no affect.
Notation
X~Bin(n,p) X has a distribution of Binomial of (trials, probability) X = Binomial Variable n = trials p = probability of success
A Bernoulli Process
a finite or infinite sequence of binary random variables, so it is a discrete-time stochastic process that takes only two values, canonically 0 and 1. The component Bernoulli variables Xi are identical and independent.
Random Variable
a function that assigns a numeric value to each simple event in a sample space
Sample space
a list of all possible outcomes - must have events that are collectively exhaustive or mutually exclusive e.g. roll a die (1,2,3,4,5,6)
Correlation Coefficient
On a scale
Conditional Probability
= probability of event A given information about the occurrence of other event B
Binomial Experiment (Special Case)
A binomial experiment has the following properties: 1. A fixed number of trials, n 2. Two possible outcomes for each trial, labelled success and failure 3. Probability of success is p, probability of failure is (1-p) 4. Trials are independent - the outcome of one trial does not affect the outcomes of any other trials.
Subjective Probability
A probability derived from an individual's personal judgment about whether a specific outcome is likely to occur. Subjective probabilities contain no formal calculations and only reflect the subject's opinions and past experience.
Discrete Probability Distributions
A table or formula listing all possible values that a discrete r.v. can take, together with the associated probabilities. - have to have numeric value for every possible outcome. - E.g. for our toss three coins example:
Cumulative Binomial Distribution Table - appendix
Can only use the n or p you have or use the table do not add - Note that P(X ≤0)= P(X=0) - P(X ≤k) →use tables - P(X=k) = P(X≤k) - P(X≤(k-1)) - P(X<k) = P(X≤[k-1]) - P(X≥k) = 1- P(X≤[k-1]) - P(X>k) = 1- P(X ≤k)
Mean Calculation with probability
Centre of this distribution is 1.5 - the mean doesn't have to be observable (not 1.5 heads in 3 coin tosses) - do not round off to observable value.
Complement
Complement of event A includes all events that are not part of A
Marginal Probabilites
Computed by adding across rows or down columns - calculated in the margins of the table (general addition rule)
Conditional Probability Equation
Conditional Probability that A occurs, given that B has occured
Covariance
Consider the r.v.s X and Y with joint pdf p(x,y); x=x1,...,xm; y=y1,...,yn. If E(X) = µx and E(Y)= µy, then the covariance between X and Y is given by
Multiplication rule for independent Events
If A and B are independent, the probability of A and B is equal to the probability of A x B
Independence
If A happens I don't change my P of B happening
Binomial distribution, Size of P
If P is large - E(X) will be closer to 1 - middle If P is small - E(X) will be closer to 0 When P is = 0.5 there is a large variance
Variance Laws
If X and Y are r.v.s and c is a constant, 1. V(c)=0 2. V(cX)=c²V(X) 3. V(X+c)=V(X) 4. V(X+Y)=V(X)+V(Y) if X and Y are independent 5. V(X-Y)=V(X)+V(Y) if X and Y are independent
Rules for Expectations
If X and Y are random variables, and c is any constant, then the following hold: 1.E(c)=c 2. E(cX)=cE(X) 3. E(X-Y)=E(X)-E(Y) 4. E(X+Y)=E(X)+E(Y) 5. E(XY)=E(X)*E(Y) only if X and Y are independent
Proving Independence of variables
If the random variables X and Y are independent, then P(X=x and Y=y) = P(X=x) . P(Y=y) p(x,y) = px(x) . py(y) In previous example, X and Y are clearly not independent: p(0, 0) = 1/8 px(0) . py(0) = 1/8 * 2/8 = 1/32 p(0, 0) ≠ px(0) . py(0)
Joint probabilities Cont
Joint probabilities: P(B1AND A1)=0.11 P(B1 AND A2)=0.06 P(B2 AND A1)=0.29 P(B2 AND A2)=0.54 AND can = intersection sign U
Joint Probabilities
Let events be known as follows: A1=Fund manager graduated from a top-20 MBA program A2=Fund manager graduated from a not top-20 MBA program B1=Fund outperforms the market B2=Fund does not outperform the market *use venn diagrams
Mean and Variance of a Binomial Distribution
Mean doesn't have to be observable Mean also not an expected value Expected value = infinitely long run average
Variance Law - 3. V(X+c)=V(X)
adding or subtracting a constant doesn't change the spread
Collectively exhaustive
at least one of the events must occur. E.g, coin toss, heads or tails collectively exhaustive, because they encompass the entire range of possible outcomes.
Simple Event
described by a single characteristic
Rules for Expectations - 2. E(cX)=cE(X)
expected value = the expected value of random variable multiplied by the constant - doesn't change anything, - just shifts up (translation relative to the constant)
Discrete Random Variables
has a countable number of possible values - eg, number of heads, number of sales - doesn't have to be finite -must be able to strictly order the values
Continuous Random Variables
has an infinite number of possible values. Number of elements in sample space is infinite as a result of continuous variation. - eg, height and weight - values cannot be strictly ordered (different instrument may result in new values between those already observed)****WHAT****
A Binomial Experiment
occurs when property 1 (a fixed number of trials ) to a bournoulli process.
Empirical probability
probabilities are based on observed data, not on prior knowledge. - Surveys
Joint Probability
probability of an occurrence involving two or more events
Priori probability
probability of an occurrence is based on prior knowledge of the process involved. where each event is equally likely
Simple Probability
probability of occurrence of a simple event
Random experiment
process that results in a number of possible outcomes, none of which can be predicted with certainty. eg. Toss a coin (heads or tails)
Complement of an event
the set of all outcomes not belonging to that event e.g. if A = {1, 3, 5}, then the complement of A = Ac = {2, 4, 6}
Rules for Expectations - 3. E(X-Y)=E(X)-E(Y) and 4. E(X+Y)=E(X)+E(Y)
use individual probabilities to determine value - do not create a new probability or a new random variable - use previous knowledge of X and Y
Variance Law - 2. V(cX)=c²V(X)
variance of a constant times x = constant squared x variance of X. (variance is in squared units)
Actual Realised Values
x and y use lower case