Chapter 02
Fundamental Principle of Counting
If a task can be broken up into 𝘒 groups, with group 𝘬 having nₖ outcomes, then the task has this many outcomes: n₁ × n₂ × ... × nₖ = ∏nₖ from k = 1 to K
Independence Theorem
If two events A and B are independent, then: Aᶜ and B are independent A and Bᶜ are independent Aᶜ and Bᶜ are independent
Uniform Probability
Indicates that each outcome in the sample space has equal probability. If the sample space of an experiment consists of 𝘕 outcomes, then the probability of each outcome is: 1 ÷ N Uniform probability is also called FAIR.
Permutation
Is an ordering of 𝘯 objects: n × (n - 1) × (n - 2) × ... × 1 = n!
Law of Total Probability
Let the events A₁, A₂, ... Aₖ be disjoint and make up the entire sample space: S = A₁ ∪ ... Aₖ Let B an event where we want P(B) If we know P(B | Aₖ) and P(Aₖ), the Law of Total Probability says that: P(B) = P(A₁)P(B | A₁) + ... + P(Aₖ)P(B | Aₖ)
Binomial Theorem
Looks at expressions in the form: (x + y)ⁿ
Probability
Probability is assigning a "size" to each event in a consistent manner. These assignments, denoted as P(A) for event A, must satisfy these axioms: Axiom 1: 0 ≤ P(A) ≤ 1, for all events A Axiom 2: P(S) = 1 Axiom 3: If A₁, A₂, ... are disjoint, then, P(A₁ ∪ A₂ ∪ ...) = P(A₁) + P(A₂) These axioms imply: P(∅) = 0
Combinatorial Methods
Rolling a die five times is equivalent to taking a sample of size five, with replacement, from the population {1, 2, 3, 4, 5, 6} Dealing a hand of five cards is equivalent to taking a simple random sample of size five from the population of 52 cards.
Ordering
Select 𝘬 objects from a possible 𝘯 objects: If the order of the objects matter, then it's ORDERED. The way to count ordered objects is via PERMUTATIONS. n! ÷ (n - k)! If the order does not matter, then it's UNORDERED. The way to count unordered objects is via COMBINATIONS. n! ÷ (n - k)!k!
Complement (Set Operations)
The compliment of A is the event consisting of all outcomes that are: - not in A Aᶜ
Difference (Set Operations)
The difference A - B is defined as: A ∩ Bᶜ
Conditional Probability
The event A given the information that event B has occurred is denoted by P(A|B) and equals: P(A|B) = P(A ∩ B) ÷ P(B) or P(A|B)P(B) = P(A ∩ B)
Mutually Exclusive / Disjoint (Set Operations)
The events A and B are said to be mutually exclusive or disjoint if they: - have no outcomes in common A ∩ B = ∅, where ∅ denotes the empty set
Intersection (Set Operations)
The intersection of A and B is the event consisting of all outcomes that are: - in both A and B A ∩ B
Sample Space
The set of all possible outcomes of a random experiment. Sample spaces are unordered objects. Denoted by 'S' Example: - flip a coin twice and the sample space is: {H,T}
Union (Set Operations)
The union of events A and B is the event consisting of all outcomes that are: - in A - or in B - or in both A and B A ∪ B
Independence
Two events A and B are independent if and only if: P(A ∩ B) = P(A)P(B) P(A | B) = P(A) Independence and disjoint are not the same! Examples: - multiple measurements on the same thing over time - measurements that are located close to one another
Math Notations
∏: product Σ: summation !: factorial binomial coefficient: see image
Subset (Set Operations)
A is a subset of B, A ⊂ B, if: - e ∈ A implies e ∈ B
Commutative Laws (Set Operations)
A ∪ B = B ∪ A A ∩ B = B ∩ A
Events
An outcome or a collection of outcomes, classified as: SIMPLE if it consists of exactly one outcome. - the event of flipping a coin twice and observing two heads is {HH} COMPOUND if it consists of more than one outcome. - the event of choosing an odd integer between 1 and 9 is {1, 3, 5, 7, 9} - the event that two coin flips result in at least one head is {HH, HT, TH}
Random Experiment
Any process involving various outcomes to which we want to assign/compute probabilities.
Bayes Theorem
Consider the events A, B, and A₁, ... Aₖ with each of the Aₖ disjoint and S = A₁ ∪ Aₖ Given that B has occurred, what is the probability that A occurs? P(A | B) = P(A ∩ B) ÷ P(B) = P(B | A)P(A) ÷ P(B)
Properties of Probability
If A and B are disjoint, P(A ∩ B) = 0 If A ⊂ B then, P(A) ≤ P(B) P(A) = 1 − P(Aᶜ), for any event A P(A) = P(A ∩ B) + P(A ∩ Bᶜ) P(A ∪ B) = P(A) + P(B) − P(A ∩ B) P(A ∪ B ∪ C) = P(A) + P(B) + P(C) − P(A ∩ B) − P(A ∩ C) − P(B ∩ C) + P(A ∩ B ∩ C)
Distributive Laws (Set Operations)
(A ∪ B) ∩ C = (A ∩ C) ∪ (B ∩ C) (A ∩ B) ∪ C = (A ∪ C) ∩ (B ∪ C)
De Morgan's Laws (Set Operations)
(A ∪ B)ᶜ =Aᶜ ∩ Bᶜ (A ∩ B)ᶜ =Aᶜ ∪ Bᶜ
Associative Laws (Set Operations)
(A ∩ B) ∩ C = A ∩ (B ∩ C) (A ∪ B) ∪ C = A ∪ (B ∪ C)