Chapter 02

Fundamental Principle of Counting

If a task can be broken up into 𝘒 groups, with group 𝘬 having nₖ outcomes, then the task has this many outcomes: n₁ × n₂ × ... × nₖ = ∏nₖ from k = 1 to K

Independence Theorem

If two events A and B are independent, then: Aᶜ and B are independent A and Bᶜ are independent Aᶜ and Bᶜ are independent

Uniform Probability

Indicates that each outcome in the sample space has equal probability. If the sample space of an experiment consists of 𝘕 outcomes, then the probability of each outcome is: 1 ÷ N Uniform probability is also called FAIR.

Permutation

Is an ordering of 𝘯 objects: n × (n - 1) × (n - 2) × ... × 1 = n!

Law of Total Probability

Let the events A₁, A₂, ... Aₖ be disjoint and make up the entire sample space: S = A₁ ∪ ... Aₖ Let B an event where we want P(B) If we know P(B | Aₖ) and P(Aₖ), the Law of Total Probability says that: P(B) = P(A₁)P(B | A₁) + ... + P(Aₖ)P(B | Aₖ)

Binomial Theorem

Looks at expressions in the form: (x + y)ⁿ

Probability

Probability is assigning a "size" to each event in a consistent manner. These assignments, denoted as P(A) for event A, must satisfy these axioms: Axiom 1: 0 ≤ P(A) ≤ 1, for all events A Axiom 2: P(S) = 1 Axiom 3: If A₁, A₂, ... are disjoint, then, P(A₁ ∪ A₂ ∪ ...) = P(A₁) + P(A₂) These axioms imply: P(∅) = 0

Combinatorial Methods

Rolling a die five times is equivalent to taking a sample of size five, with replacement, from the population {1, 2, 3, 4, 5, 6} Dealing a hand of five cards is equivalent to taking a simple random sample of size five from the population of 52 cards.

Ordering

Select 𝘬 objects from a possible 𝘯 objects: If the order of the objects matter, then it's ORDERED. The way to count ordered objects is via PERMUTATIONS. n! ÷ (n - k)! If the order does not matter, then it's UNORDERED. The way to count unordered objects is via COMBINATIONS. n! ÷ (n - k)!k!

Complement (Set Operations)

The compliment of A is the event consisting of all outcomes that are: - not in A Aᶜ

Difference (Set Operations)

The difference A - B is defined as: A ∩ Bᶜ

Conditional Probability

The event A given the information that event B has occurred is denoted by P(A|B) and equals: P(A|B) = P(A ∩ B) ÷ P(B) or P(A|B)P(B) = P(A ∩ B)

Mutually Exclusive / Disjoint (Set Operations)

The events A and B are said to be mutually exclusive or disjoint if they: - have no outcomes in common A ∩ B = ∅, where ∅ denotes the empty set

Intersection (Set Operations)

The intersection of A and B is the event consisting of all outcomes that are: - in both A and B A ∩ B

Sample Space

The set of all possible outcomes of a random experiment. Sample spaces are unordered objects. Denoted by 'S' Example: - flip a coin twice and the sample space is: {H,T}

Union (Set Operations)

The union of events A and B is the event consisting of all outcomes that are: - in A - or in B - or in both A and B A ∪ B

Independence

Two events A and B are independent if and only if: P(A ∩ B) = P(A)P(B) P(A | B) = P(A) Independence and disjoint are not the same! Examples: - multiple measurements on the same thing over time - measurements that are located close to one another

Math Notations

∏: product Σ: summation !: factorial binomial coefficient: see image

Subset (Set Operations)

A is a subset of B, A ⊂ B, if: - e ∈ A implies e ∈ B

Commutative Laws (Set Operations)

A ∪ B = B ∪ A A ∩ B = B ∩ A

Events

An outcome or a collection of outcomes, classified as: SIMPLE if it consists of exactly one outcome. - the event of flipping a coin twice and observing two heads is {HH} COMPOUND if it consists of more than one outcome. - the event of choosing an odd integer between 1 and 9 is {1, 3, 5, 7, 9} - the event that two coin flips result in at least one head is {HH, HT, TH}

Random Experiment

Any process involving various outcomes to which we want to assign/compute probabilities.

Bayes Theorem

Consider the events A, B, and A₁, ... Aₖ with each of the Aₖ disjoint and S = A₁ ∪ Aₖ Given that B has occurred, what is the probability that A occurs? P(A | B) = P(A ∩ B) ÷ P(B) = P(B | A)P(A) ÷ P(B)

Properties of Probability

If A and B are disjoint, P(A ∩ B) = 0 If A ⊂ B then, P(A) ≤ P(B) P(A) = 1 − P(Aᶜ), for any event A P(A) = P(A ∩ B) + P(A ∩ Bᶜ) P(A ∪ B) = P(A) + P(B) − P(A ∩ B) P(A ∪ B ∪ C) = P(A) + P(B) + P(C) − P(A ∩ B) − P(A ∩ C) − P(B ∩ C) + P(A ∩ B ∩ C)

Distributive Laws (Set Operations)

(A ∪ B) ∩ C = (A ∩ C) ∪ (B ∩ C) (A ∩ B) ∪ C = (A ∪ C) ∩ (B ∪ C)

De Morgan's Laws (Set Operations)

(A ∪ B)ᶜ =Aᶜ ∩ Bᶜ (A ∩ B)ᶜ =Aᶜ ∪ Bᶜ

Associative Laws (Set Operations)

(A ∩ B) ∩ C = A ∩ (B ∩ C) (A ∪ B) ∪ C = A ∪ (B ∪ C)