Math 394 midterm 1

Definition 1.3 (Event)

Events are subsets of the sample space Ω.

Corollary 1.12 (Generalization of De Morgan's Law)

For any A1,...,An ⊆Ω we have( n⋃k=1Ak)c=n⋂k=1Ack and( n⋂k=1Ak)c=n⋃k=1Ack. Big intersection and Big Union DeMorgan's generalized

Definition 1.8 (Subset)

Let Ω be a set. A is a subset of Ω if it is a set composed of elements of Ω.We write A ⊆Ω.

Definition 1.5 (Laplace Experiment)

Suppose that all outcomes of an experiment with finite sample space Ω are equally likely. Then, for all events A ⊂Ω,P(A) = # simple events in A / # simple events in Ω. We call P Laplace distribution (over Ω).

Lemma 1.6

The Laplace distribution P over Ω has the following properties:(i) P(Ω) = 1.(ii) P(A ∪B) = P(A) + P(B) for disjoint events A and B (i.e. disjoint means that A ∩B = ∅).

Lemma 2.2 with replacement order matters

The cardinality (aka "size") of the set Ω is |ΩI |= n^k.

Lemma 2.3w/out replacement order matters

The cardinality of the set Ω is |Ω|= n *(n- 1) *...*(n k + 1).

Lemma 2.6 without replacement order irrelevant

The cardinality of the set Ω, IΩI is n choose k

Definition 3.11

The events A1, . . . , An are called independent if for each k ∈{1, . . . , n} and each collection of indices 1 ≤i1 < . . . < ik ≤ n P(Ai1 ∩. . . ∩Aik) = P(Ai1) . . . P(Aik).

Lemma 3.2 (Discrete Probability Distribution)

(i) Any discrete probability measure satisfies a discrete probability measure P is fully characterized by its values on simple events. (ii) Any function p : Ω -> R which satisfies (P1) p(w) = 0 except for countably many w in Ω, (P2) p(w) > 0 for all w in Ω, (P3) The sum of all p(w) = 1 is a probability measure on (Ω,A)

Theorem 1.19 (Continuity from "Above and Below")

(i) Let A1,A2,... be an increasing sequence of events, i.e.A1 ⊂A2 ⊂A3 ⊂..., then limn→∞P(An) = P( ∞⋃k=1Ak). (ii) Let B1,B2,... be an decreasing sequence of events, i.e.B1 ⊃B2 ⊃B3 ⊃..., then limn→∞P(Bn) = P( ∞⋂k=1Bk).

Definition 1.15 (Sigma Algebra)

A collection A of subsets of Ω is called a σ-algebra if it satisfies the following conditions: (i) ∅∈A; (ii) A ∈A =⇒ Ac ∈A ("closed under complement"); (iii) A1,A2,... ∈A =⇒ ⋃∞k=1 Ak ∈A ("countable additive").

Definition 1.7 (Set)

A set is a collection of items.

Definition 1.1 (Experiment)

An experiment is any activity or process whose outcome is subject to uncertainty.

Definition 3.3 (Urn Model With Colored Balls)

Consider an urn with N balls which are labeled 1, . . . , N with balls {1, . . . , R }being red and {R + 1 , . . . , N }being white. We draw n times a ball at random from the urn and its number and/ or color is noted.

Definition 3.6 (Urn Model With Many Colored Balls)

Consider an urn with N balls which are labeled 1,...,N with the first N1balls of color 1, the second N2 balls of color 2, . . . , the last Nr balls of color r. We draw n times a ball at random from the urn and its number and/ or color is noted.

Definition 2.1 (Urn Model)

Consider an urn with n balls which are labeled 1, . . . , n . An urn model is an experiment in which k times a ball is drawn at random from the urn and its number is noted.

Lemma 3.5

Define the probability mass function of the binomial distribution as p(r) := (n choose r) (R/N)^r (1- (R/N))^(n-r)

Lemma 3.4

Define the probability mass function of the hypergeometric distribution as p(r) := (R choose n) (R-N choose r-n) / (N choose n) for r in {0, 1, . . . , n }. Then, P(Er) = p(r).

Lemma 3.9

Define the probability mass function of the multinomial distribution as (n choose n1...nr) product all (Nk/N)^r_k

Theorem 1.10 (De Morgan's Law)

For any events A and B we have(A ∪B)c = Ac ∩Bc and (A ∩B)c = Ac ∪Bc.

Lemma 1.18 (Inclusion-Exclusion Formula)

For any events A1,...,An we have P(A1 ∪... ∪An) =n∑k=1(−1)k−1 ∑1≤i1≤...≤ik≤n P(A1 ∩... ∩Ak)

Definition 2.5 (Factorial)

For n in N we define n! (read: "n factorial") as n! := n *(n 1) *...*2*1 ,and for n = 0 we define 0! := 1.

Definition 2.4 (Falling Factorial)

For r in R and k in N we define (r)_k (read: "r falling k") as (r)k := r *(r 1) *...*(r k + 1).

Definition 1.11

Given A1,...,An ⊆Ω we define n⋃k=1 Ak := A1 ∪... ∪An = {ω ∈Ω |∃k ∈{1,...n}: ω ∈Ak}= {ω ∈Ω |ω ∈Ak for at least one k ∈{1,...,n}},n⋂k=1Ak := A1 ∩... ∩An = {ω ∈Ω |∀k ∈{1,...,n}: ω ∈Ak}= {ω ∈Ω |ω ∈Ak for all k ∈{1,...,n}}. Big union big intersection

Definition 1.13

Let (A1)∞k=1 be a sequence of subsets in Ω and definelim infn→∞ An :=∞⋃n=1∞⋂k=nAk = {ω ∈Ω |∃n ≥1 : ∀k ≥n : ω ∈Ak},lim supn→∞An :=∞⋂n=1∞⋃k=nAk = {ω ∈Ω |∀n ≥1 : ∃k ≥n : ω ∈Ak}.

Definition 3.10 (Independent Events)

Let (Ω, A, P) be a probability triple. Two events A and B on (Ω, A, P) are called independent if P(A ∩B) = P(A)P(B).

Lemma 1.17

Let (Ω,A,P) be a probability triplet and A,B ⊆Ω. (i) P(Ac) = 1 −P(A); (ii) if A ⊆B then P(A) ≤P(A) + P(B \A) = P(B); (iii) P(A ∪B) = P(A) + P(B) −P(A ∩B).

Definition 4.3 (Conditional Probability)

Let A, B subset Ω be events such that P(A) > 0. The conditional probability of B given A is defined as P(B |A) := P(A ∩B) / P(A) .

Lemma 1.9

Let A,B,C ⊆Ω be three events. (i) Commutativity: A ∪B = B ∪A and A ∩B = B ∩A. (ii) Associativity:(A ∪B) ∪C = A ∪(B ∪C) and (A ∩B) ∩C = A ∩(B ∩C). (iii) Distributivity:(A ∪B) ∩C = (A ∩C) ∪(B ∩C) and (A ∩B) ∪C = (A ∪C) ∩(B ∪C).

Lemma 4.4 (Multiplication Rule)

Let A1, . . . , A n subset Ω be events with P(A1 ∩. . . A _n-1) 6= 0. Then, P(A1 ∩. . . ∩An) = P(A1) *P(A2 |A1) *. . . *P(An |A1, . . . , A_n-1).

Lemma 3.12

Let A1, . . . An be independent events. Consider events B1, . . . , Bn such that Bi = Ai or Bi = A^ci .Then the events B1, . . . Bn are independent.

Lemma 4.5 (Law of Total Probability)

Let B1, . . . , B n subset Ω be a disjoint partition of Ω, i.e. B1 . . . B n and Bi ∩ Bj = empty set for i != j. If P(Bi) > 0 for all 1 <=i <=n, then for any event A subset Ω, P(A) = Sum all over i P(A |Bi)P(Bi).

Definition 1.14 (Probability Measure on a finite sample space)

Let Ω be a finite sample space and A be the collection of all subsets of Ω. A probability measure on (Ω,A) is a function P from A into the real numbers that satisfies (i) P(A) ≥0 for all A ∈A; (ii) P(Ω) = 1; (iii) P(A ∪B) = P(A) + P(B) for all pairwise disjoint A,B ∈A. The number P(A) is called the probability that event A occurs.

Probability (Informal Definition)

Let Ω be a sample space and A a collection of events in Ω. We define a probability measure by assigning each event A ∈A a number P(A) between 0and 1. We call P(A) the probability of the event A.

Definition 1.16 (Probability Measure on general sample spaces)

Let Ω be a sample space and A a σ-algebra on Ω. A probability measure on(Ω,A) is a function P from A into the real numbers that satisfies (i) P(A) ≥0 for all A ∈A; (ii) P(Ω) = 1; (iii) if A1,A2,... ∈A is a collection of pairwise disjoint events, in that Aj ∩Ak = ∅ for all pairs j,k satisfying j 6= k, then P( ∞⋃k=1Ak)=∞∑k=1P(Ak). The triplet ( Ω, A, P) is called a probability space

Lemma 4.6 (Bayes' Rule)

P(Bk |A) = P(A |Bk)P(Bk) / sum all over i P(A |Bi)P(Bi).

Lemma 4.2

Recall the notation of product space and product measure. Let Ai in Ωi be events and let A'i be defined as above, 1 <= i <=n. Then P(A0i) = Pi(Ai) for all i = 1 , . . . , n, and the events A'1, . . . , A'n are (stochastically) independent.

Definition 1.4 (Simple Event)

Simple events are subsets of the sample space that contain only one outcome.

Lemma 3.7

The number of possible ways in which a set A with cardinality |A|= k can be partitioned into n subsets A1,...,An with cardinalities k1,...,kn such thatk1 + ...+ kn = kis given by k! / k1!...kn!

Definition 1.2 (Sample Space)

The sample space of an experiment is the set of all possible outcomes of the experiment. We denote the sample space by Ω.

Lemma 4.8 (1) x_t+1 = ax_t+ b, t= 0 ,1,...,

The solution to the first-order linear difference equation (1) is x_t= a^t (x_0 - (b/1-a)) + b/1-a

Lemma 4.7 (Solution to Gambler's Ruin)

fjarked up symbols look on slides

Definition 3.8 (Multinomial Coefficient)

multinomial coefficient works when ki are all positive and sum of all ki is k. Otherwise 0

Definition 3.1 (Discrete Probability Space)

probability space (Ω, A, P) is called discrete if there exists a finite or countably infinite subset D subset Ω such that P(D) = 1. The associated probability measure is also called discrete.

Definition 4.1 (Product Space and Product Measure)

probability spaces cross producted

Definition 2.7 (Binomial Coefficient)

r choose k = r falling k / n! unless n < 0 then = 0

Lemma 2.8 with replacement order irrelevant

|Ω| = (k+n-1) choose (n-1) = (k+n-1) choose k