CIS 262 Final
Problems about NFAs
Decidable because we can convert to equivalent DFAs and decide in the same way, but this results in exponential states, so they won't be in poly-time in this manner - instead check states in NFA as they are to solve (A, E, and universality) - EQ_NFA can't be done in poly-time because there is no simple negation of states since NFA accepts if any path leads to acceptance.
Multi tape turing machines and two way infinite tapes
do not change the computation power
TM transition function
delta(q_0, a) = (q_1, b, L) New state, symbol written out, direction of movement
To show something is in NP
- give
Decidable languages are closed under:
- union (for decidable languages L1 and L2, construct TM M such that on input w, execute M1 on w, and if it accepts, accept, otherwise, execute M2 on w, and if it accepts, accept, otherwise reject) - intersection - complement - concatenation (try every partition until 1 works such that w of finite length is split up and left half is run on M1 and right is run on M2) - Kleene star (w is finite, so try every partition of w such that w1 is in L, w2 is in L, etc)
Given a set U of elements and a collection S1, . . . , Sm of subsets of U, is there a collection of at most k of these sets whose union equals U?
1. in NP: given collection of subsets, we can check in poly-time that this covers U 2. given instance of vertex cover (graph and vertices) construct set cover instance. U is all edges. If we set the vertex cover size equal to the set cover size, all edges are covered, so all edges are in the set where a collection is made for each vertex (which includes the edges that are incident on it) 3. given vertex cover instance ==> we have set cover instance 4. given set cover instance ==> we have vertex cover instance 5. conversion takes poly-time
show a problem X is NP complete
1. show in NP (poly-time verifier, given certificate) 2. pick an NP complete problem Y to reduce to X 3. construct an algorithm to solve Y given an algorithm to solve X (construct input to the black box for X) 4. prove the correctness of algorithm iff (given yes instance of X, alg returns yes, and if alg returns yes, given input is yes) show the deconstruction 5. since alg runs in poly-time, say that Y <p X
Decidable languages are closed under union
Consider the following TM M: Given an input w, Execute M1 on w; If M1 accepts then accept Else { Execute M2 on w; If M2 accepts, accept else reject } M is a halting TM, and L(M) = Union of L1 and L2
2SAT is in
P
property of strings
a function f that maps strings to true or false - property holds for almost all strings if the fraction of strings of length n on which it is false approaches 0 as n grows
Recognizable languages are not closed under:
complement
P and EXPTIME are closed under
complement (because defined use deterministic TMs)
CoNP
complement of class NP - complement language is in NP - Ex: VALID ={formula is always true}
Dovetailing
k = 1 for i = 1 to k { - simulate M on the i-th string in sigma*, w_i, for k steps - if M accepts, stop and accept k = k + 1 countably infinite parallelism, so we are going through the first k strings in sigma* and checking each for a certain number of steps, and then checking each again plus one additional string with one additional step each
truth assignments
truth assignment p = assignment of 0/1 values to variables that leads to p(phi) = value of formula phi under truth assignment p
For all x, K(w) ≤ K_p(x) + c
where p is the description language - on input w, output p(w) (constant will have length |<M>|) -d_p(x) is the minimal program of language p that outputs x
there exists an integer c such that for every x: K(x) ≤
|x| + c - if you let c = |<M>| then |d(X)| ≤ |<M, x>| - there is always the encoding that is just the encoding of M plus length of x
NP problems
- CLIQUE = {<G, k> | G is an undirected graph and contains a clique of size k} - HAMPATH = {<G, s, t> | G is a graph with a ham path from s to t} - COLOR = {<G, k> | G can be colored using k colors} - SAT = {<phi> | phi is a satisfiable boolean formula} (phi with k variables would need to check through 2^k assignments, exp) - INVALID = {<phi> | for some truth assignment, the formula is false} - CNF-SAT = {<phi> | phi in CNF check if satisfiable} - 3SAT = {<phi> | phi in 3CNF check if satisfiable}
CLIQUE to INDEP
- G has a clique of size k iff G' has an independent set of size k' - G' = complement of G Algorithm for CLIQUE assuming that R solves INDEP - Given undirected graph G = (V, E) and k, 1. construct G' = (V, E') such that if an edge is in G it is not in G' and vice versa 2. check if G' has an independent set of size k using R (G, k is a clique iff G', k is indep set)
Max-Cut approximation
- Given G, find the largest cut - LocalSearch: find all vertices such that moving them to the other side of the cut increases the size of the cut and swap - finds a cut at least half the max-size cut = 2-approximation algorithm Proof: 1. poly-time: each swap increases cut size by at least 1. Cut cannot get larger than n^2, so algorithm halts after that long 2. cut size: at final cut, number of cut edges adjacent to each vertex is ≥ number uncut edges (more cut than uncut). So, cut(S,T) ≥ our cut/2
undecidable but recognizable languages
- HALT_TM = {<M, w> | M is a TM and M halts on w} - A_TM = {<M, w> | M is a TM that accepts w} - NE_TM = {<M> | M is a TM and L(M) is non-empty} (Recognizable: infinitely but countably many choices for w, the execution on each choice may not halt --> DOVETAILING. Undecidable: - NE_LBA = {<M> | M is an LBA and L(M) is non-empty} (construct LBA M' with input T U Q U {#} such that M' accepts string x exactly when x encodes accepting execution of M on w) - PCP (can explore all possible solutions, so search will find it if it exists)
unrecognizable languages
- L_d = {w_i | w_i is not in L(M_i)}(languages constructed via diagonalization where we have fixed enumerations of all strings and Turing Machines encoded as infinite vectors --> given input w, if w is the ith string enumeration of sigma* accepts w if the ith machine Mi does not accept w) - NA_TM = {<M, w,> | M is a TM and does not accept w} (proof by undecidability of complement) - E_TM = {<M> | M is a TM and L(M) is empty} (proof by recognizability of complement NE_TM) - EQ_TM = {<M, M'> | M and M' are TMs and L(M) = L(M')} (reduce from E_TM) - ONE_TM = {<M> | M is a TM and L(M) is a singleton set, accepts exactly 1 string} - REG_TM = {<M> | M is a TM and L(M) is regular/there exists a DFA that also accepts L(M)} (done by reducing NA_TM. Find an M' such that language is regular iff M does not accept w. Execute M on w. If M accepts w, make language non-reg by only accepting if palindrome. So if M accepts w, language is regular) - L = {<M> | M accepts at most 2 strings}
a language belongs to class P if there exists a deterministic halting TM such that:
- M decides L - time complexity of M is O(n^k) for some k (deterministic programs and DTMs are equivalent with time complexities differing by at most polynomial factor)
To prove a language is undecidable:
- Reduce to it - use closure properties
NP-complete problems
- SAT - INVALID - VERTEXCOVER = {<G, k> | G is a graph with a vertex cover (subset of vert s.t. every edge is adjacent to vertex in S) or size k} - MIN-VC (gave approx alg) - MAX-CUT given G, find largest cut
language L belongs to EXPTIME if there exists
- a deterministic TM M for L with time complexity O(exp(n^k)) - all NP are in EXPTIME - if complement of L is in NP then L is in EXPTIME
LBA
- computation power greater than regular languages, but some analysis questions are decidable - only uses first k cells, where k is the size of w (restricted memory) - on blank, must move left - palindromes, primes, as well as all regular languages - configuration given by: tape content, state of M, position of head --> |w| = k, |Q|.k.|T|^k (so bounded number of configs) - decidable A_LBA bc you can reject if it doesn't accept within ^ many steps - decidable does string encode a TM execution
encoding COLOR as SAT
- construct a boolean formula whose satisfying assignments encode assignment of 3 colors to nodes - x_ij stands for node i assigned color j, with n nodes and k colors - each clause has k literals, where we want the graph to be k colorable - each clause has one non-negated literal
Reduction proof
- construct a new machine M' from M and w - description of M and w is "baked" into description of M' - adding new states/transitions to the description of M like a wrapper - we're not checking L(M)
d(x)
- d(x) = minimal description of x - shortest string <M, w> such that M is a TM and when M is run on input w, M halts with x on its tape - shortest program description such that is outputs x - this length is the Kolmogorov complexity
SAT to INVALID
- design algorithm for SAT using algorithm R that solves INVALID - given boolean formula phi, check if R accepts ~phi. - if so (complement of the formula can be false), accept, else reject (there is a satisfying assignment for one formula if there is a way to make opposite of the formula false)
if f is a computable property that holds for almost all strings, then for any b > 0, f is false on only finitely many strings that are incompressible by b
- extremely unlikely to be random if there is an identifiable structure! M = on input i, a binary integer, find the ith string s where f(s) = false - output s - can be used to get short descriptions of strings that don't have property f - for string x that doesn't have f, K(x) ≤ n - b (so every x that fails to have property f is compressible by b)
3SAT to IndepSet
- independent sets of size k correspond to satisfying assignment of formula - node for each literal (3k nodes where there are k clauses) and edges between literals of same clause and between conflicting literals in different clauses - G has an indep set with k nodes iff the boolean formula is satisfiable - set the literal chosen from each clause to true - size of G is polynomial in the size of the formula so 3SAT <p IndepSet
K(x) = |d(x)|
- length of minimal description - optimal compression of string x - amount of info x contains - K(x) is never much larger than |x|
language L belongs to NP if there exists a polynomial time (time complexity of M is O(n^k)):
- nondeterministic TM M that decides L - OR deterministic TM V such that w is in L iff V accepts <w, c> for some certificate c
CNF-SAT to 3SAT
- replace clauses with 3-literal clauses without changing the satisfiability - to get up to 3, just duplicate literals - to get down to 3, eliminate one at a time by making a new variable that comes along with the clauses that make sure the replaced variables are satisfied: x' replaces z | w: add (x' | ~z), (x' | ~w), (z | w | ~x') --> either x' = 1 and either z = 1 or w = 1, OR x' = 0, and either z = 0 or w = 0
uncountable sets
- set of all real numbers - set of all subsets of N - set of all languages over sigma (sigma* is countable and infinite, but each language can also be an infinite set, so the set of languages is like the power set, so 2^N, like set of all subsets)
VERTEX-COVER is NP-Complete
- show in NP (the cover = certificate) - 3SAT <p VERTEX-COVER - make every literal a node and connect literals from a clause by edges - make a additional variable vertex for each variable and connect these by an edge - any vertex cover needs to choose one variable vertex for each variable and at least 2 clause vertices for each clause - the cover k = 2(m# clauses) + L#variables ==> suppose phi is satisfiable. Let p be satisfying assignment. Add L variable vertices to the cover corresponding to true literals in p. Choose a true literal in each clause. Add the other 2 clause vertices - Every edge in variable and clause pieces covered, and edges between the variable vert and clause vert covered since clause vert are chosen when variable vert are not <== suppose there is a vertex cover of size k = L + 2m. Must include exactly 1 vertex from each variable gadget and 2 from each clause gadget. Let p be the truth assignment corresponding to literals chosen in variable gadgets. p is sat. every edge between clause and variable gadgets is covered, but only 2 are covered by clause vert (every clause contains a true literal)
To show a language is unrecognizable:
- show that an unrecognizable problem reduces to it (NA_TM, E_TM) - show that the complement language is recognizable but undecidable (remember you're only trying to contradict to say that it is recognizable, so you can't assume that the TM R for the problem you're trying to recognize will halt; should be if R accepts, T accepts)
NP is closed under:
- union - concatenation - Kleene* - intersection - NOT complement
Recognizable languages are closed under:
- union (Construction by running two machines in parallel (on 2 tapes), executing one transition of each one after another Execute one transition of M1 on Tape1; If M1 stops and accepts, stop and accept; Execute one transition of M2 on Tape2; If M2 stops and accepts, stop and accept) - intersection (run one and then run the other, doesn't matter if 1st doesn't terminate because we need both machines to accept w: Given input w, execute M1 on w...) - concatenation (split input in all possible ways and run for n = 1 step, then increment n and do it again: on input w with length k: for i= 0 to k, execute M1 for n steps on prefix of w with i symbols. If M1 accepts, execute M2 on suffix of w with k - i symbols for n steps. if it accepts, stop and accept. otherwise increment i until k and then increase the number of steps you take)
Show a language is recognizable-but-undecidable
1. show recognizable by finding a TM that accepts it 2. show it is undecidable by reducing a problem to it (NE_TM, A_TM)
CNF
AND of clauses (OR of literals, which are variables or negated variables)
2CNF
AND of clauses that each have 3 literals (x | y | ~z) & (z | u | w)...
Problems about DFAs
All decidable (since DFAs halt) - A_DFA - E_DFA - EQ_DFA (intersection of L1 and ~L2 is empty) - universality (no path to rejecting state) - all of these can be done in polynomial time: all problems about DFAs are in P
K(x)
Almost every string x has K(x) > 0.999999 |x| K(x) is not a computable function. There is a constant L such that for every x, K(x) > L is not provable!
K(x) is not computable (no program can figure it out)
Assume for contradiction it is computable --> make TM M s.t. on input <n>, for each string x, test if K(x) = n - generates strings of complexity n - so we could always choose n to be large enough to not be written down as compression
Turing Machine configuration
C = (u, q, v) u, v are strings q in Q Next(C) is configuration resulting from executing 1 transition of M Executing M on w is unique sequence of configurations C_0, C_1, C_2... 1. C_0 = (epsilon, q_0, w) is initial config 2. For each i, C_i+1 = Next(C_i) 3. Either sequence is infinite or C_k is halting config - check those things if given a string and want to know if it encodes an accepting execution of M on w
To show A <p B
Give a poly-time algorithm for A using a poly-time algorithm for B - given an input w for A, show how to construct in polynomial-time f(w) such that w is in A if and only if f(w) is in B
poly-time reduction COLOR to SAT
Given G with n nodes and k colors, consider variables: x_ij for i = 1...n, and j = 1...k For each vertex i let phi_i = (x_ij &...& ~x_ik) |...| (~x_i1 &...&x_ik) phi = phi for all nodes & now for each edge (i, j) let psiij = ~(x_i1 & x_ j1) &...& ~(x_ik & x_ jk) Do and over all formulas for edges. If psi and phi are true then each vertex has exactly 1 color and vertices connected by an edge don't have the same color
There exists an integer c such that for every x: K(xx) ≤
K(x) + c - repeating as string does not substantially increase its complexity - let M be TM such that on input <N, w> where N is a TM, run N on w until halts and outputs s, then output ss. Let c = |<M>| so K(XX) ≤ |<M, d(x)>| ≤ c + K(x)
A string x is c-compressible if K(x):
K(x) ≤ |x| - c - shorter by c bits - otherwise, it is incompressible by c - if K(x) ≥ |x| then it is incompressible, or Kolmogorov random
A Turing Machine consists of
M = (Q, q_0, q_a, q_r, T, sigma, _, delta) finite set Q of states initial state q_0 accepting state q_a rejecting state q_r tape alphabet T input alphabet sigma blank _ transition function delta
correctness proof for DFAs
Prove by induction on string w IH: Assume that the claim holds for an arbitrary string x. That is, assume: ∂*(q_0, x) = q_0 if x does not end in "a"... q_1 if q_2 if q_3 BC: We want to prove the claim for w = E. ∂*(q_0, E) = q_0 since it does not end in "a"... IH: We want to prove the claim for w = x.s, where s is a symbol in ∑. ∂*(q_0, x) can be in q_0, q_1, q_2, or q_3 and s can be a or b, so we have 8 cases. Case I: δ*(q0, x) = q2 and s = b. By induction hypothesis, x ends in "ab" and does not contain aba. To prove: δ*(q0, x.b) = q0. So, δ*(q0, x.b) = δ(q2, b) = q0, by definition of δ* and δ for the machine. Intuitively, adding another b will not cause states q1, q2, or q3 since before, w will not end in "ab" or "a" since x ended in "ab", and adding another "b" will not create the substring aba.
Every TM can be encoded as a string -->
Set of all strings can be enumerated (set sigma* is countable), so set of all TMs can be enumerated, so all recognizable languages can be enumerated; but set of all languages over an alphabet is uncountable, so many languages must be unrecognizable
MIN-VC approximation
c-approximation algorithm for MIN-VC given G as input, outputs subset of vertices S such that S is a vertex cover of G and |S| ≤ c*OPT(G) - OPT(G) is the size of the smallest vertex cover of G (want small approximation factors) - our solution of adding both vertices from an edge will never find OPT, but is a poly-time 2 approximation algorithm for MIN-VC - every edge in H, our solution, must be covered by a different vertex
countable sets
can enumerate elements such that each appears at least once - every finite set is countable - set of natural numbers is countable - set of integers is countable - set of all strings sigma* - set of all Turing Machines (since each can be encoded as a string) - set of recognizable languages
prove a regular language is closed under an operation
consider a DFA M for language L - show how to construct NFA/DFA M' which accepts a string w iff can be obtained by applying the operation to a string accepted by M - Ex: complement - flip accepting and rejecting states - can construct NFA M' from other NFA (keep in mind E transitions to old states)
pairwise distinguishable strings
consider a set of strings S = {pattern, i.e. 10, 110, 1110, ...} show they are pairwise distinguishable by taking two arbitrary strings from the set and concatenating the same string onto the ends of both so that only 1 is in the set.
Simulating NTM by DTM (3 tape) causes ____
exponential slowdown
time complexity of NTM = nondeterministic programs
f(n) if on every input of length n, every execution path terminates after ≤ f(n) transitions (depth of tree) - possible execution paths = exp(f(n)) - poly-time NTM = O(n^k)
For any n it is impossible to prove K(x) ≤
n
If a language L and its complement ~L are both recognizable
then both must be decidable (you can combine the two recognizers, since recognizable languages are closed under complement, and run them in parallel, each one step at a time, and if the machine for L accepts then accepts, and if the machine for ~L accepts, then reject --> decidable because they are recognizers and one of them will accept a string)
For every n, there is an incompressible string x:
with length n - there are 2^n binary strings of length n. - we want to be able to compress all so that they can have length < n - but there are only 2^n - 1 strings of length < n since if we sum over 2^i to n - 1 we get this
Pr[x is c-compressible] = Pr[K(x) < |x| - c]
≤ (1/2)^c
