Alg Des Ch08: Dynamic Programming

What are the 3 steps involved in solving a problem by dynamic programming?

1) Formulate the answer as a recurrence relation or recursive algorithm; 2) Show that the number of different parameter values taken on by your recurrence is bounded by a (hopefully small) polynomial; and 3) Specify an order of evaluation for the recurrence so the partial results you need are always available when you need them.

What is the Minimum Weight Triangulation problem?

A Triangulation of a polygon is a set of nonintersecting diagonals that partitions the polygon into triangles. Every edge belongs to one triangle. The Minimum Weight Triangulation is the the Triangulation that minimizes the sum of the diagonal lengths.

What is a Context-Free Grammar?

A set of recursive rules for defining valid sequences of strings. Production Rules (aka Productions, or Rules) are mappings from Symbols to Strings. The Strings can either be Non-Terminal, meaning they are composed of more Rules, hence the recursion, or Terminal, meaning they are just a sequence of characters.

How can dynamic programming be used to calculate binomial coefficients?

Build up Pascal's Triangle, where n choose k is M[n,k] = M[n-1, k-1] + M[n-1, k]. Basis cases are when k = 0 and k=n. Note that there is no recursion and partial results are stored.

Are recursive algorithms computed in a depth-first or breadth-first manner?

Depth-first.

What does it mean that we assume the rules of our Context-Free Grammar are in Chomsky Normal Form?

Every Rule maps a Symbol to a string of either one Terminal or exactly two Nonterminals. Any Context-Free Grammar can easily be transformed into Chomsky Normal Form, so there is no loss of generality with this assumption.

What is a simple way to get most of the benefits from dynamic programming in the context of a recursive program that often requires a little extra programming? Example?

Explicitly caching results from recursive calls. Naively calculating Fibonacci numbers is exponential in time, while caching the results uses linear space and is linear in time.

What is the Integer Partition without Rearrangement Problem?

Given a sequence of non-negative numbers, how can they be partitioned into k or less ranges, without rearrangement, such that the maximum sum over any range is minimized.

What is the Longest Increasing Sequence problem and what is the recurrence relation used in a dynamic programming solution?

Given a sequence of numbers, we seek the longest monotonically increasing subsequence. The subsequence need not be a run, meaning the numbers do not have to be adjacent. If Li is the length of the longest such sequence ending with element Si, then it is equal to 1 plus the maximum of Lj for all j<i, requiring that Sj<Si. L0 = 0. Note that the longest increasing sequence may not be Sn, since it may not contain the last number. Rather it is the maximum Li over all i.

If D[i,j] is the edit distance between the portions of string P and string T up to indices i and j respectively, how can this be expressed recursively?

It is the minimum of 3 different possibilities. 1) D[i-1,j-1] (+ 1 if the tail elements are not the same), corresponding to a match or a substitution; 2) D[i,j-1] + 1, indicating that P was missing an element that we can insert; or 3) D[i-1,j] + 1, indicating that P had an extra element that we can delete. Note that the only way that D doesn't increase is if the tail elements are the same.

What does it mean to parse a string S according to a given Context-Free Grammar G?

It means to construct a Parse Tree of rule substitutions that express S as a single Non-Terminal symbol in G.

Generally speaking, how does dynamic programming ensure both correctness and efficiency?

It systematically searches all possibilities to guarantee efficiency, and stores the consequences/results of subproblems in a results matrix to save computational costs and improve efficiency.

What is the recurrence relation for parsing a string S according to a given Context-Free Grammar using Dynamic Programming? What is the Big Oh complexity?

M[i,j,X] is a boolean function that is True if string S[i,j] can be generated by symbol X. If X generates a terminal string, then it is just a check to see if S[i,j] is that string. Otherwise, X generates two rules YZ, which will be applied to a left and right portion of S[i,j], separated at position k. In other words, for fixed k, M[i,j,X] = M[i,k,Y] AND M[k+1,j,Z], where an overall OR is taken for each position K. There are 3 parameters, but the size of the Grammar is fixed. So we can ignore X. i, j can each take on n variables (actually since j>=i we have n(n+1)/2), and each involves scanning k over at most n numbers. So O(n^3).

What is the running time for solving the Integer Partition without Rearrangement Problem using Dynamic Programming? How can it be improved?

M[k,n] is computed for all possible k and n, so that is a k by n matrix. For each element, the cost function is computed for up to n different potential divider positions, and each one involves a sum of up to n elements for the right-most partition. That makes O(kn^3). However, we can pre-calculate the partial sums for O(kn^2).

What is the cost function and recurrence relation used to solve the Integer Partition without Rearrangement Problem?

M[n,k] is the minimum possible cost over all partitionings of the first n numbers into k ranges, where that cost is the largest sum over any partition. We express this in terms of the placement of the right-most divider at position i. The cost will be the larger of either the right-most partition, which is the sum of elements from i+1 to n, or M[i, k-1]. This is minimized over all possible placements i. Basis cases are when either k or n equals 1. M[1,k] = s1 and M[n,1] = sum from 1 to n.

How can dynamic programming be used most efficiently to calculate Fibonacci numbers.

Remove recursion by calculating F(k) for increasing k until k reaches n. Furthermore, only the previous 2 numbers need to be stored. This will be linear in time and constant in space.

What tradeoff underlies dynamic programming?

Space for Time. Dynamic programs take up more space to store previous results that likely will have to be recomputed. If these recomputations are slow, then significant time can be saved by just looking them up.

What property of combinatorial problems ensures that Dynamic Programming can produce a correct algorithm?

The Principle of Optimality, which means that partial solutions can be optimally extended with regard to the State after the partial solution, instead of the specifics of the partial solution itself. I.e., future decisions are made based on the consequences of previous decisions, not the actual decisions themselves.

What is the name of the dynamic programming algorithm for approximate string matching?

The Wagner-Fischer algorithm.

What does n choose k mean?

The number of different ways of picking k items from a group of n items, with order not counting. Or phrased differently, it's the number of subsets of size k from a set of n distinct items.

What is the recurrence relation for finding the Minimum Weight Triangulation of a polygon with n points using Dynamic Programming? What is the Big Oh complexity?

The points are numbered sequentially around the polygon. T[i,j] is the weight of the triangulation covering the portion of the polygon between points i and j, not including the length of the edge directly between i and j to avoid double counting. If we find an additional vertex k that completes the i,j,k triangle, this divides the polygon into two new triangles. Therefore we can write T[i,j] = T[i,k] + T[k,j] + d(i,k) + d(k,j), where we take a minimum over all k between i and j. The basis case is if j = i+1, meaning they are sequential neighbors, where T[i,i+1] = 0. The final triangulation is given by T[1,n]. The matrix T[i,j] can be built up for increasing gap sizes between i and j. There are n choose 2 combinations of i and j, and each involves scanning for the minimum k between i and j, for O(n^3) time and O(n^2) space.

How can dynamic programming be used to approximately match two string?

The recurrence relation for the edit distance can be used to construct a matrix of edit distances for every possible pair of indices along P and T. The optimal matching will be a path from the top left to the bottom right of the matrix, and can be reconstructed by either explicitly recording the parent of each cell during construction, or by working back from the top with a bit of computation.

What problem structure attribute often makes dynamic programming a good choice?

When the combinatorial objects involved have an inherent left-right order among components, such as character strings, leaves in rooted trees / search trees, points around a polygon, integer sequences, and elements of a permutation.

What is the recurrence relation for the binomial coefficients? What is the intuition?

n choose k = n-1 choose k-1 plus n-1 choose k. These cover the two cases of whether the nth element is included in the subset. If so, then you have to choose k-1 elements from the n-1 remaining. If not, then you have to choose all k from the n-1. Every possible subset satisfies exactly one of these 2 conditions.