CS 430 Ch 2 and Ch 3.1-3.2
Recursive property
(n/k)=((n-1)/(k-1)) + ((n-1)/k) 0 < k < n (n/k) = 1 k = 0 or k = n
Binomial Coefficient
(n/k)=n!/k!(n-k)! Divide-and-conquer Recursive Very inefficient - like recursive Fibonacci number of terms computed by recursive binomial 2(n/k)-1
Worst-Case Complexity Analysis of Quicksort
- Array is repeatedly sorted into an empty sub-array which is less than then pivot and a sub-array of n-1 containing items greater than pivot - If there are k keys in the current sub-array, k-1 key comparisons are executed - Array is sorted in non-decreasing order - T(n) is specified because analysis is for the every-case complexity for the class of instances already sorted in non-decreasing order - T(n) = time to sort left sub-array + time to sort right sub-array + time to partition - T(n) = T(0) + (T(n-1)+n-1 - T(n) T(n-1) + n - 1 for n > 0 - T(0) = 0 - From B16 in Appendix B - T(n) = n(n-1)/2
Quicksort
- Array recursively divided into two partitions and recursively sorted - Divisions based on pivot - Pivot divides the two sub-arrays - All items < pivot placed in sub-array before pivot - All item >= pivot placed in sub-array after pivot
Worst-case Quicksort
- At least n(n-1)/2 - Use induction to show it is the worst case 1. W(n)<=n(n-1)/2
Establish a recursive property and solve bottom-up
- B[i,j] = B[i,j] = B[i-1, j-1] + B[i-1, j] where 0 < j < i - B[i,j] = 1 where j = 0 or j =1 - Solve an instance of the problem bottom up - compute rows in B in sequence starting with row 1 - At each iteration, the values needed for that iteration have already been computed
Dynamic Programming
- Bottom-up approach to problem solving - Instance of problem divided into smaller instances - Smaller instances solved first and stored for later use by solution to solve larger instances - Look it up instead of re-compute - Iterative solution to the Fibonacci Sequence -Storage has to be reserved
Basic Operation of Quicksort
- Comparison of S[i] with pivot item - Input size n
Establish a recursive property for shortest path
- D^k[i,j] = minimum (case 1, case 2) = minimum (D^(k-1)[i,j], D^(k-1)[i,k] + D^(k-1)[k,j]) Case 1: At least one shortest path from vi to vj uses only vertices in set {v1, v2, ..., Vk} as intermediate vertex does not use vk - D^k[i,j] = D^(k-1)[i,j] Case 2: All shortest path from vi to vj uses only vertices in set {v1, v2, ..., vk} as intermediate vertex does use vk - Path = vi, ..., vk, ..., vj where vi,..., vk consists only of vertices in {v1, v2, ..., vk-1} as intermediates: Cost of path = D^(k-1)[i,k] - And where vk, ..., vj consists only of vertices in {v1, v2, ... vk-1} as intermediate: Cost of path = D^(k-1)[k,j]
Step to develop a dynamic programming algorithm
- Establish a recursive property that gives the solution to an instance of at he problem - Compute the value of an optimal Solution in a bottom-up fashion by solving smaller instances first
Dynamic Programming Steps for Shortest path
- Establish a recursive property to compute D^k from D(k-1) - Solve an instance of the problem in bottom-up fashion by repeating the process for k = 1 to n
Floyd's Algorithm for Shortest Paths - Algorithm 3.3
- Every-case time complexity - Basic operation - instructions in the for j loop - Input size: |V| - T(n) = n^3
Brute Force
- For every vertex, determine lengths of all paths from that vertex to every other vertex and compute minimum lengths - Complete Graph G: (n -2)!
Divide and Conquer General method
- Given a function to compute on n inputs, split the inputs into k distinct subsets 1 < k <= n yielding k subproblem. (most common is 2) - These subprograms must be solved and then a method must be found to combine solutions into a solution of the whole - If the subprograms are still relatively large, then the divide-and-conquer strategy may possibly be reapplied. - Often the subproblems resulting from a divide-and-conquer design are of the same type as the original problem, making this naturally expressed by a recursive procedure. - Smaller and smaller ____ of the same kind are generated, eventually producing subproblems that are small enough to be solved without _____.
Binary Search
- Locate key x in an array of size n sorted in non-decreasing order - Compare x with the middle element - if equal, done - quit. Else 1. Divide the array into two sub-arrays approximately half as large - If x is smaller than middle item, select left sub-array - If x is larger than the middle item, select right sub-array 2. Conquer(solve) the sub-array: Is x in the sub-array using recursion until the sub-array is sufficiently small? 3. Obtain the solution to the array from the solution to the sub-array
Merge
- Merges the two arrays U and V created by the recursive calls to mergesort - Input size 1. h the number of items in U 2. m the number of items in V - Basic Operation: 1. Comparison of U[i] to V[j]
Optimization Problem
- Multiple candidate solutions - Candidate solution has a value associated with it - Solution to the instance is a candidate solution with an optimal value - minimum/maximum
Shortest Path
- Optimization Problem - Candidate Solution: path from one vertex to another - Value of candidate solution: length of the path - Optimal Value - minimum length - Possible multiple shortest paths
Mergesort
- Sort an array S of size n (for simplicity, let n be a power 2) - Divide S into 2 sub-arrays of size n/2 - Conquer(solve) recursively - each sub-array until array is sufficiently small(size 1) - Combine merge the solutions to the sub-arrays into a single sorted array (sort them as it returns) Check Figure 2.2
Distributive Sorts
- Sorts by examining entries one digit at a time - Bucket Sort/Radix Sort examining the least significant digit of the value to be sorted first and then assign to the appropriate bucket dependent on the value of the digit. After all items have been distributed, the buckets are merged in order and then the process is repeated using the next place value. - Radix Exchange Sort us the values expressed in binary digits. Usually require time proportional Nlogp(K) where p is the radix of the number system employed and K is the maximum size of the values. - Sensitive to the distribution of magnitudes - Require considerable additional storage.
Comparative Sorts
- The major activity within the sorts is to compare two values with the list to determine action 1. Adjacent values (bubble sort or Interchnge sort) 2. Values a distance d apart (shell sort) - Can take a time from proportionally N^2 down to roughly proportional to Nlog2N Insensitive to the distribution of magnitudes - Take good advantage to any natural order in the data - Require no extra storage (other than possibly one temporary)
divide and conquer
- Top-down approach to problem solving - blindly divide problem into smaller instances and solve the smaller instances - Technique works efficiently for problems where smaller instances are unrelated - Inefficient solution to problems where smaller instances are related - Recursive solution to the Fibonacci sequence - Divide-and-conquer should not be applied to Fibonacci because Fibonacci is too small.
Address Calculation Sorts
- Usually operate by transforming a value to be sorted into an address close to where the symbol is expected to end up. - This can be one of the fastest type of sorts if enough storage space is available. - The time is proportional if it were known that no two values would be assigned the same address/slot (collision) - Often the storage is made to be double the needed size for the values. - Hashing is another name for address calculation
Average-Case Time Complexity of Quicksort
- Value of pivotpoint is equally likely to be any of the numbers from 1 to n. - Average obtained is the average when every possible ordering is sorted the same number of times - An = p = 1n1n(Ap - 1 + An - p + n -1 ) A(n) = n: summation p=1 1/n(A(p-1) + A(n-p))+ n-1 - 1/n is the probability pivot point is p - Average time to sort subarrays when pivotpoint is p: [A(p-1)+A(n-p)] - Time to partition: n - 1 -Solving equation and using Theorem B22: A(n) in element if theta(nlogn)
Worst-case time complexity Analysis of Mergesort
- W(n) = time to sort U + time to sort V + time to merge - W(n) = W(h) + W(m) + h + m +1 (minus one for not comparing the n again) - First analysis assumes n is a power of 2 1. h = ⌊ n/2 ⌋ = n/2 2. m = n - h = n - n/2 = n/2 3. h + m = n/2 + n/2 = n - W(n) = W(n/2) W(n/2) + n - 1 - = 2W(n/2) + n-1 for n>1 and n a power of 2 - W(1) = 0 - From B19 in Appendix B - W(n) = nlog2n - (n-1) ∈ θ(nlog2n)
Adjacency Matrix W
- W[i,j] = Weight of the path from vi -> vj if there is an edge - W[i, j] = infinity if there is no edge from vi -> vj - W[i, j] = 0 if i = j see figure 3.3
Algorithm 3.2 Binomial Coefficient using Dynamic Programming
- bin2 - The work done by bin2 as a function of n and k - bin in element of theta(nk)
Worst Case Complexity Analysis of binary search
- n is a power of 2 and x > S[n] - W(n) = W(n/2) + 1 for n > 1 and n power of 2 - W(1) = 1 1. W(n/2) = the number of comparisons in the recursive call 2. 1 comparison at the top level - Example B1 in Appendix B: 1. W(n) = log2n+1 - n not a power of 2 - W(n) = ⌊ log2n ⌋ + ∈ θ(log2n)
Worst-Case Analysis Mergesort
- n not a power of 2 - W(n) = W(⌊ n/2 ⌋) + W(ᴦ n/2 ך) + n-1 - From Theorem B4: - W(n) ∈ θ(nlog2n)
Dynamic Programming Solution to the all-pairs shortest path
- n vertices in the graph - Create a sequence of n+1 arrays D^k wher e0 <= k <=n - D^k[i,j] = length of a shortest path from vi to vj using only vertices in the set {v1, v2,...,vk} - D^n[i,j] = length of the shortest path from vi to vj - D^0 = W and D^n = D see figure on the book
pascal
1 1 1 1 2 1 1 3 3 1 1 4 6 4 1 1 5 10 10 5 1 1 6 15 20 15 6 1 ......
When developing a recursive algorithm for a problem:
1. Develop a way to obtain the solution to an instance from the solution to one or more smaller instances 2. Determine the terminal condition(s) that smaller instance(s) is (are) approaching. 3 Determine the solution in the case of the terminal condition(s).
Where did the idea divide and conquer came from?
Battle of Austerliz, December 2 1805, Napolean split the Austro-Russian Army and was able to conquer 2 weaker armies.
Sorting algorithms categories:
Comparison Sorts Distributive Sorts Address Calculation Sorts
When not to use divide and conquer
Divide and conquer algorithms should be avoided in the following cases: 1. An instance of size n is divided into 2 or more instances each almost of size n. 2. An instance of size n is divided into almost n instances of size n/c, where c is constant Neither of these is acceptable for large values of n
Hashing Function characteristics
Easy to compute (low overhead Avoid collisions Make use of the entire value being stored
Other Distributive Sorts
Flashsort Postmansort Smaplesort
Hashing Function Examples
Folding: Divide the value into several parts, (fold) add the parts and use the last number of digits for the address. Additive: Add each digit or character as they are stored internally and then use the modulo operator to determine what slot or address to place the value Modulo: Simply take the value to be stored and modulo by the number of buckets or slots
Weighted Directed Graph
Graphs that has arrows that direct the directions of the edges
Other Comparitive Sorts
Heapsort Mergesort Quicksort Insertion Sort Selection Sort
Worst case of Mergesort:
Loop exited with one index at exit point and the other one less than the exit point
Every-case Complexity Analysis of Partition
T(n) = n - 1
What kind of approach is divide-and-conquer?
Top-down approach, solution to a top-level instance of a problem is obtained by going down and obtaining solutions to smaller instances.
Dynamic Programming Solution to the Binomial Coefficient Problem
Using the recursive property, construct an array B containing solutions to smaller instances Construct Array B such that B[i,j]=(i/j)
