COP3337 Module 8

Runtime analysis: Finding the max value.

https://drive.google.com/file/d/1dVrIw1XYc8RxeIPVOpml9CzQ1cMg7Oh8/view?usp=sharing

Growth rates for different input sizes.

https://drive.google.com/file/d/1oT4kZMokhoeABehm2D0ObzoFSkCkMSmd/view?usp=sharing

Linear search algorithm.

import java.util.Scanner; public class LinearSearch { public static int linearSearch(int [] numbers, int key) { int i; for (i = 0; i < numbers.length; ++i) { if (numbers[i] == key) { return i; } } return -1; /* not found */ } public static void main(String [] args) { Scanner scnr = new Scanner(System.in); int [] numbers = {2, 4, 7, 10, 11, 32, 45, 87}; int i; int key; int keyIndex; System.out.print("NUMBERS: "); for (i = 0; i < numbers.length; ++i) { System.out.print(numbers[i] + " "); } System.out.println(); System.out.print("Enter a value: "); key = scnr.nextInt(); keyIndex = linearSearch(numbers, key); if (keyIndex == -1) { System.out.println(key + " was not found."); } else { System.out.println("Found " + key + " at index " + keyIndex + "."); } } }

Runtime analysis of nested loop: Selection sort algorithm.

https://drive.google.com/file/d/1F7WRgad3CaSpF5f8KaLrGCRAoAcKo4Fo/view?usp=sharing

Runtime complexities for various pseudocode examples.1

https://drive.google.com/file/d/1g9bGbHp3U9bebHEOq_uOXQUHxCtlmehN/view?usp=sharing

Linear search algorithm checks each element until key is found.

https://drive.google.com/file/d/1u-wEHrcW1By1oA73zy3plvggiPqwD-YK/view?usp=sharing

Merge sort recursively divides the input into two halves, sorts each half, and merges the lists together.

https://drive.google.com/file/d/1upPucHuecRTGSJrAQEH95C9SPY2QoWFs/view?usp=sharing

Given list {20 14 85 3 9}, what value will be in the 0 element after the first pass over the outer loop (i = 1)?

14 Each outer loop pass inserts one element into the sorted part starting with the element at index 1. In this case, 14 is inserted at index 0, and 20 is moved to index 1. The sorted part is {14 20} and the unsorted part is {85 3 9}.

Determine the simplified Big O notation.

10 · O(N^2) = O(10 · N^2) = O(N^2) 10 + O(N^2) = O(10 + N^2) = O(N^2) 3·N · O(N^2) = O(3·N·N^2) = O(3·N^3) = O(N^3) 2·N^3 + O(N^2) = O(2·N^3 + N^2) = O(N^3)

How many times longer will sorting a list of 500 elements take compared to a list of 50 elements?

100 500 / 50 = 250000 / 2500 = 100. Selection sort's runtime grows quadratically with the input size. If the input increases in size by X times, the runtime increases X times.

Merge sort runtime and memory complexity. How many recursive partitioning levels are required for a list of 2048 elements?

11 log2 2048 = 11

Given list {1 9 17 18 2}, how many swaps will occur during the outer loop execution (i = 4)?

3 2 will be swapped with 18, 17, and 9, after which the element is in the correct position and all elements are sorted.

Given list {18 23 34 75 3}, how many total comparisons will insertion sort require?

7 Iterations 1, 2, and 3 requires 1 comparison each, and iteration 4 requires 4 comparisons.

O notation

Big O notation is a mathematical way of describing how a function (running time of an algorithm) generally behaves in relation to the input size. In Big O notation, all functions that have the same growth rate (as determined by the highest order term of the function) are characterized using the same Big O notation. In essence, all functions that have the same growth rate are considered equivalent in Big O notation. Given a function that describes the running time of an algorithm, the Big O notation for that function can be determined using the following rules: If f(x) is a sum of several terms, the highest order term (the one with the fastest growth rate) is kept and others are discarded. If f(x) has a term that is a product of several factors, all constants (those that are not in terms of x) are omitted.

Binary search

Binary search is a faster algorithm for searching a list if the list's elements are sorted and directly accessible (such as an array). Binary search first checks the middle element of the list. If the search key is found, the algorithm returns the matching location. If the search key is not found, the algorithm repeats the search on the remaining left sublist (if the search key was less than the middle element) or the remaining right sublist (if the search key was greater than the middle element).

Which of the following Big O notations is equivalent to O(734·N)?

Constants in product terms are omitted. For O(734·N), the constant 734 is omitted, yielding O(N).

Which of the following Big O notations is equivalent to O(N+9999)?

For O(N+9999), N is the highest order term, yielding O(N)

Answer the following questions assuming each comparison takes 1 µs.

Given a list of 1024 elements, what is the runtime for linear search if the search key is less than all elements in the list? 1024 µs Linear search assumes the elements are not sorted and will check all elements if the key is not found. Thus, linear search's runtime is proportional to the number of list elements.

Insertion sort

Insertion sort is a sorting algorithm that treats the input as two parts, a sorted part and an unsorted part, and repeatedly inserts the next value from the unsorted part into the correct location in the sorted part.

Binary search

Linear search may require searching all list elements, which can lead to long runtimes. For example, searching for a contact on a smartphone one-by-one from first to last can be time consuming. Because a contact list is sorted, a faster search, known as binary search, checks the middle contact first. If the desired contact comes alphabetically before the middle contact, binary search will then search the first half and otherwise the last half. Each step reduces the contacts that need to be searched by half.

Which of the following Big O notations is equivalent to O(12·N +6·N^3 + 1000)?

O(12·N + 6·N^3 + 1000) = O(6·N^3) = O(N^3)

O(N + N^2) has a _____ runtime complexity.

O(N + N^2) reduces to O(N^2). The highest power is 2, so the runtime complexity is quadratic.

What is the big-O notation for the worst-case runtime? negCount = 0; for(i = 0; i < N; ++i) { if (numbers[i] < 0 ) { ++negCount; } }

O(N)

A linear search has a _____ runtime complexity.

O(N). A linear search may need to search all N elements within a list.

A selection sort has a _____ runtime complexity.

O(N^2)The total number of operations for selection sort is proportional to N · N.

What is the big-O notation for the worst-case runtime? nVal = N; steps = 0; while (nVal > 0) { nVal = nVal / 2; steps = steps + 1; }

O(log N) nVal is decreased by half each iteration, so the loop will execute log2 N times.

Quicksort

Quicksort is a sorting algorithm that repeatedly partitions the input into low and high parts (each part unsorted), and then recursively sorts each of those parts. To partition the input, quicksort chooses a pivot to divide the data into low and high parts. The pivot can be any value within the array being sorted, commonly the value of the middle array element. Ex: For the list {4 34 10 25 1}, the middle element is located at index 2 (the middle of indices 0..4) and has a value of 10.

What is the big-O notation for the best-case runtime? i = 0; belowMinSum = 0.0; belowMinCount = 0; while (i < N && numbers[i] <= maxVal) { belowMinCount = belowMinCount + 1; belowMinSum = numbers[i]; ++i; } avgBelow = belowMinSum / belowMinCount;

The big-O notation for an algorithm's best-case runtime represents the growth rate for the shortest execution. The shortest execute occurs if numbers[0] > maxVal, in which case the loop does not execute.

O(N log N) has a _____ runtime complexity.

The first N indicates the runtime complexity is linear, then multiplying with log N results in a log-linear runtime complexity.

Binary search algorithm.

import java.util.Scanner; public class BinarySearch { public static int binarySearch(int [] numbers, int key) { int mid; int low; int high; low = 0; high = numbers.length - 1; while (high >= low) { mid = (high + low) / 2; if (numbers[mid] < key) { low = mid + 1; } else if (numbers[mid] > key) { high = mid - 1; } else { return mid; } } return -1; // not found } public static void main(String [] args) { Scanner scnr = new Scanner(System.in); int [] numbers = {2, 4, 7, 10, 11, 32, 45, 87}; int i; int key; int keyIndex; System.out.print("NUMBERS: "); for (i = 0; i < numbers.length; ++i) { System.out.print(numbers[i] + " "); } System.out.println(); System.out.print("Enter a value: "); key = scnr.nextInt(); keyIndex = binarySearch(numbers, key); if (keyIndex == -1) { System.out.println(key + " was not found."); } else { System.out.println("Found " + key + " at index " + keyIndex + "."); } } }

Determine the midpoint and pivot values.

numbers = {1 2 3 4 5}, i = 0, k = 4 3 midpoint = 0 + (4 - 0)/2 = 2 The midpoint is 2, so the pivot value is the element at numbers[2]. numbers = {55 7 81 26 0 34 68 125}, i = 3, k = 7 34 midpoint = 3 + (7 - 3)/2 = 5 The midpoint of index 3 and index 7 is 5, so the pivot value is the element at numbers[5].

Given list {10 20 6 14 7}, what will be the list after completing the second outer loop iteration (i = 2)? Use curly brackets in your answer.

{6 10 20 14 7} 6 < 20, so 6 and 20 are swapped. 6 < 10, so 6 and 10 are swapped. The sorted part is now {6 10 20}, and the unsorted part is {14 7}.

Determine the index j and the left and right partitions.

-numbers = {1 2 3 4 5}, i = 0, k = 4 2 j = (i + k) / 2 = (0 + 4) / 2 = 2 j is the midpoint in the list that divides the list into two halves. -numbers = {1 2 3 4 5}, i = 0, k = 4 1 2 3 Elements at indices from i to j, inclusive, are in the left partition. -numbers = {1 2 3 4 5}, i = 0, k = 4 4 5 Elements at indices from j + 1 to k are in right partition. -numbers = {34 78 14 23 8 35}, i = 3, k = 5 4 j = (3 + 5) / 2 = 8 / 2 = 4. As the recursive partitioning continues, partitions are continuously divided in half. -numbers = {34 78 14 23 8 35}, i = 3, k = 5 Elements at indices 3 to 4 are in the left partition. This partition will again be divided into two halves. Each of those halves will have one element. -numbers = {34 78 14 23 8 35}, i = 3, k = 5 35 Elements from indices 5 to 5 are in the right partition. This partition has 1 element, and is already sorted.

Given list {10 11 12 13 14 5}, how many comparisons will be made during the third outer loop execution (i = 3)?

1 Only a single comparison is needed for elements already in sorted order.

Which function best represents the number of operations in the worst-case? i = 0; sum = 0; while (i < N) { sum = sum + numbers[i]; ++i; }

2 operations execute before the loop. The loop condition and 2 operations in the loop executes N times, yielding 3N operations. The loop condition is checked 1 additional time before the loop terminating. The total operations as a function of N can be expressed as 2 + 3N + 1, or 3N + 3.

Merge sort runtime and memory complexity. 1) How many recursive partitioning levels are required for a list of 8 elements?

3 1st: {a b c d} {e f g h} 2nd: {a b} {c d} {e f} {g h} 3rd: {a} {b} {c} {d} {e} {f} {g} {h} For N elements, the number of levels is log2 N. log2 8 = 3.

Using the Big O runtime complexity, how many times longer will sorting a list of 20 elements take compared to sorting a list of 10 elements?

4 20 / 10 = 400 / 100 = 4. A list with twice as many elements requires 4 times as many comparison.

How many times longer will sorting a list of 20 elements take compared to sorting a list of 10 elements?

4 A list with twice as many elements requires 4 times as many comparisons.

In the worst case, assuming each comparison takes 1 µs, how long will insertion sort algorithm take to sort a list of 10 elements?

45 (1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9) * 1 µs = 45 µs

When sorting a list with 50 elements, indexSmallest will be assigned to a minimum of _____ times

49 The outer loop always assigns to indexSmallest at the beginning of each iteration. The inner loop never assigns to indexSmallest if the list is already sorted. A list of 50 elements causes 49 outer loop iterations, and thus a minimum of 49 assignments to indexSmallest.

Given list {10 11 12 13 14 7}, how many comparisons will be made during the final outer loop execution (i = 5)?

5 7 is the smallest element, and will be compared to and swapped with all other elements before reaching the correct position at index 0.

O(5) has a _____ runtime complexity.

5 does not change with the input size, so the runtime of O(5) remains constant.

Merge sort runtime and memory complexity How many elements will the temporary merge list have for merging two partitions with 250 elements each?

500 The temporary list must be large enough to merge all elements from the two partitions.

18.1 Searching and algorithms

An algorithm is a sequence of steps for accomplishing a task. Linear search is a search algorithm that starts from the beginning of a list, and checks each element until the search key is found or the end of the list is reached.

Assume quicksort always chooses the smallest element as the pivot.

Given numbers = {7 4 2 25 19}, i = 0, and k = 4, what is contents of the low partition? Use curly braces in your answer. {2} The low partition is {2}, and the high partition is {4 7 25 19}. How many partitioning "levels" of are required for a list of 5 elements? 4 1st: {a} {b c d e} 2nd: {a} {b} {c d e} 3rd: {a} {b} {c} {d e} 4th: {a} {b} {c} {d} {e} How many partitioning "levels" are required for a list of 1024 elements? 1023 In the worst case, for N elements, there will be N - 1 levels. How many total comparisons are required to sort a list of 1024 elements? 1047552 (1024 - 1) * 1024 = 1023 * 1024 = 1047552 comparisons.

Given list: { 4 11 17 18 25 45 63 77 89 114 }.

How many list elements will be checked to find the value 77 using binary search? 2 Binary search first checks the value 25 at index 4 ((0+9)/2 = 4), then searches the right sublist at index 7 ((5+9)/2 = 7) finding the value 77. How many list elements will be checked to find the value 17 using binary search? 3 Binary search first checks the value 25 at index 4 ((0+9)/2 = 4), then searches the left sublist at index 1 ((0+3)/2 = 1), and then searches the right sublist at index 2 ((2+3)/2 = 2), finding the value 17. Given an array with 32 elements, how many list elements will be checked if the key is less than all elements in the list, using binary search? 5 Binary search checks indices 15, 7, 3, 1, and 0. After checking index 0, the algorithm returns -1, indicating the key was not found.

Assume quicksort always chooses a pivot that divides the elements into two equal parts.

How many partitioning levels are required for a list of 8 elements? 3 1st: {a b c d} {e f g h} 2nd: {a b} {c d} {e f} {g h} 3rd: {a} {b} {c} {d} {e} {f} {g} {h} How many partitioning "levels" are required for a list of 1024 elements? 10 log2 1024 = 10. Assuming equal-size partitions, for N elements, the number of levels is log2 N. How many total comparisons are required to sort a list of 1024 elements? 10240 log2 1024 * 1024 = 10 * 1024 = 10240 comparisons. There will be log2 N partitioning levels, and each level requires N comparisons.

What is the big-O notation for the worst-case runtime? for (i = 0; i < N; ++i) { if ((i % 2) == 0) { outVal[i] = inVals[i] * i; } }

O(N). outVal[i] = inVals[i] * i; executes every other loop iteration, so the total number of operations include: 1 operation at the start of the loop, 3 operations every loop iteration, 1 operation every other loop iteration, and 1 operation for the final loop condition check.

Sorting: Introduction

Sorting is the process of converting a list of elements into ascending (or descending) order. For example, given a list of numbers {17 3 44 6 9}, the list after sorting is {3 6 9 17 44}. You may have carried out sorting when arranging papers in alphabetical order, or arranging envelopes to have ascending zip codes (as required for bulk mailings).

Algorithm analysis Worst-case algorithm analysis

To analyze how runtime of an algorithm scales as the input size increases, we first determine how many operations the algorithm executes for a specific input size, N. Then, the big-O notation for that function is determined. Algorithm runtime analysis often focuses on the worst-case runtime complexity. The worst-case runtime of an algorithm is the runtime complexity for an input that results in the longest execution. Other runtime analyses include best-case runtime and average-case runtime. Determining the average-case runtime requires knowledge of the statistical properties of the expected data inputs.

The complexity of the algorithm below is O(1). for (i = 0; i < 24; ++i) { if (timeHour < 6) { tollSchedule[i] = 1.55; } else if (timeHour < 10) { tollSchedule[i] = 4.65; } else if (timeHour < 18) { tollSchedule[i] = 2.35; } else { tollSchedule[i] = 1.55; } }

True.

The complexity of the algorithm below is O(1). if (timeHour < 6) { tollAmount = 1.55; } else if (timeHour < 10) { tollAmount = 4.65; } else if (timeHour < 18) { tollAmount = 2.35; } else { tollAmount = 1.55; }

True. The if-else statements execute a constant number of operations, which has a big-O complexity of O(1)

A contact list is searched for Bob. Assume the following contact list: Amy, Bob, Chris, Holly, Ray, Sarah, Zoe

What is the first contact searched? Holly Binary search first checks the middle contact. Bob is not found and comes before Holly, so the search continues with the first half of the contacts: Amy, Bob, Chris. What is the second contact searched? Bob The middle contact in the first half of the contact list is Bob. So, the desired contact is found.

Simplified runtime analysis: A constant number of constant time operations is O(1).

https://drive.google.com/file/d/16tISmsoAm-4a2eg_HC8dZQXKvID0IDxd/view?usp=sharing

Quicksort.

https://drive.google.com/file/d/19KW2nfQAMOZ80guq4kDOfhzs4mB0pjhi/view?usp=sharing

Runtime complexities for various pseudocode examples.2

https://drive.google.com/file/d/1LR6veq7ut6G4msfT0IlCVWiInOFj7xrH/view?usp=sharing

Rules for determining Big O notation of composite functions.

https://drive.google.com/file/d/1MNWrh62uyPTz22ol7rTKWvhl1JHD0z5i/view?usp=sharing

Insertion sort algorithm.

public class InsertionSort { public static void insertionSort(int [] numbers) { int i; int j; int temp; // Temporary variable for swap for (i = 1; i < numbers.length; ++i) { j = i; // Insert numbers[i] into sorted part // stopping once numbers[i] in correct position while (j > 0 && numbers[j] < numbers[j - 1]) { // Swap numbers[j] and numbers[j - 1] temp = numbers[j]; numbers[j] = numbers[j - 1]; numbers[j - 1] = temp; --j; } } } public static void main(String [] args) { int [] numbers = {10, 2, 78, 4, 45, 32, 7, 11}; int i; System.out.print("UNSORTED: "); for (i = 0; i < numbers.length; ++i) { System.out.print(numbers[i] + " "); } System.out.println(); /* initial call to quicksort with index */ insertionSort(numbers); System.out.print("SORTED: "); for (i = 0; i < numbers.length; ++i) { System.out.print(numbers[i] + " "); } System.out.println(); } }

Figure 18.8.1: Quicksort algorithm.

public class QuickSort { public static int partition(int [] numbers, int i, int k) { int l; int h; int midpoint; int pivot; int temp; boolean done; /* Pick middle element as pivot */ midpoint = i + (k - i) / 2; pivot = numbers[midpoint]; done = false; l = i; h = k; while (!done) { /* Increment l while numbers[l] < pivot */ while (numbers[l] < pivot) { ++l; } /* Decrement h while pivot < numbers[h] */ while (pivot < numbers[h]) { --h; } /* If there are zero or one items remaining, all numbers are partitioned. Return h */ if (l >= h) { done = true; } else { /* Swap numbers[l] and numbers[h], update l and h */ temp = numbers[l]; numbers[l] = numbers[h]; numbers[h] = temp; ++l; --h; } } return h; } public static void quicksort(int [] numbers, int i, int k) { int j; /* Base case: If there are 1 or zero entries to sort, partition is already sorted */ if (i >= k) { return; } /* Partition the data within the array. Value j returned from partitioning is location of last item in low partition. */ j = partition(numbers, i, k); /* Recursively sort low partition (i to j) and high partition (j + 1 to k) */ quicksort(numbers, i, j); quicksort(numbers, j + 1, k); } public static void main(String [] args) { int [] numbers = {10, 2, 78, 4, 45, 32, 7, 11}; int i; System.out.print("UNSORTED: "); for (i = 0; i < numbers.length; ++i) { System.out.print(numbers[i] + " "); } System.out.println(); /* Initial call to quicksort */ quicksort(numbers, 0, numbers.length - 1); System.out.print("SORTED: "); for (i = 0; i < numbers.length; ++i) { System.out.print(numbers[i] + " "); } System.out.println(); } }

Figure 18.6.1: Selection sort algorithm.

public class selectionSort { public static void selectionSort(int [] numbers) { int i; int j; int indexSmallest; int temp; // Temporary variable for swap for (i = 0; i < numbers.length - 1; ++i) { // Find index of smallest remaining element indexSmallest = i; for (j = i + 1; j < numbers.length; ++j) { if (numbers[j] < numbers[indexSmallest]) { indexSmallest = j; } } // Swap numbers[i] and numbers[indexSmallest] temp = numbers[i]; numbers[i] = numbers[indexSmallest]; numbers[indexSmallest] = temp; } } public static void main(String [] args) { int numbers [] = {10, 2, 78, 4, 45, 32, 7, 11}; int i; System.out.print("UNSORTED: "); for (i = 0; i < numbers.length; ++i) { System.out.print(numbers[i] + " "); } System.out.println(); /* initial call to quicksort with index */ selectionSort(numbers); System.out.print("SORTED: "); for (i = 0; i < numbers.length; ++i) { System.out.print(numbers[i] + " "); } System.out.println(); } }

The list is sorted into descending order: {20 15 10 5 0}

true. Each element is greater than the next element.

The list is sorted into descending order: {99.87 99.02 67.93 44.10}

true. Each element is greater than the next element.

The list is sorted into ascending order: {great greater greatest}

true. The first five characters of each string are equivalent. "great" contains the fewest characters, so "great" is less than "greater" or "greatest". The second 'r' in "greater" is less than the 's' in "greatest", so "greater" is less than "greatest".

The list is sorted into descending order: {F D C B A}

true. The list of characters are in reverse alphabetical order.

The list is sorted into ascending order: {chopsticks forks knives spork}

true. The list of strings are in alphabetical order.