Big O - Examples and Exercises

Example 7: Which of the following are equivalent to O(N)? Why? O(N + P), where P < N/2 O(2N) O(N + logN) O(N + M)

- If P < N/2, then we know that N is the dominant term so we can drop the O(P). - O(2N) is O(N) since we drop constants. - O(N) dominates O(log N), so we can drop the O(log N). - There is no established relationship between N and M, so we have to keep both variables in there. Therefore, all but the last one are equivalent to O(N).

Example 8: Suppose we had an algorithm that took in an array of strings, sorted each string, and then sorted the full array. What would the runtime be?

- Let s be the length of the longest string. - Let a be the length of the array. - Sorting each string is O(s log s). - We have to do this for every string (and there are strings), so that's O(a * s log s). - Now we have to sort all the strings. Since you need to compare strings, each string comparison takes O(s) time. There are O(a log a) comparisons, therefore this will take O(a * s log a)time. If you add up these two parts, you get O(a * s(loga + log s)).

Example 5: What about this strange bit of code? void printUnorderedPairs(int [] arrayA, int [] arrayB) { for (int i = 0; i < arrayA.length; i++) { for (int j = 0; j < arrayB.length; j++) { for (int k = 0; k < 100000; k++) { System.out.println(arrayA[i] + "," + arrayB[j]); } } } }

100,000 units of work is still constant, so the runtime is O(ab).

Example 3: This is very similar code to the above example, but now the inner loop starts at i + 1. void printUnorderedPairs(int [] array) { for (int i = 0; i < array.length; j++) { for (int j = i + 1; j < array.length; j++) { System.out.println(array[i] + "," + array[j]); } } }

Alternatively, we can figure out the runtime by thinking about what the code "means." It iterates through each pair of values for (i, j) where j is bigger than i. There are N^2 total pairs. Roughly half of those will have i < j and the remaining half will have i > j. This code goes through roughly N^2 / 2 pairs so it does O(N^2) work.

Example 15: The following code prints all Fibonacci numbers from 0 through. n. However, this time, it stores (i.e., caches) previously computed values in an integer array. If it has already been computed, it just returns the cache. What is its runtime? void allFibb(int n) { int [] memo = new int[n + 1]; for (int i = 0; i < n; i++) { System.out.println(i + ": "" + fib(i, memo)); } } int fib(int n, int [] memo) { if (n <= 0) return 0; else if (n == 1) return 1; else if (memo[n] > 0) return memo[n]; memo[n] = fib(n - 1, memo) + fib(n - 2, memo); return memo[n]; }

At each call to fib(i), we have already computed and stored the values for fib(i -1) and fib (i -2). We just look up those values, sum them, store the new result, and return. This takes a constant amount of work. We're doing a constant amount of work n times, so this is O(n) time. This technique, called memoization, is a very common one to optimize exponential time recursive algorithms.

Example 14: The following code prints all Fibonacci numbers from 0 through n. What is its time complexity? void allFib(int n) { for (int i = 0; i <= n; i++) { System.out.println(i + ": " + fib(i)); } } int fib(int n) { if (n <= 0) return 0; else if (n == 1) return 1; return fib(n - 1) + fib(n - 2); }

Based on what we showed on page 45, this is 2^n + 1 - 2, Therefore the runtime to compute the first n Fibonacci numbers (using this terrible algorithm) is still O(2^n).

Example 16: The following function prints the powers of 2 from 1 through n (inclusive). For example, if n is 4, it would print 1, 2, and 4. What is its runtime? int powersOf2(int n) { if (n < 1) { return 0; } else if (n == 1) { System.out.println(1); return 1; } else { int prev = powersOf2(n / 2); int curr = prev * 2; System.out.println(curr); } }

Let's walk through a call like powersOf2(50). The runtime, then, is the number of times we can divide 50 (or n) by 2 until we get down to the base case (1). As we discussed on page 44, the number of times we can halve n until we get 1 is O(log n).

Additional Problem #3: The following code computes a % b. What is its runtime? int mod(int a, int b) { if (b <= 0) { return -1; } int div = a / b; return a - div * b; }

O(1). It does a constant amount of work.

Additional Problem #4: The following code performs integer division. What is its runtime (assume a and b are both positive)? int div(int a, int b) { int count = 0; int sum = b; while (sum <= a) { sum += b; count++; } return count; }

O(a / b). The variable count will eventually equal a/b. The while loop iterates count times. Therefore, it iterates a/b times.

Additional Problem #12: The following code computes the intersection (the number of elements in common) of two arrays. It assumes that neither array has duplicates. It computes the intersection by sorting one array (array b) and then iterating through array a checking (via binary search) if each value is in b. What is its runtime? int intersection(int[]a, int [] b) { mergesort(b); int intersect = 0; for (int x: a) { if (binarySearch(b, x) >= 0) { intersect++; } } return intersect; }

O(b log b + a log b). First, we have to sort array b, which takes O(b log b) time. Then, for each element in a, we do binary search in O(log b) time. The second part takes O(a log b) time.

Additional Problem #1: The following code computes the product of a and b. What is its runtime? int product(int a, int b) { int sum = 0; for (int i = 0; i < b; i++) { sum += a; } return sum;

O(b) The for loop just iterates through b.

Additional Problem #2: The following code computes a^b. What is its runtime? int power(int a, int b) { if (b < 0) { return 0; //error } else if (b == 0) { return 1; } else { return a * power(a, b - 1); } }

O(b). The recursive code iterates through b calls, since it subtracts one at each level.

Additional Problem #11: The following code prints all strings of length k where the characters are in sorted order. It does this by generating all strings of length k and then checking if each is sorted. What is its runtime? void printSortedStrings(int remaining) { printSortedStrings(remaining, ""); } void printSortedStrings(int remaining, String prefix) { if (remaining == 0) { if (isInOrder(prefix)) { System.out.println(prefix); } } else { for (char c = 'a'; c <= 'z'; c++) { printSortedStrings(remaining - 1, prefix + c); } } } boolean isInOrder(String s) { boolean isOrder = true; for (int i = 1; i < s.length(); i++) { if (s.charAt(i - 1) > s.charAt(i)) { isInOrder = false; } } return isInOrder; }

O(kc^k), where k is the length of the string and c is the number of characters in the alphabet. It takes O(c^k) time to generate each string. Then, we need to check that each of these is sorted, which takes O(k) time.

Additional Problem #10: The following code sums the digit in a number. What is its big O time? int sumDigits(int n) { int sum = 0; while (n > 0) { sum += n % 10; n /= 10; } return sum; }

O(log n). The runtime will be the number of digits in the number. A number with d digits can have a value up to 10^d. If n = 10^d, then d = log n. Therefore, the runtime is O(log n).

Additional Problem #5: The following code computes the [integer] square root of a number. If the number is not a perfect square (there is no integer square root), then it returns -1. It does this by successive guessing. If n is 100, it first guesses 50. Too high? Try something - lower halfway between 1 and 50. What is its runtime? int sqrt(int n) { return sqrt_helper(n, 1, n); } int sqrt_helper(int n, int min, int max) { if (max < min) return -1; // no square root int guess = (min + max) / 2; if (guess * guess == n) { // found it! return guess; } else if (guess * guess < n) { //too low return sqrt_helper(n, guess + 1, max); // try higher } else { //too high return sqrt_helper(n, min, guess - 1); //try lower } }

O(log n). This algorithm is essentially doing a binary search to find the square root. Therefore, the runtime is O(log n).

Additional Problem #7: If a binary search tree is not balanced, how long might it take (worst case) to find an element in it?

O(n), where n is the number of nodes in the tree. The max time to find an element is the depth tree. The tree could be a straight list downwards and have depth n.

Additional Problem #8: You are looking for a specific value in a binary search tree, but the tree is not a binary search tree. What is time complexity of this?

O(n). Without any ordering property on the nodes, we might have to search through all the nodes.

Additional Problem #9: The appendToNew method appends a value to an array by creating a new, longer array and returning this longer array. You've used the appendToNew method to create a copyArray function that repeatedly calls appendToNew. How long does copying an array take? int [] copyArray(int [] array) { int [] copy = new int[0]; for (int value: array) { copy = appendToNew(copy, value); } return copy; } int [] appendToNew(int [] array, int value) { //copy all elements over to new array int [] bigger = new int[array.length + 1]; for (int i = 0; i < array.length; i++) { bigger[i] = array[i]; } //add new element bigger[bigger.length - 1] = value; return bigger; }

O(n^2), where n is the number of elements in an array. The first call to appendToNew takes 1 copy. The second call makes 2 copies. The third call takes 3 copies. And so on. The total time will be the sum of 1 through n, which is O(n^2).

Additional Problem #6: The following code computes the [integer] square root of a number. If the number is not a perfect square (there is no integer square root), then it returns -1. It does this by trying increasingly large numbers until it finds the right value (or is it too high). What is its runtime? int sqrt(int n) { for (int guess = 1; guess * guess <= n; guess++) { if (guess * guess == n) { return guess; } } return -1; }

O(sqrt(n)). This is just a straightforward loop that stops when guess * guess > n (or, in other words, when guess > sqrt(n)).

Example 10: The following method checks if a number is prime by checking for divisibility on numbers less than it. It only needs to go up to the square root of n because if n is divisible by a number greater than its square root then it's divisible by something smaller than it. What is the time complexity of this function? boolean isPrime(int n) { for (int x = 2; x *x <= n; x++) { if (n%x == 0) { return false; } } return true; }

The for loop will start when x = 2 and end when x* x = n. Or, in other words, it stops when x = square root of n (when x equals the square root of n).

Example 4: This is similar to the above, but now we have two different arrays. void printUnorderedPairs(int [] arrayA, int [] arrayB) { for (int i = 0; i < arrayA.length; i++) { for (int j = 0; j < arrayB.length; j++) { if (arrayA[i] < arrayB[j]) { System.out.println(arrayA[i] + "," + arrayB[j]); } } } }

The if-statement within j's for loop is O(1) time since it's just a sequence of constant-time statements.

Example 2: What is the runtime of the below code? void printPairs(int[] array) { for (int i = 0; i < array.length; i++) { for (int j = 0; j < array.length; j++) { System.out.println(array[i] + "," + array[j]); } } }

The inner for loop has O(N) iterations and it is called N times. Therefore, the runtime is O(N^2).

Example 6: The following code reverses an array. What is its runtime? void reverse(int [] array) { for (int i = 0; i < array.length / 2; i++) { int other = array.length - i - 1; int temp = array[i]; array[i] = array[other]; array[other] = temp; } }

This algorithm runs in O(N) time. The fact that it only goes through half of the array (in terms of iterations) does not impact the big O time.

Example 9: The following code sums the values of all the nodes in a balanced binary search tree. What is its runtime? int sum(Node node) { if (node == null){ return 0; } return sum(node.left) + node.value + sum(node.right); }

This code touches each node in the tree once and does a constant time amount of work with each "touch" (excluding the recursive calls). Therefore, the runtime will be linear in terms of the number of nodes. If there are N nodes, then the runtime is O(N).

Example 11: The following code computes n! (in factorial). What is its time complexity? int factorial (int n) { if (n < 0) { return -1; } else if (n == 0) { return 1; } else { return n * factorial(n - 1); } }

This is just a straight recursion from n to n - 1 to n - 2 down to 1. It will take O(n) time.

Example 12: This code prints all permutations of a string with all distinct characters. void permutation (String str) { permutation (str, ""); } void permutation (String str, String prefix) { if (str.length() == 0) { System.out.println(prefix); } else { for (int i = 0; i < str.length(); i++) { String rem = str.substring(0, i) + str.substring(i + 1); permutation(rem, prefix + str.charAt(i)); } } }

This means the number of nodes in the tree is less than e * n!. The constant drops out, leaving us with O(n!) calls to permutation, each of which takes O(n) time. THe total runtime is thus O(n * n!), or O((n + 1)!).

Example 1: What is the runtime of the below code? void foo(int [] array) { int sum = 0; int product = 1; for (int i = 0;i < array.length; i++) { sum += array[i]; } for (int i = 0; i < array.length; i++) { product *= array[i]; } System.out.println(sum + ", " + product); }

This will take O(N) time. The fact that we iterate through the array twice doesn't matter.

Example 13: The following code computes the Nth Fibonacci number. int fib(int n) { if (n <= 0) return 0; else if (n == 1) return 1; return fib(n - 1) + fib(n - 2); }

We can use the earlier pattern we'd established for recursive calls: O(branches^depth). There are 2 branches per call, and we go as deep as N, therefore the runtime is O(2^N).