ics 46 analysis of algorithms/big o notation notes review

Given the formula, T(N) = 4 • 10^-5 • N(logN)^2. Compute how long it will take to run when N = 1,000,000 (106). Show your work/calculations by hand, approximating logarithms, finish/simplify all the arithmetic.

- T(10^6) = (4 • 10^-5) • 10^6(log10^6)^2 - recall log base 2 10^6 is 20, and 10^6 is a million - T(10^6) = (4 • 10^-5) • 10^6(20)^2 - T(10^6) = (4 • 10^-5) • 10^6(400) - T(10^6) = (4 • 10^-5) • 400,000,000 - T(10^6) = 4/100000 • 400,000,000 - T(10^6) = 4/1 • 4000 - T(10^6) = 16000 seconds or 4 hours

suppose you have 4 functions, and you run them on an input of 100 and then 200. function a's time doubles, function b'2 time quadruples, function c's time stays the same, and function d's time increments by 5. estimate the complexity class of each function based on this data

- doubles O(N) ~ quadruples O(N^2) ~ no increase O(1) ~ increase by 5, O(logN) tip: doubling signatures!

suppose you have an O(N^2) function. it takes 4 seconds for 10,000 inputs. how long will it take for 20,000 inputs?

- since we know its O(N^2), we know its doubling signature if 4 times the time, so it should be 16 - OR, if we calculated the formula T(N) = 4 • 10^-8 • N^2, we could plug our numbers in and get the same answer

Recently a new O(N (log2 N)2) algorithm was developed for solving linear systems with symmetric diagonally dominant elements (don't worry about this terminology). If we time the method implementing this algorithm for N = 1,000 we find that T (N) = 4.0 seconds. (1) Write a formula, T(N) that computes the approximate time that it takes to run the algorithm for any input of size N. Show your work/calculations by hand, approximating logarithms, finish/simplify all the arithmetic.

- we have T(N) = c • O(N(logN)^2) - plug in 4 for T(N) and 1,000 for N - we now have 4 = c• 1,000(log1,000)^2 - log base 2 1,000 = 10 (remember 2^10 = 1024) - 4 = c • 1,000(10)^2 - 4 = c • 100,000 - c = 4/100,000 = 4^10^-5 - re-plug into first equation - T(N) = 4 • 10^-5 • N(logN)^2 (approximately)

suppose you have an O(N^2) function. calculate the "technology" constant c in its running time formula if the algorithm takes 4 seconds for 10,000 inputs.

- we have T(N) = c • O(N^2) - plug in 4 for T(N) and 10,000 for N - we now have 4 = c • 10,000^2 - c = 4/10,000^2 = 4^10^-8 - re-plug into first equation - T(N) = 4 • 10^-8 • N^2 (approximately)

assume all logs are to base two 2^10 Log10^6 10 = LogN, what is N if Log10^12 returns T 40, what does Log10^6 return?

1024 20 2^10 = 1024 20

Briefly explain why O(Log2 N) and O(Log10 N) are really the same complexity class.

Logarithms of different bases are proportional: different by a constant factor. But we drop constant factors when describing complexity classes. The base change formula is: Log2 N = (Log10 N / Log10 2).

write these complexity classes in order from lowest complexity (least complex aka slowest growing) to highest complexity and give an example of each O(N^2) O(N) O(1) O(NLogN) O(LogN)

O(1) - pass argument/parameters O(LogN) - binary search sorted array O(N) - linear searching an array O(NLogN) - fast sort O(N^2) - slow sort

suppose you have a function which executes 14N+9 instructions in the worst case. using big-o notation, find the time complexity of the function

O(N) explanation: we drop all constant and focus on the most quickly growing term

C++'s standard sorting method on arrays is what complexity class?

O(NLogN)

suppose you have a function which executes 8N^2 + 12N + 6 instructions in the worst case. using big-o notation, find the time complexity of the function (and justify why you drop some constants)

O(N^2) 1 - first, we can drop the 6 (even though we lose accuracy) because as N grows larger the 6 does not matter too much 2 - we can drop the 12N for the same reason - wea re only concerned with the most quickly growing term 3 - we can drop the constant from 8N^2 because it's machine-dependent (i.e., depends on the number of instructions a different compiler generates) 4 - O(N^2)

What "better-known"/simpler complexity class is equivalent to O(N log N^2); briefly explain why.

O(NlogN) because by log properties we can convert it to O(2NLogN) and we can drop the constant 2

big theta

a combination of big-O and big-Omega, which bounds a function from below and above

for all BIG ENOUGH N, Ta(N) < Tb(N) is true when a is in...?

a is in lower complexity class than b (aka if a is a lower complexity class than b (for example, O(N) and O(N^2)), then for all big enough N (where they intersect) a will take less time)

analysis of algorithms

a mathematical area of Computer Science in which we analyze the resources (mostly time, but sometimes space) used by algorithms to solve problems

what are the complexity classes of the following: a.) searching an unsorted array b.) slow sorting an array c.) binary searching an array (searching a sorted array) d.) quick sort

a.) O(N) b.) O(N^2) c.) O(LogN) d.) O(NLogN) in most cases but O(N^2) in the worst case

draw the following (one is impossible) a.) O(N) always faster than O(N^2) b.) O(N) is never faster than O(N^2) c.) O(N) sometimes faster

a.) should show an increasing line beneath a parabola b.) impossible, O(N) is lower complexity (simpler) than O(N^2) so for some N, it will eventually run faster c.) should show an increasing line intersecting a parabola explanation: b.) cannot draw a parabola under a line without them intersecting c.) For example, if algorithm a is O(N) with a constant of 100, and algorithm b is O(N^2) with a constant of 1, then for values of N in the range [1,100], Tb(N) = 1N^2 <= 100N = Ta(N) but for all values bigger than 100, Ta(N) = 100N <= 1N^2 = Tb(N)

assuming you have two different complexity classes a and b. the lower complexity class is better for _______??

all sizes beyond where the curves cross

why does it make no sense for a function to be complexity class O(1/N)

because the time decreases as the problem size increases (plugging in 2 gives .5 but plugging in 4 gives .25 which is a shorter amount of time)

big omega

bounds a function from below instead of from above.

To work correctly, binary searching requires a precondition that an array is sorted. It seems like this function should check this precondition and throw an exception if it is not met. What is the problem of precondition checking here? Your argument should discuss information about all relevant complexity classes.

checking the precondition would require a loop through the array which is O(N) in the worst case and binary searching is O(LogN), increasing the time to O(LogN+N) which is equal to O(N)

It is guaranteed that FOR ALL SIZES BEYOND SOME VALUE OF N, the algorithm in the lower complexity class will run _______?

faster

big-o notation

it means that the resource is "bounded above by growth on the order of"; typically represents the worst case behavior

what can complexity classes tell us about small size problems

not much