Big O Concepts / Notation
Formula for sum of natural number to n
1+ 2+ ... + n = n(n+1) / 2, for n a natural number.
Algorithm with 2 steps. When do you add runtimes?
Add: If you algorithm is in the form "do this, then, when you're all done, do that."
Amortized Time
Allows us to describe that, yes the worst case happens every once in a while, but once it happens, it won't happen again for so long that the cost is "amortized" (gradually write off). https://stackoverflow.com/questions/200384/constant-amortized-time
Ω
Big Omega Lower bound on efficiency
ϴ
Big Theta Means both O and Ω. If an algorithm is both O(N) and Ω(N)it is ϴ(N)
O(1)
Constant Algorithm that will always execute in the same time or space regardless of the size of the input data.
O(2^n)
Exponential Algorithm whose growth doubles with each addition to the input dataset.
Complexity
How do the resource requirements of a program or algorithm scale: what happens as the size of the problem being solved gets larger? Complexity affects performance but not the other way around.
Performance
How much time/memory/disk/ ... is actually used when a program is run. Depends on the machine, compiler, etc. as well as the code.
O(n)
Linear Algorithm whose performance will grow linearly and in direct proportion to the size of the input dataset. Ex: Driving: as distance increase, driving time increases in linear fashion. For each loop
Big O Notation
Mathematical notation that describes the limiting behavior of a function when the argument tends towards a particular value or infinity. Helps you to analyze the scalability and efficiency of algorithms. O is upper bound
Algorithm with 2 steps. When do you multiply runtimes?
Multiply: If you algorithm is in the form "do this each time you do that."
Equivalent to O(N)? Why? O(N + M)
No, no established relationship between N and M, so we have to keep both.
Runtime of a recursive function with 2 branches is typically:
O(2^depth)
Complexity of: void recursiveFun4(int n, int m, int o) { if (n <= 0) { printf("%d, %d\n",m, o); } else { recursiveFun4(n-1, m+1, o); recursiveFun4(n-1, m, o+1); } }
O(2ⁿ) exponential, since each function call calls itself twice unless it has been recursed n times.
Complexity of: int recursiveFun1(int n) { if (n <= 0) return 1; else return 1 + recursiveFun1(n-1); }
O(N)
Complexity of: int recursiveFun2(int n) { if (n <= 0) return 1; else return 1 + recursiveFun2(n-5); }
O(N)
Complexity of: int recursiveFun5(int n) { for (i = 0; i < n; i += 2) { // do something } if (n <= 0) return 1; else return 1 + recursiveFun5(n-5); }
O(N²)
Runtime of a recursive function with multiple branches is typically:
O(branches^depth)
Complexity of: int recursiveFun3(int n) { if (n <= 0) return 1; else return 1 + recursiveFun3(n/5); }
O(log(N))
Sorting a string is usually ...? why?
O(n log n) It's the optimal value for a comparison sort. Also the complexity of many languages' default sort implementation.
Recursion Tree
Useful for visualizing what happens when a recurrence is iterated. It diagrams the tree of recursive calls and the amount of work done at each call. The two features of a recursive function to identify are: 1) The tree depth (how many total return statements will be executed until the base case) 2) The tree breadth (how many total recursive function calls will be made) Example T(n) = 2T(n/2) + n2. The recursion tree for this recurrence has the following form:
Equivalent to O(N)? Why? O(N + log N)
Yes, N is dominant term
Equivalent to O(N)? Why? O(N + P), where P < N/2
Yes, N is dominant term so we can drop the p
Equivalent to O(N)? Why? O(2N)
Yes, we drop constants
the minimum height of a binary tree with n nodes is:
log2(n)
O(log n)
logarithmic Ex: Think divide and conquer problems, like looking someone up in a phonebook Finding an item in a sorted array with a binary search or a balanced search tree as well as all operations in a Binomial heap
O(n^2)
quadratic Algorithm whose performance is directly proportional to the square of the size of the input data.
