C949 Data Structures and Algorithms: Lesson 10 Take 2

Determine Big O notation

1. If f(N) is a sum of several terms, the highest order term (the one with the fastest growth rate) is kept and others are discarded. 2. If f(N) has a term that is a product of several factors, all constants (those that are not in terms of N) are omitted.

Runtime Analysis Steps

1. Total # of operations (assignment, addition, comparison, etc.) 2. Total Loop iterations 3.

All functions that have the same growth rate are considered equivalent in

Big O notation.

Θ notation

provides a growth rate that is both an upper and lower bound

Asymptotic notation

the classification of runtime complexity that uses functions that indicate only the growth rate of a bounding function - constant is factored out

worst case runtime

the runtime complexity for an input that results in the longest execution

worst case

the scenario where the algorithm does the maximum possible number of operations

best case

the scenario where the algorithm does the minimum possible number of operations - N != 0 - must describe the contents of the data being processed.

auxiliary space complexity

the space complexity not including the input data

Algorithm runtime analysis often focuses on the

worst-case runtime complexity

auxiliary space complexity function

S(N) = k

runtime complexity function

T(N)

To analyze how runtime of an algorithm scales as the input size increases:

1. Determined # of operation executes for specific input size, N 2. big-O notation for that function determined

lower bound

A function f(N) that is ≤ the best case T(N), for all values of N ≥ 1 - always best case

upper bound

A function f(N) that is ≥ the worst case T(N), for all values of N ≥ 1 - always worst case

Common Big O complexities

Constant, Logarithmic, Linear, Log-linear, Quadratic, Exponential

Worst case example

No items in the list are less than value. The worst case is when no items in the list are less than value. All array elements are compared against value, then value is returned.

Constant

O(1) Ex: Find minimum

Log-linear

O(N log N) Ex: Merge sort

Linear

O(N) Ex: Linear search

Quadratic

O(N²) Ex: Selection Sort

Exponential

O(c^N) Ex: Fibonacci

Big O Notation of Composite Functions: c · O(f(N))

O(f(N)) Ex: 10 + O(N²) = O(10 + N²) = O(N²)

space complexity function

S(N) = N + k k - a constant representing memory used for things like the loop counter and list pointers

Neither best nor worst case example

The first half of the list has elements greater than value and the second half has elements less than value. About half of the list items need to be analyzed in this scenario, which is neither the best nor worst case.

Best case example

The first item in the list is less than value. The best case is when the first item in the list is less than value. The algorithm returns after 1 comparison.

space complexity

a function, S(N), that represents the number of fixed-size memory units used by the algorithm for an input of size N

runtime complexity

a function, T(N), that represents the number of constant time operations performed by the algorithm on an input of size N - lower & upper bound

Big O notation

a mathematical way of describing how a function (running time of an algorithm) generally behaves in relation to the input size - all functions have same growth rate

theoritical analysis of an algorithm

describes runtime in terms of # of constant time operations (not nanoseconds)

constant time operations

for a given processor, always operates in the same amount of time, regardless of inputs - allows the efficiency of algorithms to be compared. Ex) arithmetic, assignment, comparisons of fixed values, read/write array of particular input

For large N, the difference in computation time varies

greatly with the rate of growth of the function f

Ω notation

provides a growth rate for an algorithm's lower bound

O notation

provides a growth rate for an algorithm's upper bound

Big O Notation of Composite Functions: c + O(f(N))

O(f(N)) Ex: 10 · O(N²) = O(10 · N²) = O(N²)

Big O Notation of Composite Functions: g(N) + O(f(N))

O(g(N) + O(f(N))) Ex: 2·N³ + O(N²) = O(2·N³ + N²) = O(N³)

Big O Notation of Composite Functions: g(N) · O(f(N))

O(g(N) · O(f(N))) Ex: 3·N · O(N²) = O(3·N·N²) = O(3·N³) = O(N³)

Logarithmic

O(log N) Ex: Binary search