Big O Notation

Example : Logarithmic operation.

A binary search on a sorted list would have logarithmic characteristics given that later operations are less expensive than the initial ones. So although N would increase the size of the list and cost of the search. An increment of N would have a less-than-linear increase in cost.

Logarithms & Big O

A logarithm of a value x given any base will slow in growth rate as x increases. This mirrors the behaviour of something like a binary search where each iteration starts off costly but decreases as the algorithm progresses.

Example : Constant operation

Determining if a number is even or odd; using a constant-size lookup table. Ie. the look-up table could be an array of booleans of size 10, where the value is true/false depending on the index used. A true value means that the number is even. Only values 0 to 9 could be used.

What is O(n!)

Where cost is a factorial of the input data. So an input of 1 is a cost of 1, input of 2 is 2, input of 3 is 6 and so on.

Example : Linear operation

Finding an item (search) in an unsorted list where the size of the list is N. That is assuming the worst case (upper-bound) where the item is the last one found.

Logarithm

Given an operation with an exponent, for example 5² = 25, if we only had the base and the target number, the logarithm is the exponent that we are trying to find. So, the logarithm of 25 would be 2 if the base is 5. So 5ⁿ = 25 then n = 2.

Factorial

In mathematics, the factorial of a non-negative integer n, denoted by n!, is the product of all positive integers less than or equal to n. So 5! = 1*2*3*4*5 = 120. Note that 0! = 1

What part of code is Big O used to describe ?

It focuses on the part of an algorithm that grows in time or space as we increase the size of the *input data* that we are operating on.

Factorials and Permutations

Its most basic occurrence is the fact that there are n! ways to arrange n distinct objects into a sequence (i.e., permutations of the set of objects). So, with 3 distinct objects, there are 1*2*3 = 6 possible arrangements. 123, 312, 231, 321, 132, 213 (note the cycling of objects).

log b (x)

Logarithm of x to base b. So, if x was 25 and b was 5, the logarithm would be 2.

What is O(n log n) ?

Loglinear, quasi-linear oir linearithmic. A combination of linear growth with an additional logarithmic growth rate that makes it more performant than O(n²) or exponential growth but not quite as performant as O(n) and much less than O(log n). This describes the order of functions that implement merge sort or quicksort in the best or average case.

Example : Exponential Operation

Never encountered and exponential operation, but if you were to have a constant dimension size but an increasing number of dimensions, this is the very fast growth you would expect (e.g. 2D to 3D in Cubeville).

Big O comparisons - relative performance

O(1) is constant and excellent for large inputs, O(log n) does increase with large data sets but with increasingly moderate growth. O(n) is linear and grows directly in proportion to its input size. O(n log n) is worse than linear but scales reasonably well compared to O(n²) which has quadratic growth and even worse O(Cⁿ) which has exponential growth.

Example : Factorial Operation

Solving the travelling salesman problem via brute-force (every permutation) would be a factorial operation where N is the number of possible customers.

What does 'O' stand for ?

The *growth rate* of a function is also referred to as the *order* of the function (with the function representing the operation of algorithm).

Example : Quadratic operation.

The naive implementation or worst case of bubble sort would be a quadratic order function whereby N is the size of the list to sort. This is because, if the smallest element is at the far end of the list, then it will take N passes of the array to move it to the beginning. Therefore it is N (one pass) * N (number of passes) operations (N²).

What is O(1) ?

This represents a constant cost. It basically means that the function will cost the same, regardless of the input data. So an input of 1 might cost 1, and input of 2 would also cost 1.

What is O(logN)

This represents a cost controlled by a logarithm. This means that, although cost increases overall as the input increases it isn't as expensive as linear because the algorithm (e.g. a loop) features a series of operations which decrease in cost as it progresses. For example, a binary search operates on increasingly small sets of data as it moves towards the goal, each operation is half the cost of the last.

What is O(n) ?

This represents a linear cost, i.e. the cost of the function will *increase* at a linear rate as its input increases. Perhaps an input of 1 will cost 1, an input of 2 will cost 2 and so on. so the cost increases at the same rate that input increases.

What is O(n²) ?

This represents a quadratic (ie, squared cost). so an input of 1 would be a cost of 1, an input of 2 would be 4, an input of 3 would be 9. Note that the *growth* there is 1, 2, 5 with a linear input, so growth increases with N, as well as the value.

What is O(Cⁿ) ?

This represents an exponential growth rate where the input size is used as an exponent to some constant C. So, if the constant was 10, input of 1 would be 10, of 2 would be 100, 3 would be 1000 and so on, growth is exponential. By the time we have an input of 100 we have 1E100...ie. a 1 with one hundred noughts afterwards !

What is Big O notation for ?

big O notation is used to *classify* functions (usually algorithms in comp sci) by how they respond (e.g., in their processing time or working space requirements) to changes in input size. Ie. *efficiency*, normally of its *upper-bound* (worst case).