Algorithms

How does an insertion sort work?

Moves items from an unsorted collection to a new, sorted collection by inserting items in their proper place one at a time. This algorithm has poor performance compared to other sorting algorithms.

What's the time complexity of this function: function f(arr) { let res = 0; for (let i=0; i<80; ++i) { res = arr[0]+i; } return res; }

O(1)

What kind of sorting algorithm is this: FUNCTION f(collection) FOR i = FIRST INDEX of collection to LAST INDEX of collection - 1 SET minIndex to i FOR j = (i + 1) to LAST INDEX of collection IF collection[j] < collection[minIndex] THEN SET minIndex to j END IF END FOR SET tmp to collection[i] SET collection[i] to collection[minIndex] SET collection[minIndex] to tmp END FOR RETURN collection END FUNCTION

selection sort

What is recursion?

A process in which we repeat the same steps until complete...in programming it refers to a procedure/function that calls itself.

What is an algorithm?

A sequence of steps/instructions on how to complete a task in a certain amount of time. Ex.: think the recipe for baking a cake as an algorithm.

How many base cases must a recursive function have?

Atleast 1

When writing a recursive function, should the recursive case or the base case(s) be declared first?

Base case(s), otherwise we'd create an infinite loop.

Why is a binary search named so?

Because at every iteration we divide the data set in to two parts, and determine whether the query is either "here", or "there". This duality of options is typical of binary systems, hence the name.

What are some common algorithms that can or must use recursion?

Binary search, merge sort, quick sort, factorial operations, Fibonacci sequence, BFS, DFS

How does a merge sort work?

Breaks the collection into sub-collections and sorts each sub-collection. Sorted sub-collections are merged (using recursion) to form larger sorted collections until the entire collection is sorted. Merge sort has good performance compared to other sorting algorithms.

What advantages can recursion have over iteration?

Can be more intuitive/easier to impliment

A binary search is refered to as a _____ and _____ algo. Why?

Divide and conquer. It divides a collection in half with every iteration, checks if we found the query, and if not divides the remaining collection in half again, allowing us to keep eliminating about half the wrong answers.

Describe the format of a recursive function in pseudocode

FUNCTION recursiveF IF base case 1 THEN Handle case 1 .... ELSE CALL recursiveF with some operation that breaks the problem into a smaller bit END IF END FUNCTION

What is the time complexity of a binary search?

If poorly constructed we basically create a linked list instead of a binary search tree, so the upper bound would be O(n). If properly constructed, it's Theta(log n)

How is time complexity expressed?

Instead of actual time, it's expressed as a function "f" that estimates the time it takes to execute a program relative to an input size of "n". Therefore f(n) is the growth rate relative to input size. With time complexity, we are generally concerned with how the runtime growth rate changes when "n" approaches infinity, AKA the asymptotic behavior of the complexity. We can further drill down into time complexity by finding the worst-case, the average, and the best case, but unless otherwise specified, we assume that we are referring to the upper bound.

How does a linear search work?

It iterates through an array, list, linked list, etc. until it finds the target item.

All these sorting algorithms are comparison algorithms. What does this mean?

It means that every time we run through an iteration, we compare an element to another to see if it should go right or left

What is the Order, or Big O notation of "n"?

It represents an upper bound function as the input size grows, AKA it's the "worst case scenario" runtime. Note that although Big O is normally used to represent the upper bound, it can also be used to express the average case.

What is runtime?

It's either: the physical time duration of an algo, and/or is used synonymously with "time complexity".

Describe what time complexity is

It's the metric used to describe the efficiency of an algo, estimates the time it takes for a program to execute.

We can solve problems using recursion or using loops. When we use loops, what technique are we using?

Iteration

What advantages does iteration have over recursion?

Iteration tends to run quicker and be less restricted by system limitations.

In more precise terms, how do we determine time complexity?

Look at the statements within the codeblock: forget about any statements that are constants (lines of code that only run a certain set amount of time(s) no matter the input size). The runtime for loops will be linear, meaning O(n), unless the loop only executes a constant amount of times (ex. in ruby: 3.times do { ...something...} ). If it's a loop within a loop, it's quadratic represented by n^2. Add all the complexities of relevant statements together, remember to drop the constants, and simplify the expression to the largest term (the number with the greatest power).

For a given algorithm, the Big Omega Ω function is the lower bound of that algorithm. What does this mean?

Lower bound = best-case scenario = the minimum runtime that's possible for that algo = Omega. If that Omega

Describe constant growth rate

O(1) The runtime stays constant regardless of the input size. Accessing arrays usually takes constant time. Ex.: In an airport, a TSA officer announces for passengers to take off shoes, and regardless of how many people are in line, he only says it once.

List the algorithmic growth rates from most efficient to least efficient

O(1), O(log n), O(n), O(n log n), O(n^2), O(2^n), O(n!)

What is the time complexity of a recursive fibonacci function?

O(2^n)

What's the time complexity of this function: function f(n) { if (n <= 1) { return 1; } return f(n-1)+f(n-2); }

O(2^n)

Describe exponential growth rate

O(2^n) It's where the growth doubles for each additional piece of data. This runtime is often seen in recursive functions.

What's the time complexity of this function: function f(coll, item) { let lo = 0, hi = coll.length; while (lo < hi) { let mid = Math.floor(lo+hi/2); if (coll[mid] === item) return true; else if (coll[mid] < item) lo = mid+1; else hi = mid; } return false; }

O(log n)

Describe logarithmic growth rate

O(log n) The runtime grows by one unit every time we double the input, AKA, we halve the data set with each iteration. The closer the input size gets to infinity, the more the curve flattens. This runtime is often seen in binary searches.

Describe log-linear growth rate

O(n log n) It halves the data each time for each of n times. The graph looks like a line that slightly curves upwards. This runtime is seen in almost no data structures, but in several different sorting algos like merge sort and heap sort.

What is the time complexity of a linear search?

O(n)

What is the time complexity of this function: function f(arr) { let res = 0; for (let i=0; i<arr.length; ++i) { res += arr[i]; } return res; }

O(n)

What's the time complexity of this function: function f(arr) { let res = 0; for (let i=0; i<arr.length; ++i) { for (let j=0; j<80; ++i) { res += res*i*j; } } return res; }

O(n)

Describe linear growth rate

O(n) When the runtime of the function is directly proportional to the size of the input data. This runtime is often seen with a single for loop.

What's the time complexity of this function: function f(arr) { let res = 0; for (let i=0; i<arr.length; ++i) { for (let j=0; j<arr.length; ++i) { res += res * i * j; } } return res; }

O(n^2)

Describe quadratic growth rate

O(n^2) It's directly proportional to the square of the size of the input data, graphed it looks like a parabola. This runtime is often seen in a for loop nested in a for loop.

How does a bubble sort work?

Passes through a collection multiple times. In each pass, it compares adjacent items and swaps them according to the desired sorting order. The algorithm finishes when there are no swaps done for a complete pass. Bubble sort has poor performance when compared to other sorting algorithms.

How does a quick sort work?

Picks a pivot point and compares each item to the pivot point. It moves each item according to the pivot point. When there are no more items to move in the first iteration, we repeat these steps for the "left" and "right" sides of the collection. Quick sort has good performance compared to other sorting algorithms.

Why are algorithms important?

Programs attempt to automate solutions to a problem. Algorithms help us solve these problems in the most efficient way possible, so that programs are fast and accurate.

How does a selection sort work?

Selects the item which should be sorted "next" and moves it to the front (or back) of the collection. Selection sort has poor performance compared to other sorting algorithms.

What stops a recursive function and unwinds the stack?

The base case

What qualities does a block of code have that makes it an algorithm?

The instructions must be a sequence of logical steps, the instructions must accomplish a goal, and when executed the code must finish in a finite amount of time.

If none of the base cases are true the first time that we call a function, how will they be true when the function is called recursively? Explain the technique for writing a base case.

The nature of recursion is to break a problem down into smaller pieces with each call, so that eventually when the base case is read it will return the answer to the smallest version of the problem, thus letting us unwind the stack by putting all the solutions back together.

If the base case(s) aren't yet hit in a recursive function, what gets executed/called?

The recursive case, which calls on it's own function again.

For a given algorithm, the Big Theta Θ function is the tight bound of that algorithm. What does this mean?

Tight bound = when the upper bound (Big O) and lower bound (Omega) of an algo is the same function. Ex.: an algo taking Theta(n log n) takes at least (n log n), AKA Omega/lower bound etc., AND no more than (n log n), AKA Big O/upper bound etc.

What is the time complexity of a selection sort?

Worst: O(n^2) average: Theta(n^2) best: Omega(n^2)

According to youtube guy Abdul Bari....True or false: Big O, Big Omega, and Big Theta are not bound to upper limit/bound, lower limit/bound, average, etc. We can use any notation we want.

True

True or false: a binary search only works on sorted collections

True

True or false: unlike other search algos, a linear search works also works on an unsorted collection.

True

For a given algorithm, the Big O is the upper bound for that algorithm. What does this mean?

Upper bound = worst-case scenario for that algorithm = the maximum runtime that's possible for that algo = Big O. If that algo is graphed, everything below the "line" is where the runtime could actually be.

In simple terms, how do we determine time complexity?

We calculate how quickly the program runs relative to additional data being fed into the program. We estimate assuming the input is infinite so we know it's worst-case scenario. Add up how many instructions the algo will execute relative to the size of its input, then simplify the expression to the largest term and drop any constants.

Like all algos, a linear search is language agnostic, meaning:...

We can impliment it in any language

When might a recursive solution make more sense to use over an iterative solution?

When an algo can be defined as a combination of sub-results starting with known cases...when you have recurring structures such as linked lists and trees, recurring subproblems, when the definition itself is recursive such as in fibonacci, etc

When might an iterative solution make more sense than a recursive one?

When an algo requires running straight through a data set, passing over a collection, etc.

What is the time complexity for a merge sort?

Worst: O(n log n) average: Theta(n log n) best: Omega(n log n)

What is the time complexity of a quick sort?

Worst: O(n^2) average: Theta(n log n) best: Omega(n log n) What makes Quick Sort useful, however, is that Quick Sort's best case is also (n log n), and to get from worst case to best case usually just means adjusting which element we point our pivot to (aka: set the pivot to roughly the middle value of a collection, and avoid setting the pivot to values close to the beginning or end of the collection). Therefore, Merge Sort and Quick Sort give us better runtimes and are more effecient than the other sorting algorithms.

What is the time complexity of a bubble sort?

Worst: O(n^2) average: Theta(n^2) best: Omega(n)

What is the time complexity of an insertion sort?

Worst: O(n^2) average: Theta(n^2) best: Omega(n)

What kind of sorting algorithm is this: FUNCTION f(collection) REPEAT SET swapped to false FOR i = FIRST INDEX of collection to LAST INDEX of collection - 1 IF collection[i] > collection[i + 1] THEN SET tmp to collection[i] SET collection[i] to collection[i + 1] SET collection[i + 1] to tmp SET swapped to true END IF END FOR UNTIL swapped is FALSE RETURN collection END FUNCTION

bubble sort

Code an iterative binary search in JavaScript

function binarySearch(list, query) { let lowIndex = 0; let highIndex = list.length - 1; while(lowIndex <= highIndex) { let midIndex = Math.floor((lowIndex + highIndex) / 2); if(list[midIndex] > query) { highIndex = midIndex--; } else if(list[midIndex] < query) { lowIndex = midIndex++; } else { return list[midIndex]; } } return "Item not found"; };

Code a recursive solution for a binary search in JavaScript

function binarySearch(list, query, left, right) { if(right < left) { return "Item not found"; } let searchResult; let mid = Math.floor((right - left) / 2) + left; if(list[mid] = query) { searchResult = list[mid]; return searchResult; } else if (query < list[mid]) { return binarySearch(list, query, left, mid--); } else { return binarySearch(list, query, mid++, right); } };

Code a recursive function in javascript that finds the nth fibonacci number

function fibonacci(n) { if(n < 1) { return "Invalid input"; } else if(n == 1 || n == 2) { return 1; } else { return fibonacci(n - 1) + fibonacci(n - 2); } }

Code a recursive function in JavaScript that finds the factorial of a number

function findFactorial(number) { if(number < 0) { return "Error"; } else if(number == 0 || number == 1) { return 1; } else { return findFactorial(number - 1) * number; } }

Write a quicksort algorithm in JavaScript

function quickSort(list){ let leftPointer = 0; let rightPointer = list.length - 1; return quickSortHelper(list, leftPointer, rightPointer); } function quickSortHelper(list, leftPointer, rightPointer) { if (leftPointer >= rightPointer) { return list; } let pivot = rightPointer; let index = partition(list, leftPointer, rightPointer, pivot); quickSortHelper(list, leftPointer, index - 1); quickSortHelper(list, index, rightPointer); } function partition(list, leftPointer, rightPointer, pivot){ while(leftPointer <= rightPointer){ while(list[leftPointer] < list[pivot]){ leftPointer++; } while(list[rightPointer] > list[pivot]){ rightPointer--; } if(leftPointer >= rightPointer){ return leftPointer; }else{ swap(list, leftPointer, rightPointer); leftPointer++; rightPointer--; } } return leftPointer; } function swap(list, leftPointer, rightPointer) { let temp = list[leftPointer]; list[leftPointer] = list[rightPointer]; list[rightPointer] = temp; }

What kind of sorting algorithm is this: FUNCTION f(collection) SET sortedCollection = [FIRST element in collection] REMOVE FIRST element from collection FOR each value in collection SET sortedCollectionIndex to FIRST INDEX of sortedCollection WHILE sortedCollectionIndex < LAST INDEX of sortedCollection IF value <= sortedCollection[sortedCollectionIndex] THEN INSERT value INTO sortedCollection AT sortedCollectionIndex EXIT WHILE ELSE IF sortedCollectionIndex = LAST INDEX of sortedCollection + 1 THEN APPEND value TO sortedCollection EXIT WHILE END IF INCREMENT sortedCollectionIndex END WHILE END FOR RETURN sortedCollection END FUNCTION

insertion sort

What kind of sorting algorithm is this: FUNCTION f(collection) IF collection length <= 1 THEN RETURN collection ELSE SET mid to ROUND DOWN((LAST INDEX of collection - FIRST INDEX of collection) / 2) CALL mergeSort WITH FIRST INDEX to mid - 1 elements of collection RETURNING left CALL mergeSort WITH mid to LAST INDEX elements of collection RETURNING right CALL merge WITH left AND right RETURNING the new collection END IF END FUNCTION FUNCTION merge(left, right) INIT collection to empty array WHILE left AND right are NOT empty IF FIRST element of left <= FIRST element of right THEN APPEND FIRST element of left to collection REMOVE FIRST element of left from left ELSE IF FIRST element of left > FIRST element of right THEN APPEND FIRST element of right to collection REMOVE FIRST element of right from right END IF END WHILE IF left is NOT empty THEN APPEND left to collection END IF IF right is NOT empty THEN APPEND right to collection END IF RETURN collection END FUNCTION

merge sort