Sorting algorithms, Algorithms -
Selection Sort
An algorithm which passes over the list, finds the smallest item and moves it to the left, then repeats the exercise for the remaining list until it is all sorted.
O(n)
Best case time complexity for insertion sort?
O(n log n)
Best case time complexity for merge sort?
Descending
Falling, going from largest to smallest
Collections.sort()
How to sort a list in Java?
Arrays.sort()
How to sort an array in Java?
Bubble Sort Best
O(n)
Insertion Sort Best
O(n)
Bubble Sort
O(n^2) public void sort(List<Integer> input) { int size = input.size(); for(int i = 0; i < size; i++){ for(int j = i+1; j <size;j++){ if(input.get(i) > input.get(j)){ swap(input, i ,j); } } } } number of comparisons: n + (n-1) + (n-2) + ... + 1 = n * (n-1)/2 = O(n^2) another implementation: public void sort(List<Integer> input) { boolean swapped = false; int finish_index = input.size()-1; do{ swapped = false; for(int i = 0; i < finish_index; i++){ if(input.get(i) > input.get(i+1)){ swap(input, i , i+1); swapped = true; } } finish_index--; }while (swapped); }
Insertion Sort
O(n^2) pseudocode: mark first element as sorted last_sorted_index=1; for each unsorted element: - extract the element - for i = last_sorted_index to 1 if a[i]>current element move it to the right by 1 else put element on i position code: public void sort(List<Integer> input) { for(int i = 1; i< input.size();i++){ int element = input.get(i); int j = i; while(j>0 && input.get(j-1) > element){ input.set(j, input.get(j-1)); j--; } input.set(j,element); } } just the same, each element takes 1 round to get to rightful position. For 1st element the round takes 1 comparison, for 2nd - two comparisons, and so on to n-1. So the whole number of comparisons = n * (n-1)/2 = O(n^2)
Ascending
Rising, going from smallest to largest
Target
The item we are searching for in a search algorithm.
if
a Python keyword that makes a selection
for
a Python keyword that starts a loop
Tree Best
n(log n)
O(n^2)
Average time complexity for insertion sort?
Quick Sort
In this sorting algorithm, a pivot is chosen, and all the elements moved either side of the pivot. This is repeated with another pivot either side, recursively until done.
No
Is heap sort stable?
a[n]
Python code that represents the nth member of an array called a.
selection sort space
o(1)
Bubble Sort
A Sorting Algorithm that passes over the list many times, compares each pair of adjacent items and swaps them if they are in the wrong order.
Selection
A code construct that makes a choice between two or more outcomes
Iteration
A code construct, also known as a loop.
Variable
A named value in a computer program that can be changed by the program code as it runs. "temp" and "num" are examples in our bubble sort program.
Sorting Algorithm
A process commonly used to sort data
Binary Search
A search algorithm that divides the search space in half each time until it finds the target, faster than linear but requires the array to be sorted.
Linear Search
A search algorithm which looks at every element in turn until it finds the target, it is slow but works even on unsorted data.
List
A set of data that can be sorted into order
Merge Sort
A sorting algorithm that sorts partial lists then merges them together.
Search Algorithm
A structured process that finds a target in a set of data.
Array
A variable that can hold list items
Lower bound on comparison-based sorts
Any comparison-based sorting algorithm can not run faster then n*log(n) time by comparison-based we mean the we can not access the "guts" of objects, it can only compare objects. To prove the lower bound, we use pigeonhole principle: if there are n pigeons and n-1 holes, two pigeons will end up in 1 hole. Now, let's say we are sorting array of n elements. There are n! possible inputs. Let's call k - maximum number of comparisons made by algorithm. What is the number of different executions of algorithm? A new execution can be triggered only by comparison. Let's imagine that each comparison induces a 0 or 1 in a k-bit string. E.g. 0 if a[i]<a[j] and 1 if a[i]>=a[j]. There are 2^k such strings, or possible executions. Now, pigeons are n! possible inputs, holes are 2^k possible executions. If 2^k < n!, then two different inputs would have just the same execution path of the algorithm, which is not possible. So: 2^k >= n! n! = n * (n-1) * (n-2) * ... * 1 half of these numbers are >= n/2 n! >= (n/2)^(n/2) 2^k >= (n/2) ^ (n/2) k >= (n/2) * log(n/2) = theta(n*log(n))
O(n log n)
Average case time complexity for merge sort?
O(n log n)
Average case time complexity for quicksort?
O(n^2)
Average case time complexity for selection sort?
O(n log n)
Best case time complexity for quicksort?
O(n^2)
Best case time complexity for selection sort?
Heap Sort
Heap data structure performs Heapify in linear time and lookups in log(n) time we have 1 heapify and n lookups so running time is O(n * log(n)) public void sort(List<Integer> input) { PriorityQueue<Integer> heap = new PriorityQueue<>(input); for(int i = 0; i < input.size(); i++){ input.set(i, heap.poll()); } }
Collections.sort(list, Collections.reverseOrder()) or with comparator
How to sort a list in desc order in Java?
Arrays.sort(arr, new Comparator<Integer>() { @Override public int compare(Integer o1, Integer o2) { return o2.compareTo(o1); } })
How to sort an array in desc order in Java?
// Merge arr1[0..n1-1] and arr2[0..n2-1] // into arr3[0..n1+n2-1] public static void mergeArrays(int[] arr1, int[] arr2, int n1, int n2, int[] arr3) { int i = 0, j = 0, k = 0; // Traverse both array while (i<n1 && j <n2) { // Check if current element of first // array is smaller than current element // of second array. If yes, store first // array element and increment first array // index. Otherwise do same with second array if (arr1[i] < arr2[j]) arr3[k++] = arr1[i++]; else arr3[k++] = arr2[j++]; } // Store remaining elements of first array while (i < n1) arr3[k++] = arr1[i++]; // Store remaining elements of second array while (j < n2) arr3[k++] = arr2[j++]; }
Merge two sorted arrays
Merge Sort
Perfect example of divide-and-conquer algorithm MergeSort(array) - MergeSort(left half of array) - MergeSort(right half of array) - merge(left, right) i=0, j=0 for(k=0..size-1) if(left[i]<right[j]) a[k] = left[i] i++ else a[k] = right[j] j++ runs in O(nlogn) - very fast! but, requires additional space!
Master Method
Recursion in the form: T(n) <= a * T(n/b) + O(n^d) logical meaning of values: a - represents speed of subproblems peliferation b - represents speed of work-per-subproblem reduction
Pivot
Used in Quick Sort, items are compared to this element, and placed one side or the other.
Heap sort
What is an in-place algorithm that is guaranteed O(nlgn)?
When the arrays is already sorted
When does the best case of insertion sort happen? (O(n) time complexity)
O(n^2)
Worst case time complexity for insertion sort?
O(n log n)
Worst case time complexity for merge sort?
O(n^2)
Worst case time complexity for quicksort?
O(n^2)
Worst case time complexity for selection sort?
public class BubbleSort { public static void sort(Comparable[] A) { for(int i = 0; i < A.length; i++) { for(int j = 0; j < A.length; j++) { if(less(A[i], A[j])) { swap(A, i, j); } } } } private static void swap(Comparable[] A, int i, int j) { Comparable tmp = A[i]; A[i] = A[j]; A[j] = tmp; } private static boolean less(Comparable a, Comparable b) { return a.compareTo(b) < 0; } }
Write a code for bubble sort
public class Insertion { public static void sort(Comparable[] A) { for(int i = 0; i < A.length; i++) { for(int j = i; j > 0 && less(A[j], A[j - 1]); j--) { exch(A, j, j - 1); } } } public static void exch(Comparable[] A, int i, int j) { Comparable tmp = A[i]; A[i] = A[j]; A[j] = tmp; } public static boolean less(Comparable a, Comparable b) { return a.compareTo(b) < 0; } }
Write a code for insertion sort
public static void mergeSort(int[] A) { int[] aux = new int[A.length]; mergeSort(A, aux, 0, A.length - 1); } private static void mergeSort(int[] A, int[] aux, int lo, int hi) { if(lo >= hi) return; int mid = lo + (hi - lo) / 2; mergeSort(A, aux, lo, mid); mergeSort(A, aux, mid + 1, hi); merge(A, aux, lo, mid, hi); } private static void merge(int[] A, int[] aux, int lo, int mid, int hi) { // copy to aux[] for(int i = lo; i <= hi; i++) { aux[i] = A[i]; } // merge back to A[] int i = lo; int j = mid + 1; for(int k = lo; k <= hi; k++) { if(i > mid) { A[k] = aux[j++]; } else if(j > hi) { A[k] = aux[i++]; } else if(aux[j] < aux[i]) { A[k] = aux[j++]; } else { A[k] = aux[i++]; } } }
Write a code for merge sort
public static void quickSort(int[] A) { int lo = 0; int hi = A.length - 1; quickSort(A, lo, hi); } public static void quickSort(int[] A, int lo, int hi) { if(lo >= hi) return; int position = partition(A, lo, hi); quickSort(A, lo, position - 1); quickSort(A, position + 1, hi); } private static int partition(int[] A, int lo, int hi) { int i = lo; int j = hi + 1; int k = A[lo]; while(true) { // find item on lo to swap while(A[++i] < k) { if(i == hi) break; } // find item on hi to swap while(A[--j] > k) { if(j == lo) break; } if(i >= j) break; swap(A, i, j); } swap(A, lo, j); return j; } private static void swap(int[] A, int i, int j) { int tmp = A[i]; A[i] = A[j]; A[j] = tmp; }
Write a code for quicksort
public class Selection { public static void sort(Comparable[] A) { for(int i = 0; i < A.length; i++) { int min = i; for(int j = i + 1; j < A.length; j++) { if(less(A[j], A[min])) { min = j; } } exch(A, i, min); } } public static boolean less(Comparable a, Comparable b) { return a.compareTo(b) < 0; } public static void exch(Comparable[] A, int i, int j) { Comparable tmp = A[i]; A[i] = A[j]; A[j] = tmp; } }
Write a code for selection sort
Radix Sort
assumes that data are small integers represented by binary strings, and sorts them one bit at a time for least to most significant bit. Requires n additional space. Can actually sort decimal digits quite the same. On first pass through the array: count how many elements you have for each value (for values 0,1 in case of bit strings, for values [0,1,2, 3,...,9] in case of decimal digits. Calculate beginning and end indexes for elements with each value in resulting array. From end to beginning, start populating new array by putting elements to their buckets (also from end to beginning) Visualization: https://www.cs.usfca.edu/~galles/visualization/RadixSort.html
selection sort advantage
fewest number of swaps...in situations (sorting integers is not one of them) where swapping our data can take a very long time due to very large amounts of it to move back and forth, swapping might have such a LARGE constant on it that it would overshadow everything else. In that case, we might want selection sort.
Quick Sort
motivation: nlogn algorithm that would work in-place: algorithm: QSort(array) 1) find pivot p 2) partition around the pivot - so that every element to the left is less than the pivot, and every element to the right is bigger then the pivot finds new pivot position k 3) qsort(left from k), qsort(right from k) Partitioning routine: a) let p be the 1st element. If not - just swap (1st element, p) b) i = 1 - generic iterator k = 1 - left/right separator c) for i = 1 .. n-1 if(a[i]<a[k]) swap(i,k) k++ i++ d) swap(0,k-1), return k-1 worst case - n^2 randomized pivot selections - n*log(n) to get n*log(n) speed, we need each pivot to give at least 25-75 split this happens with probability alpha = 0.5 see full proof in the text book
insertion sort best case
n
merge sort advantage
n * lg n in all cases, even worst case
Heapsort Average
n(log n)
Heapsort Best
n(log n)
Heapsort Worst
n(log n)
Merge Sort Average
n(log n)
Merge Sort Best
n(log n)
Merge Sort Worst
n(log n)
Quicksort Average
n(log n)
Quicksort Best
n(log n)
Quicksort Worst
n(log n)
Tree Average
n(log n)
Bubble Sort Average
n^2
Bubble Sort Worst
n^2
Insertion Sort Average
n^2
Insertion Sort Worst
n^2
Selection Sort Best
n^2
Tree Worst
n^2
insertion sort worst case
n^2
quick sort worst case
n^2
selection sort average case
n^2
selection sort best case
n^2
selection sort worst case
n^2
insertion sort average case
n^2 but half the time as worst case
Merge sort best case
nlg(n)
merge sort average case
nlg(n)
merge sort worst case
nlg(n)
quick sort average case
nlg(n)
quick sort best case
nlg(n)
Bucket sort, Counting sort, Radix Sort
non-comparison based algorithms Bucket Sort: used when sorting samples from uniform distribution on [0,1] E.g. sorting array of size n. Pre-allocate k buckets with width 1/k In one pass on the array: if element is between 0 and 1/k - put it to 1st bucket, between 1/k and 2/k - 2nd bucket, and so on. Because input data is supposed to be distributed uniformly at random, the number of elements in each bucket will be extremely small, and you can use insertion sort to sort each bucket. Counting Sort: the same idea, but data are small integers, e.g. from [0,..,k]. again bucketing the data, but this time to k buckets
binary search best case
o(1)
binary search space
o(1)
insertion sort space
o(1)
binary search worst case/avg
o(lgn)
merge sort space
o(n)
quick sort advantage
on average, fastest known comparison-based sorting algorithm; i.e. the only faster algorithms we know of are designed for specific kinds of data, rather than working on anything we can compare. Both quicksort and mergesort have the same order of growth, but in terms of constant factors of that n * lg n term, quicksort's constants are lower.
insertion sort disadvantage
quadratic for any randomly arranged array, i.e. it's not better than quadratic unless the array *is* sorted a lot already
selection sort disadvantage
quadratic in all cases, can't even be improved upon if the array is partially sorted. Works okay for small arrays but insertion sort works better for those arrays...so in general, Selection Sort is not generally useful. It's only useful in the one case listed as an advantage above.
quick sort disadvantage
the quadratic worst case. It's very unlikely to occur but it *can* happen, making quicksort not the best choice if you *need* to be at n * lg n time.
quick sort space
unstable partition takes o(1) at most o(logn) after partition
merge sort disadvantage
uses linear extra memory (the temporary array needed by merge)...this is especially a problem if you have a VERY large array to sort, since you might not even *have* enough memory left over to create another array that size. Also, not quite as fast as quicksort's average case, when you consider the constant factors (since they both have n * lg n order of growth)
insertion sort advantage
works well for partially sorted or completely sorted arrays; also good for small arrays since quicksort and mergesort tend to be overkill for such arrays
