Searching, Sorting, Concurrency, and Threads

There are 3 types of internal sorts

(i) SELECTION SORT :- Ex:- Selection sort algorithm, Heap Sort algorithm (ii) INSERTION SORT :- Ex:- Insertion sort algorithm, Shell Sort algorithm (iii) EXCHANGE SORT :- Ex:- Bubble Sort Algorithm, Quick sort algorithm

Limitations Of Insertion Sort :

* It is relatively efficient for small lists and mostly - sorted lists. * It is expensive because of shifting all following elements by one.

External Sorts Example

Merge Sort

Bubble sort disadvantages

Often considered the most inefficient sorting method since it must exchange items before the final location is known. These "wasted" exchange operations are very costly

Divide and Conquer

A program design strategy in which tasks are broken down into subtasks, which are broken down into sub-subtasks, and so on, until each piece is small enough to code comfortably. These pieces work together to accomplish the total job.

Give a 5-integer array such that it elicits the worst-case running time for insertion sort.

A simple example is: [5, 4, 3, 2, 1]. Any 5-integer array in descending order would work.

Selection Sort

A sort algorithm that repeatedly scans for the smallest item in the list and swaps it with the element at the current index. The index is then incremented, and the process repeats until the last two elements are sorted.

(f) Fill in the blank using the choices below: A. QuickSort B. MergeSort C. SelectionSort D. InsertionSort E. HeapSort N. (None of the above) _____: Bounded by Ω(N log N) lower bound.

A, B, C

(f) Fill in the blank using the choices below: A. QuickSort B. MergeSort C. SelectionSort D. InsertionSort E. HeapSort N. (None of the above) _____: Never compares the same two elements twice.

A, B, D

Merge Sort Algorithm

Algorithm: 1. If less than two elements, return a copy of the list (base case!) 2. Sort the first half using merge sort. (recursive!) 3. Sort the second half using merge sort. (recursive!) 4. Merge the two sorted halves to obtain the final sorted array.

Quick Sort

An example of a divide-and conquer algorithmic technique. It is also called partition exchange sort. Uses recursive call for sorting the elements.

Merge Sort

An example of the divide and conquer strategy. This algorithm sorts a list of items by dividing it into two halves, sorting each half recursively, and then merging the sorted halves back together.

(f) Fill in the blank using the choices below: A. QuickSort B. MergeSort C. SelectionSort D. InsertionSort E. HeapSort N. (None of the above) _____: Has a worst-case runtime that is asymptotically better than Quicksort's worst-case runtime.

B, E

Selection Sort Time Complexity

Best: O(n^2) Average: O(n^2) Worst: O(n^2)

(f) Fill in the blank using the choices below: A. QuickSort B. MergeSort C. SelectionSort D. InsertionSort E. HeapSort N. (None of the above) _____: In the worst case, performs Θ(N) pairwise swaps of elements.

C

Binary Search

a search algorithm that starts at the middle of a "sorted set" of numbers and removes half of the data; this process repeats until the desired value is found or all elements have been eliminated.

Linear Search

a search algorithm which checks each element of a list, in order, until the desired value is found or all elements in the list have been checked.

What is Sorting?

A process through which the data is arranged in ascending or descending order.

Insertion sort is one of the elementary sorting algorithms with ____ worst-case time.

O(n^2)

combine

Once we have solved all the subproblems, we combine their solutions to get the final answer to the original problem.

Give some reasons as to why someone would use merge sort over quicksort.

Some possible answers: Merge sort has Θ(N log N) worst-case runtime versus quicksort's Θ(N^2). Mergesort is stable, whereas quicksort typically isn't. Mergesort can be highly parallelized because, as we saw in the first problem, the left and right sides don't interact until the end. Mergesort is also preferred for sorting a linked list.

TRUE OR FALSE: Binary Search is useful when there are large number of elements in an array and they are sorted.

TRUE! A necessary condition for Binary search to work is that the list/array should be sorted.

True or False: The merge-sort tree associated with an execution of merge sort on a sequence of size n has height ⌈logn⌉

TRUE: The height of the merge-sort tree associated with an execution of merge sort on a sequence of n integers is O(log n). Merge sort is a logarithmic-time sorting algorithm, which means that the time complexity of the algorithm grows logarithmically with the size of the input.

Quick Sort Running time analysis

The advantage of this quicksort is that we can sort "in-place", i.e., without the need for a temporary buffer depending on the size of the inputs.(cf. mergesort)

Suppose you are sorting the following list of numbers in ascending order using bubble sort: [16, 5, -1, 4, 12, 17, 3, 10, 5, 9]. After the first pass through the numbers, what value would appear on the right of the list? A. -1 B. 9 C. 17 D. 5 E. 16

The largest value, 17, would 'bubble up' to the right of the list during the first pass.

Suppose you are sorting the following list of words into alphabetical order using bubble sort: [apple, orange, banana, papaya, lemon, pumpkin, squash, tomato]. After the first pass through the list, what word would appear on the right of the list? A. tomato B. apple C. papaya D. pumpkin E. squash

The largest value, tomato, would 'bubble up' to the right of the list during the first pass.

What happens to the list in selection sort?

The list is divided into two sublists, sorted and unsorted, which are divided by an imaginary wall.

When is insertion sort used?

used when the data is nearly sorted (due to its adaptiveness) or when the input size is small (due to its low overhead).

Selection Sort Steps

ð The list is divided into two sublists, sorted and unsorted, which are divided by an imaginary wall. ð We find the smallest element from the unsorted sublist and swap it with the element at the beginning of the unsorted data. ð After each selection and swapping, the imaginary wall between the two sublists move one element ahead, increasing the number of sorted elements and decreasing the number of unsorted ones. ð Each time we move one element from the unsorted sublist to the sorted sublist, we say that we have completed a sort pass. ð A list of n elements requires n-1 passes to completely rearrange the data.

Features of Binary Search

1. It is great to search through large sorted arrays. 2. It has a time complexity of O(log n) which is a very good time complexity. It has a simple implementation.

Description: First, the values split until single-element groups And sorts sub-group while gradually merging When to use: Used for sorting linked list Based on the characteristics above, what sorting algorithm is being described? 1. Merge Sort 2. Bubble Sort 3. Selection Sort 4. Quick Sort 5. Heap Sort 6. Insertion Sort 7. None of the above

1. Merge Sort

Description: Compares adjacent element pairs and swaps if they are not ascending When to use: Goof for understanding sorting and sort small data set Based on the characteristics above, what sorting algorithm is being described? 1. Merge Sort 2. Bubble Sort 3. Selection Sort 4. Quick Sort 5. Heap Sort 6. Insertion Sort 7. None of the above

2. Bubble Sort

You are working on an embedded device(an ATM) that only has 4 KB of free memory and you wish to sort the 2,000,000 transactions with-drawal history by the amount of money withdrawn (discarding the original order of transactions). Which sorting algorithm should be used for the scenario above? 1. InsertionSort 2.MergeSort 3. HeapSort 4. CountingSort 5. None of the above

3. HeapSort There is no point in at least talking about using bubble, insertion, selection sorts as they are not very efficient in memory wise and performance wise. Merge sort is space O(n), so you better look for something else.

Description: For each location, it checks whether it's less than the elements to its right. When to use: Beneficial to learn sorting algorithms Based on the description above, which sorting algorithm should be the best to use? 1. Merge Sort 2. Bubble Sort 3. Selection Sort 4. Quick Sort 5. Heap Sort 6. Insertion Sort 7. None of the above

3. Selection Sort

Description: Repeatedly splits elements based on chosen pivot number. When to use: Used for sorting arrays and medium-sized data set. Based on the description above, which sorting algorithm should be the best to use? 1. Merge Sort 2. Bubble Sort 3. Selection Sort 4. Quick Sort 5. Heap Sort 6. Insertion Sort 7. None of the above

4. Quick Sort

Description: Proceeds by heapify, swap, and insert When to use: Ideal for big data sets Based on the description above, which sorting algorithm should be the best to use? 1. Merge Sort 2. Bubble Sort 3. Selection Sort 4. Quick Sort 5. Heap Sort 6. Insertion Sort 7. None of the above

5. Heap Sort

To determine which of your Facebook friends were early adopters, you decide to sort them by their Facebook account IDs, which are 64-bit integers. (Recall that you are super popular, so you have very many Facebook friends.) Which sorting algorithm should be used for the scenario above? 1. InsertionSort 2.MergeSort 3. HeapSort 4. CountingSort 5. Radix Sort 6. None of the above

5. Radix Sort

Description: Sorts by comparing, moving, and placing When to use: Used to sort small data set and check already sorted list. Based on the description above, which sorting algorithm should be the best to use? 1. Merge Sort 2. Bubble Sort 3. Selection Sort 4. Quick Sort 5. Heap Sort 6. Insertion Sort 7. None of the above

6. Insertion Sort

Q-7: For which of the problems would the bubble sort algorithm provide an appropriate solution. Choose all that apply. A. Arranging books on a bookshelf by author's last name. B. Sorting a basket of laundry into socks, shirts, shorts, and sheets. C. Arranging a deck of cards from the lowest to the highest value cards. D. Sorting a stack of paper money into denominations -- i.e., $1, $5, $10 etc. E. Looking up a name in the phone book.

C. True. Bubble sort would be appropriate for sorting cards by their face value D. True. Bubble sort would be appropriate for sorting paper money by their denominations since we know that $1 come before $5 and $5 come before $10, etc.

Suppose you are sorting the following list of words in alphabetical order using bubble sort: [apple, banana, lemon, tomato, orange, squash, papaya, pumpkin]. Which of the following gives the correct order of the list after two passes through the list? A. [apple, banana, lemon, squash, tomato, orange, papaya, pumpkin] B. [apple, banana, lemon, orange, papaya, squash, tomato, pumpkin] C. [apple, banana, lemon, orange, papaya, pumpkin, squash, tomato] D. [apple, banana, lemon, orange, papaya, pumpkin, tomato, squash] E. [apple, banana, lemon, tomato, orange, squash, papaya, pumpkin]

C. [apple, banana, lemon, orange, papaya, pumpkin, squash, tomato] The two largest values, squash and tomato, would 'bubble up' to the right of the list after two passes.

Quicksort Worst Case

Each partition gives unbalanced splits and we get: Θ(n2)

Quicksort Best Case:

Each partition splits array in halves and gives: Θ(nlogn) [using Divide and Conquer master theorem]

(T/F) Quicksort has a worst-case runtime of Θ(NlogN), where N is the number of elements in the list that we're sorting.

False, quicksort has a worst-case runtime of Θ(N^2), if the array is partitioned very unevenly at each iteration.

True or False: Heapsort is stable

False. Stability for sorting algorithms means that if two elements in the list are defined to be equal, then they will retain their relative ordering after the sort is complete. Heap operations may mess up the relative ordering of equal items and thus are not stable. As a concrete example (taken from StackOverflow), consider the max heap: 21, 20a, 20b, 12, 11, 8, 7.

conquer

For each subproblem, we use recursion to solve it again. This means we break down the subproblem into even smaller pieces until we reach a point where we can solve it directly.

Which of these are true about Linear Search Algorithm? I. It is used for unsorted and unordered small list of elements. II. It has a time complexity of O(n), which means the time is linearly dependent on the number of elements, which is not bad, but not that good too. III. It doesn't have a very simple implementation

I and II III. LSA does have a simple implementation

We have a system running insertion sort and we find that it's completing faster than expected. What could we conclude about the input to the sorting algorithm? I. The input is small or the array is nearly sorted. Note that insertion sort has a best-case runtime of Θ(N), which is when the array is already sorted. II. The input is large and the array is nearly sorted. III. The worst-case (and average-case) complexity of the insertion sort algorithm is O(n²).

I and III

Divide

If the number of inputs is less than a certain limit (let's say, 1 or 2), solve the problem directly with a simple method and return the result. Otherwise, break up the inputs into smaller groups of 2 or more pieces each.

Merge Sort: How many merges?

In general: log2(n) merges of n elements. Ex: Merge sort on 32 elements. log2(32) = x 2^(x)= 32 xln(2)= ln(32) x = ln(32)/ln(2) x= 5ln(2)/ln(2) x = 5 merges

Merge Sort Input and Output

Input: List a of n elements. Output: Returns a new list containing the same elements in sorted order.

You are running a library catalog. You know that the books in your collection are almost sorted in ascending order by title, with the exception of one book which is in the wrong place. You want the catalog to be completely sorted in ascending order. Which sorting algorithm should be used for the scenario above? 1. InsertionSort 2.MergeSort 3. RadixSort 4. HeapSort 5. CountingSort

Insertion sort will run in O(n) time in this setting. InsertionSort would be the most appropriate choice for efficiently sorting the nearly ordered books in the library catalog, especially considering the small disruption caused by only one book being out of place.

Sorting can be classified into two types:

Internal Sorts External Sorts

Insertion Sort

Most common sorting technique used by card players. Again, the list is divided into two parts: sorted and unsorted. In each pass, the first element of the unsorted part is picked up, transferred to the sorted sublist, and inserted at the appropriate place. A list of n elements will take at most n-1 passes to sort the data.

Bubble Sort

Moving through a list repeatedly, swapping elements that are in the wrong order. In other words: It makes multiple passes through a list. It compares adjacent items and exchanges those that are out of order. Each pass through the list places the next largest value in its proper place. In essence, each item "bubbles" up to the location where it belongs.

(f) Fill in the blank using the choices below: A. QuickSort B. MergeSort C. SelectionSort D. InsertionSort E. HeapSort N. (None of the above) _____: Runs in best case Θ(log N) time for certain inputs.

N

When does the worst-case for QuickSort occurs?

The worst-case occurs when the list is already sorted and last element chosen as pivot.

True or False: In merge sort, the total number of elements per level is always n.

True

True or False: Quicksort involves partitioning, and 2 recursive calls.

True

True or False: In Merge Sort, it takes n appends to merge all pairs to the next higher level. Multiply the number of levels by the number of appends per level.

True Example: say we are merging two pairs of 2-element lists: 8 appends for 8 elements

True or False: In Merge Sort, it takes log2(n) merges to go from n groups of size 1 to a single group of size n.

True.

True or False: The Bubble Sort algorithm has a time complexity of O(N^2)

True.

There are many sorting algorithms, such as: i. Selection Sort ii. Bubble Sort iii. Insertion Sort iv. Merge Sort v. Quick Sort Which are the foundation for fasters and more efficient algorithms?

i. Selection Sort ii. Bubble Sort iii. Insertion Sort

Internal Sorting

when all of the elements that needs to be sorted are in memory.

External Sorting

when elements to be sorted cannot fit into memory all at once. The data is divided into chunks/blocks. A chunk is loaded on memory, sorted, then another chunk is loaded.