Test 1--Algorithms
Logarithmic Efficiency
Cuts problem size by constant fraction on each iteration
Karatsuba's multiplication algorithm
Formulated the first integer multiplication algorithm to break the O(n^2) barrier; Based on 2 key ideas: divide and conquer strategy and math trick due to Gauss
Worst-case complexity
Most commonly used, an upper bound on running time for any possible input of size n
What are properties of logarithms?
Product rule, quotient rule, power rule
When the running time T(n) of an algorithm depends on other characteristics of the input besides its size n, we distinguish between what kinds of analysis?
Worst-case complexity, best-cast complexity, average-case complexity
Towers of Hanoi recursive complexity
2^n - 1 = O(2^n)
Napier's Bones
A mechanical device for doing multiplication and division based on the lattice method was invented by John Napier around 1600
What other subject did Al-Kwarizimi influence?
Algebra
Factorial Efficiency
Algorithm generates all permutations of n-element set
What is a search algorithm?
An algorithm that searches for an item in an array, or linked list, or more complex data structure ( binary tree, AVL tree, etc.)
What is Big-Omega?
Asymptotic lower bound;
Insertion Sort Analysis
Best case: O(n), worst case: O(n^2), average case O(n^2)
What is the essential feature of an algorithm?
Can be executed automatically, in a machine-like way
Sequential search
Check every element of a list sequentially until a match is found; best case: bigTheta(1), worst case: bigTheta(n)
Master Theorem
Cookbook approach for solving recurrences of the form T(n) = aT(n/b) + bigTheta(n^d) where a>=1, b>1 and d are constants
What is integer factorization?
Decomposition of a composite number into a product of smaller integers (easier than GCD)
Exponential Algorithm
Ex: Towers of Hanoi; exponential algorithms are not practical. If the disk moves at a rate of 1 disk per second, then it would take 2^32 = 136 years to move 32 disks
Average-case complexity
Expected performance averaged over all possible inputs of size n; very useful but much more difficult analysis
What are two ways to analyze algorithm scalability?
Experimentation: implement algorithm and run it with different input sizes; complexity: abstract form of analysis that doesn't require implementation
What reduced multiplication and division times?(change)
Logarithmic stuff (change this I wasn't paying attention)
Quadratic Efficiency
Loop inside loop = "nested loop"
Cubic Efficiency
Loop inside nested loop
Best-case complexity
Lower bound on running time for any possible input of size n; usually not very informative and rarely used
Who is Alan Turing?
Played by Benedict Cumberbatch (it was gr8). Gave first formal definition of 'algorithm' by creating the turing machine
What is Moore's Law?
Predicts the number of transistors that fit on a computer chip will double every one and a half to two years;
n-Log-n Efficiency
Some divide and conquer algorithm
Where did the word computer come from?
Someone who computes, it was a job. Electronic computer distinguished between human computer and nowadays computer (was hella big though)
Towers of Hanoi
There an n disks labeled 1,2,3,...,n and three towers labeled A,B, and C; No disk can be on top of a smaller disk at any time; all the disks are initially placed on tower A, Only one disk can be moved at a time, and it must be the top disk on the tower
Interpolation Search
Uses knowledge about the distribution of values in an array to improve the efficiency of the search; Average case O(log(log(n))), worst case O(n)
Recursive Algorithm
a form of decomposition where a problem is solved by decomposing it into one of more simpler problems hat have the same form as the original
Recursive Function
a function which calls itself somewhere in the function body
Tree Recursion
a recursive function calls itself two or more times before returning a value
Recursion
a technique for defining data structure or algorithms in terms of themselves
What is the average case for quicksort?
bigTheta(nlgn)
Quicksort
divide-and-conquer algorithm; partition the array into two subarrays. Sort the two subarrays by recursive calls to quick sort. The subarrays are sorted in place. No extra work is needed to combine them
What is an algorithm?
finite set of precise instructions for performing computation something 'cause he flipped the fukkin slide too fast
Divide and conquer
important strategy for algorithm design that is typically implemented using tree recursion
Sorting
sort a sequence of n elements in non-decreasing order
What does the running time of quicksort depend on?
whether the partition is balanced or not
What are the 3 steps for divide and conquer?
1) Divide the problem into two or more smaller instances of the same problem, ideally of about the same size; 2) conquer the sub problems by solving them recursively. If they are small enough, just solve them in a straightforward manner; 3) Combine the solutions to create a solution to the original problem
Kolmogorov
A famous Russian mathematician and one of the founders of algorithmic complexity theory, conjectured in 1952 that grade-school multiplication is asymptotically optimal, which means that any multiplication algorithm must perform bigTheta(n^2) elementary equestions
Tail Recursion
An algorithm uses tail recursion if it both uses linear recursion and makes a recursive call as its very last operation
What is the second abstraction of complexity analysis?
Analysis of the running time for input size n for large n; ignores lower order terms and constant factors; written with asymptotic notation (omega theta, big O)
What is Big-Theta?
Asymptotic tight bound;
What is Big-O notation?
Asymptotic upper bound; cannot compare algs in same complexity class; gives sensible comparisons for large ns and differen't complexity classes
Selection Sort Anaylysis
Best case, worst case, avg case are all O(n^2)
Binary Search
First look at the middle element of the array. If not equal to the target element, the target is either in the left sub-array or the right sub-array. Divide the array in half and repeat the search in the half that contains the target; Worst case O(log(n)), average case O(log(n)), best case O(1)
What other names is the greatest common divisor known by?
Greatest common factor, highest common factor, greatest common measure, highest common divisor
Substitution method
Guess the form of the solution then use mathematical induction to show it's correct, Use recursion trees
Why are the bases of logarithms ignored in asymptotic notation?
If a base of a a log is changed from one constant to another, the value is altered by only a constant.
Merge Sort
Invented by John Von Neumann in 1945; it is a divide and conquer algorithm; Divide the n-element sequence to be sorted into two subsequences of n/2 elements each, Sort the two subsequences recursively using merge sort, merge the two sorted subsequences to product the sorted answer
Partitioning
Key step of quicksort algorithm; given the selected pivot, partition the remaning elements into two smaller sets
Insertion Sort
Like sorting a hand of playing cards; start with an empty left hand and the cards face down on the table, remove one card at a time from the table and insert it into the correct position in the left hand, compare it with each of the cards already in the hand from right to left, the cards held in the left hand are sorted
What is the origin of the word 'algorithm'?
Muhammed ibn Musa al-Khwarizmi (how the fk) wrote a book about mathematical processes, then centuries later Europeans called them 'algorithms' due to the latin form of his name (oh okay) 'algorithmi'
Merge Sort Analysis
Running time T(n) of Merge sort: Divide: computing the middle takes bigTheta(1), Conquer: solving 2 subproblems takes 2T(n/2), combine: merging n elements takes bigTheta(n)....Total: T(1) = bigTheta(1) and T(n) = 2T(n/2)+bigTheta(n)
How are recurrence relations solved?
Substitution method or master theorem
When do logarithms occur?
The data set keeps getting divided by 2
What is a turing machine?
Theoretical generalized computer; finite number of states (like the brain); sensor could read/write/erase symbols onto tape
How is the greatest common divisor notated for two integers a and b?
gcd(a,b) or (a,b)
When does the best case occur for quicksort?
when every call to partition results in the most balanced partition
When does the worst case occur for quicksort?
when every call to partition results in the most unbalanced partition
What have algorithms been used for in the past thousands of years?
Mathematics: addition, subtraction, multiplication, division, square roots, equations, prime numbers, pie (yum)
Exponential Efficiency
Algorithm generates all subsets of n-element set
Constant Efficiency
Algorithm ignored input (i.e., can't even scan input)
Linear Efficiency
Algorithm scans its input (at least)
How can you analyze a graph to visualize growth orders?
As you go to the right, a faster growing function eventually becomes larger;
Open Addressing
Collision resolution method where all elements are stored in the hash table itself; when collisions occur, use a systematic procedure to store elements in free slots on the table
Chaining
Collision resolution method where you store all elements that hash to the same slot in a linked list; store a pointer to the head of the linked list in the hash table slot
What is asymptotic analysis?
Comparison of running times of functions in the limit (asymptotically); measure of how fast each function grows; as 'm' goes to infinity (big-O basically)
What is the inverse of logarithmic growth?
Exponential. Logarithmic is extremely slow, exponential is extremely fast;
Selection Sort
Find the smallest element in the array, exchange it with the element in the first position, find the second smallest element and exchange it with the element in the second position, continue until the array is sorted
Other than computers, what else have algorithms been run on?
Humans! Humans did computing back in the day
What is prime factorization?
Integer factorization where the integers are prime numbers
What is a slide rule?
Invented by Napier; used for calculations
Grade-school "long" multiplication
Involves the multiplications of an n-digit number by a single digit, plus the addition of n numbers which have at most 2n digits. Complexity O(n^2)
Why study sorting?
It is the best studied problem in computer science, with a variety of different algorithms; most of the interesting ideas we will encounter in the course can be taught in the context of sorting such as divide- and-conquer, randomized algorithms, and lower bounds
What is the first abstraction of complexity analysis?
Operation count: identify key operation of algorithm; count number of times operation is performed
Who is John Napier?
Scottish mathematician, physicist, astronomer, theologian; invented logarithms popularized decimal points; thought the pope was the antichrist, world was supposed to end in 1688 or 1700 (lol oops)
Lattice Method of Multiplication
Set up the grid, fill in products, add diagonally right to left and carry as necessary to the next diagonal. Complexity of O(n^2)
Egyptian Multiplication
Start with two numbers. On the left start with 1, on the right use the multiplicative. Keep doubling both numbers until the left side gets a close as possible to the original multiplicative but not larger. Subtract the left side numbers from the original multiplicative until you reach 0. Start the left side numbers that are being subtracted. Add the corresponding right side of numbers of the starred positions. Complexity O(n^2)
What is the general form of a recurrence relation?
T(n) = bigTheta(1) if n<=c, T(n) = aT(n/b)+f(n), where a is the number of recursive calls, n/b is the size of the input for a recursive call, and f(n) is the extra work for dividing a problem before the recursive calls and then combining the solutions to the subproblems after return from the recursion
Hashing
Technique that where the average case access time can be O(1); Basic Idea: Items are stores in an array of size N, the preferred position in the array is computed using a hash function of the item's key; when adding an item, if the preferred position is occupied, a collision resolution method is used to find another position to store it; Search will cost bigTheta(n) time in the worst case, however all operations can be made to have an expected complexity of bigTheta(1)
What are the two parts of a recursive definition?
The base case: a stopping condition; the recursive step: an expression of the computation or definition in terms of itself
Why are the bases of exponentials NOT ignored in asymptotic notation?
The bases differ by an exponential factor, not a constant factor
What is a logarithm?
The common (base-10) logarithm of a number is the power to which 10 must be raised to give the number; defined for ALL positive numbers, not for negative numbers;
What is the greatest common divisor?
The largest positive integer that divides two integers without a remainder
How are logarithms used in computer science?
We use base 2, 'cause we always work with binary; log2n tells us how many bits we need to represent n possibilities
Rehashing
When the load factor exceeds a threshold, double the table size (smallest prime > 2 * old table size); rehash each record in the old table into the new table
Grade- school addition in binary
You can add two binary numbers one column at a time starting from the right, just as you add two decimal numbers; complexity is O(n). Addition is much easier than multiplication in terms of algorithmic complexity
Recurrence Relations
arise when analyzing the running time of recursive algorithms, especially divide-and-conquer algorithms
Why do we need the recursive step in recursive functions?
execution must "drive" computation to a base case so the recursion will stop. The recursive step is intended to ensure that the computation eventually terminates
When can you not use the master theorem?
if T(n) is not monotone, f(n) is not a polynomial, b cannot be expressed as a constant
Gauss' Optimization
input a,b,c,d; output ac, bd, ad+bc; ad+bc=(a+b)(c+d)-ac-bd
Worst-case time complexity of a computational problem
is the worst-case complexity of the best algorithm that can solve the problem
Worst-case time complexity of an algorithm
its running time on the worst-case input instance
What are 'keys' for asymptotic notation?
lilO() is <; lilOmega() is >; theta() is =; bigO() is <=; bigOmega() is >=;
State the order of growth rates from slowest to fastest.
log(n) --> n --> n*log(n) --> n^2 --> n^3 --> 2^n --> n!
What is the relationship of logs to exponentials?
logxY = z can be written as x^Z = Y; the log is the exponent
Linear Recursion
the simplest form of recursion, where the recursive function makes at most one recursive call each time it is invoked
