Midterm
1.4 C++ Classes
**1.4.1 BASIC CLASS SYNTAX** *Public Member Functions* - mutators (normally "void" which does not return anything, but modifies what is happening in the list/vector/array) - accessors (normally int, double, etc. any type that can access/return that particular type depending on the parameters & private variables) *Private Variables (Information Hiding)* - variables that can only be accessed by public member functions in the implementation file - By using private data members, we can change the internal representation of the object without having an effect on other parts of the program that use the object *In a class, all members are private by default; therefore, you MUST have "public:" declared in the h files *Constructors* = is a method that describes how an instance of the class is constructed - If no constructor is explicitly defined, one that initializes the data members using language defaults is automatically generated - Must have constructors in order to initialize the private member variable(s) either to the parameter variable(s) or to 0 *Each instance of a class is called an object Class TableItem -> TableItem A (object) - initializing TableItem A will be able to call the public member functions declared within the TableItem.h file and implemented in the TableItem.cpp files ex: A.asPos(); **1.4.2 EXTRA CONSTRUCTOR SYNTAX AND ACCESSORS** *Default Parameters* = can be used in any function, but they are most commonly used in constructors *Initialization List* = what the constructor uses prior to the body of the constructor - used to initialize the data members directly - using initialization lists instead of an assignment statement in the body saves time in the case where the data members are class types that have complex initializations. ex: Class Node can have a constructor with two parameters which then can be a pointer data member in a Class TableItem class Node { public: string word; int position; Node *next; Node(string word, int position) : word(word), position(position), next(0) {} }; class TableItem { private: Node *header; Node *tail; }; *Explicit Constructor* - You should make all one-parameter constructors explicit to avoid behind-the-scenes type conversions. ex: Node *x = new Node(w, count); - The use of explicit means that a one-parameter constructor cannot be used to generate an implicit temporary *Implicit Type Conversion* = a temporary object is created that makes an assignment (or parameter to a function) compatible *Constant Member Function* - Accessor = A member function that examines but does not change the state of its object (normally int, double, etc. any type that can access/return that particular type depending on the parameters & private variables) - Mutator = A member function that changes the state of the object (normally "void" which does not return anything, but modifies what is happening in the list/vector/array) • cannot be applied to constant objects *By default, all member functions are mutators Const (in member functions) 1) Const Accessor = to make a member function an accessor, we must add the keyword const AFTER the closing parenthesis that ends the parameter type list 2) Call-By-Constant Reference = appropriate for large objects that should not be altered by the function and are expensive to copy (within the parameter) ex: string randomItem( const vector<string> & arr ); // returns a random item in arr & does not alter the array 3) Return-By-Constant Reference = we do not want to allow changes to be made by the caller by using the return value; in this case it is needed since arr itself (within the above parameter) is a non-modifiable vector **1.4.3 SEPARATION OF THE INTERFACE (.h) AND IMPLEMENTATION (.cpp)** *Preprocessor Commands* - The interface is typically placed in a file that ends with .h, which also ends with #endif - Source code that requires knowledge of the interface must #include the interface file (.h file) - Inclusion Guards (#ifndef and #define) = to prevent the danger that an interface might be read twice in the course of compiling a file *Scope Resolution Operator* - the :: is called the scope resolution operator - used to identify the class that certain member functions are part of *Signatures Must Match Exactly* - the signature of an implemented member function must match exactly the signature listed in the class interface **1.4.4 VECTOR AND STRING** - basics
1.5 C++ Details
**1.5.1 POINTERS** - Pointer Variable = is a variable that stores the address where another object resides (to store a list of items, we could use a contiguous array, but insertion into the middle of the contiguous array requires relocation of many items) • along with each object is a link to the next object • this link is a pointer variable, because it stores a memory location of another object *Dynamic Object Creation* IntCell *m; m = new IntCell; // Preferred in this text *Garbage Collection and Delete* - Memory Leak = When an object that is allocated by new is no longer referenced, the delete operation must be applied to the object (through a pointer). Otherwise, the memory that it consumes is lost (until the program terminates) -> memory leak - Automatic Variable = do not use new when an automatic variable can be used instead. In the original program, the IntCell is not allocated by new but instead is allocated as a local variable. In that case, the memory for the IntCell is automatically reclaimed when the function in which it is declared returns *Address-of Operator (&)* = This operator returns the memory location where an object resides **1.5.2 LVALUES, RVALUES, AND REFERENCES** vector<string> arr( 3 ); const int x = 2; int y; ... int z = x + y; string str = "foo"; vector<string> *ptr = &arr; - Lvalue = is an expression that identifies a non-temporary object (if you have a name for a variable, it is an Lvalue) • arr, str, arr[x], &x, y, z, ptr, *ptr, (*ptr)[x] - Rvalue = is an expression that identifies a temporary object or is a value (such as a literal constant) not associated with any object (normally on the right side of an assignment, which assigns a non-temporary variable, Lvalue, equal to a temporary variable, Rvalue) • if the function call computes an expression whose value does not exist prior to the call and does not exist once the call is finished unless it is copied somewhere, it is likely to be an rvalue • "foo" and "x + y" - Lvalue Reference = is declared by placing an & after some type. An lvalue reference then becomes a synonym (i.e., another name) for the object it references. string str = "hell"; string & rstr = str; // rstr is another name for str rstr += 'o'; // changes str to "hello" bool cond = (&str == &rstr); // true; str and rstr are same object string & bad1 = "hello"; // illegal: "hello" is not a modifiable lvalue string & bad2 = str + ""; // illegal: str+"" is not an lvalue string & sub = str.substr( 0, 4 ); // illegal: str.substr( 0, 4 ) is not an lvalue - Rvalue Reference = is declared by placing an && after some type. An rvalue reference has the same characteristics as an lvalue reference except that, unlike an lvalue reference, an rvalue reference can also reference an rvalue (i.e., a temporary). string str = "hell"; string && bad1 = "hello"; // Legal string && bad2 = str + ""; // Legal string && sub = str.substr( 0, 4 ); // Legal **1.5.3 PARAMETER PASSING** - Call-by-Value = the actual argument is copied into the formal parameter • The local parameters are copies of the original arguments passed in • Changes made in the function to these variables do not affect originals Where Call-by-Value is Not Sufficient As the Only Parameter-Passing Mechanism in C++ 1) double average( double a, double b ); // returns average of a and b 2) void swap( double a, double b ); // swaps a and b; wrong parameter types 3) string randomItem( vector<string> arr ); // returns a random item in arr; inefficient - Call-by-Reference/Call-by-lvalue-Reference • The local parameters are references to the storage locations of the original arguments passed in. • Changes to these variables in the function will affect the originals • No copy is made, so overhead of copying (time, storage) is saved ex: void swap( double & a, double & b ); // swaps a and b; correct parameter types 1. Call-by-value is appropriate for small objects that should not be altered by the function. 2. Call-by-constant-reference is appropriate for large objects that should not be altered by the function and are expensive to copy. 3. Call-by-reference is appropriate for all objects that may be altered by the function. - Call-by-rvalue-Reference = since an rvalue stores a temporary that is about to be destroyed, an expression such as x=rval (where rval is an rvalue) can be implemented by a move instead of a copy; often moving an object's state is much easier than copying it, as it may involve just a simple pointer change. - What we see here is that x=y can be a copy if y is an lvalue, but a move if y is an rvalue. - This gives a primary use case of overloading a function based on whether a parameter is an lvalue or rvalue, such as: string randomItem( const vector<string> & arr ); // returns random item in lvalue arr string randomItem( vector<string> && arr ); // returns random item in rvalue arr vector<string> v { "hello", "world" }; cout << randomItem( v ) << endl; // invokes lvalue method cout << randomItem( { "hello", "world" } ) << endl; // invokes rvalue method **1.5.4 RETURN PASSING** - Return-by-value = Return by value is the simplest and safest return type to use. When a value is returned by value, a copy of that value is returned to the caller. As with pass by value, you can return by value literals (eg. 5), variables (eg. x), or expressions (eg. x+1), which makes return by value very flexible. - Return-by-reference = values returned by reference must be variables (you can not return a reference to a literal or an expression) • When a variable is returned by reference, a reference to the variable is passed back to the caller. The caller can then use this reference to continue modifying the variable, which can be useful at times. Return by reference is also fast, which can be useful when returning structs and classes. **1.5.6 THE BIG FIVE: DESTRUCTOR, COPY CONSTRUCTOR, MOVE CONSTRUCTOR, COPY ASSIGNMENT OPERATOR=, MOVE ASSIGNMENT OPERATOR= - encBlock* lastBlock = new encBlock(myBlock); (COPY CONSTRUCTOR EXPLICITLY CALLED) - encBlock vidBlock = myBlock; (COPY CONSTRUCTOR IMPLICITLY CALLED) - An assignment from one existing object to another existing object will call the copy assignment operator - An assignment from one initialized object to another existing object will call the copy constructor 1) *Destructor* = called whenever an object goes out of scope or is subjected to a delete 2) *Copy Constructor and Move Constructor* = is a constructor which creates an object by initializing it with an object of the same class, which has been created previously - The copy constructor is used to: • Initialize one object from another of the same type. • Copy an object to pass it as an argument to a function. • Copy an object to return it from a function. 3) *Copy Assignment and Move Assignment Operator* = The assignment operator is called when = is applied to two objects that have both been previously constructed. lhs=rhs is intended to copy the state of rhs into lhs - If rhs is an lvalue, this is done by using the copy assignment operator - If rhs is an rvalue (i.e., a temporary that is about to be destroyed anyway), this is done by using the move assignment operator. *Defaults* - Shallow Copy = shallow copies of a pointer just copy the address of the pointer -- it does not allocate any memory or copy the contents being pointed to! • two class instances that contain pointers that point to the same object - Deep Copy = a clone of the entire object is made • when a class contains pointers as data members, and deep semantics are important, we typically must implement the destructor, copy assignment, and copy constructors ourselves *EXAMPLE OF THE BIG FIVE* ~IntCell( ); // Destructor IntCell( const IntCell & rhs ); // Copy constructor IntCell( IntCell && rhs ); // Move constructor IntCell & operator= ( const IntCell & rhs ); // Copy assignment IntCell & operator= ( IntCell && rhs ); // Move assignment **1.5.7 C-STYLE ARRAYS AND STRINGS** - basics
6.3 Binary Heap
*Binary Heap* (O(logN)) = Its use is so common for priority queue implementations that, in the context of priority queues, when the word heap is used without a qualifier, it is generally assumed to be referring to heaps - Two Properties (structure property and heap order property) **6.3.1 STRUCTURE PROPERTY** *Complete Binary Tree* = A heap is a binary tree that is completely filled, with the possible exception of the bottom level, which is filled from left to right - a complete binary tree of height h has between 2h and 2h+1 − 1 nodes. This implies that the height of a complete binary tree is logN , which is clearly O(logN) - For any element in array position i, the left child is in position 2i, the right child is in the cell after the left child (2i + 1), and the parent is in position i/2 . Thus, not only are links not required, but the operations required to traverse the tree are extremely simple and likely to be very fast on most computers. - A heap data structure will, then, consist of an array (of Comparable objects) and an integer representing the current heap size. **6.3.2 HEAP-ORDER PROPERTY** *Heap Order Property = The property that allows operations to be performed quickly - Since we want to be able to find the minimum quickly, it makes sense that the smallest element should be at the root - In a heap, for every node X, the key in the parent of X is smaller than (or equal to) the key in X, with the exception of the root (which has no parent) - By the heap-order property, the minimum element can always be found at the root. Thus, we get the extra operation, findMin, in constant time. **6.3.3 BASIC HEAP OPERATIONS** *Insert* - To insert an element X into the heap, we create a hole in the next available location, since otherwise, the tree will not be complete - If X can be placed in the hole without violating heap order, then we do so and are done. Otherwise, we slide the element that is in the hole's parent node into the hole, thus bubbling the hole up toward the root. We continue this process until X can be placed in the hole *Percolate Up/Bubble Up* *Delete Min* - deleteMins are handled in a similar manner as insertions. Finding the minimum is easy; the hard part is removing it. When the minimum is removed, a hole is created at the root. - Since the heap now becomes one smaller, it follows that the last element X in the hea must move somewhere in the heap. If X can be placed in the hole, then we are done. This is unlikely, so we slide the smaller of the hole's children into the hole, thus pushing the hole down one level. We repeat this step until X can be placed in the hole. Thus, our action is to place X in its correct spot along a path from the root containing minimum children. - make sure not to assume that there are always two children, so this usually involves an extra test *Percolate Down* **6.3.4 OTHER HEAP OPERATIONS** 1) decreaseKey = The decreaseKey(p, delta) operation lowers the value of the item at position p by a positive amount delta . Since this might violate the heap order, it must be fixed by a percolate up. This operation could be useful to system administrators: They can make their programs run with highest priority 2) increaseKey = The increaseKey(p, ) operation increases the value of the item at position p by a positive amount . This is done with a percolate down. Many schedulers automatically drop the priority of a process that is consuming excessive CPU time. 3) remove = The remove(p) operation removes the node at position p from the heap. This is done by first performing decreaseKey(p,∞) and then performing deleteMin(). When a process is terminated by a user (instead of finishing normally), it must be removed from the priority queue. 4) buildHeap = This constructor takes as input N items and places them into a heap. *For the perfect binary tree of height h containing 2^(h + 1) − 1 nodes, the sum of the heights of the nodes is 2^(h + 1) − 1 − (h + 1)*
3.6 The Stack ADT
*Stack* = is a list with the restriction that insertions and deletions can be performed in only one position, namely, the end of the list, called the top **3.6.1 THE STACK MODEL** - The fundamental operations on a stack are push, which is equivalent to an insert, and pop, which deletes the most recently inserted element - A pop or top on an empty stack is generally considered an error in the stack ADT - Stacks are sometimes known as *LIFO* (last in, first out) lists - essentially all that you can do to a stack is push and pop **3.6.2 IMPLEMENTATION OF STACKS** *Linked List Implementation of Stacks* - The first implementation of a stack uses a singly linked list. - We perform a push by inserting at the front of the list. - We perform a pop by deleting the element at the front of the list. - A top operation merely examines the element at the front of the list, returning its value. (Sometimes the pop and top operations are combined into one.) *Array Implementation of Stacks* - An alternative implementation avoids links and is probably the more popular solution. It uses the back, push_back, and pop_back implementation from vector, so the implementation is trivial. - To push some element x onto the stack, we increment topOfStack and then set theArray[topOfStack] = x. To pop, we set the return value to theArray[topOfStack] and then decrement topOfStack. **3.6.3 APPLICATIONS** *Three Applications of Stacks* 1) Balancing Symbols = a program that checks whether everything is balanced Algorithm Using Stacks to Balance Symbols: Make an empty stack. Read characters until end of file. If the character is an opening symbol, push it onto the stack. If it is a closing symbol and the stack is empty, report an error. Otherwise, pop the stack. If the symbol popped is not the corresponding opening symbol, then report an error. At end of file, if the stack is not empty, report an error. 2) Postfix/Reverse Polish Notation Expressions 6523 + 8 ∗ +3 + ∗ where 3 is at the top of the stack and the addition sign adds 3 + 2 which pushes 5 into the stack and then 8 is pushed into the stack and multiplies 8 * 5 to get 40 pushed into the stack, and adds 40 + 5 which pushes 45 into the stack and adds 3 + 45 which pops out 3 and 45 and pushes into the stack 48 and is multiplied by 6 which pushes in 288 and 48 and 6 is popped out 3) Infix to Postfix Conversion = Not only can a stack be used to evaluate a postfix expression, but we can also use a stack to convert an expression in standard form (otherwise known as infix) into postfix - When an operand is read, it is immediately placed onto the output. Operators are not immediately output, so they must be saved somewhere. The correct thing to do is to place operators that have been seen, but not placed on the output, onto the stack. We will also stack left parentheses when they are encountered. We start with an initially empty stack - symbols of higher precedence are only allowed to be stacked on top of the lower precedence symbols - next symbols of lower or equal precedence to that of the ones on the stack currently, pops the current symbols of higher or equal precedence into the output, and push the lower symbol into the stack - all alphabet characters are outputted automatically - open parentheses do not get removed except when a closed parentheses is being processed (therefore, symbols of lower precedence can be stacked on top of the open parentheses) - when the closed parentheses appear, the parentheses disappear and whatever symbols were in between is outputted ex: Suppose we want to convert the infix expression a + b * c + (d * e + f) * g into postfix. - A correct answer is a b c * + d e * f + g * +.
CHAPTER 1 PROGRAMMING: A GENERAL OVERVIEW
1.1 What's This Book About? 1.2 Mathematics Review 1.3 A Brief Introduction to Recursion 1.4 C++ Classes
CHAPTER 2: ALGORITHM ANALYSIS
An algorithm is a clearly specified set of simple instructions to be followed to solve a problem. Once an algorithm is given for a problem and decided (somehow) to be correct, an important step is to determine how much in the way of resources, such as time or space, the algorithm will require • How to estimate the time required for a program. • How to reduce the running time of a program from days or years to fractions of a second. • The results of careless use of recursion. • Very efficient algorithms to raise a number to a power and to compute the greatest common divisor of two numbers
2.3 What to Analyze
The most important resource to analyze is generally the running time
6.2 Simple Implementations
There are several obvious ways to implement a priority queue: 1) Simple Linked List: performing insertions at the front in O(1) and traversing the list, which requires O(N) time, to delete the minimum 2) Binary Search Tree: this gives an O(logN) average running time for both operations
CHAPTER 3: LISTS, STACKS AND QUEUES
Virtually every significant program will use at least one of these structures explicitly, and a stack is always implicitly used in a program, whether or not you declare one 1) Introduce the concept of Abstract Data Types (ADTs). 2) Show how to efficiently perform operations on lists. 3) Introduce the stack ADT and its use in implementing recursion. 4) Introduce the queue ADT and its use in operating systems and algorithm design.
3.4 Implementation of Vector
basics
1.6 Templates
look inside book
1.3 A Brief Introduction to Recursion
**4 STEPS IN RECURSION** *Recursive* = A function that is defined in terms of itself *1) Base Case* = the value for which the function is directly known without resorting to recursion - You must always have some base cases, which can be solved without recursion. *2) Making Progress* = For the cases that are to be solved recursively, the recursive call must always be to a case that makes progress toward a base case. *3) Design Rule* (Assume that all the recursive calls work) - when designing recursive programs, you don't have to try to trace through the myriad of recursive calls - the computer is being allowed to work out the complicated details. *4) Compound Interest Rule* = Never duplicate work by solving the same instance of a problem in separate recursive calls. - this rule is the reason that it is generally a bad idea to use recursion to evaluate simple mathematical functions, such as the Fibonacci numbers ex: int f( int x ) { if( x == 0 ) { return 0; } else { return 2 * f( x - 1 ) + (x * x); } *Using recursion for numerical calculations is usually a bad idea. We have done so to illustrate the basic points. Although we are defining a function in terms of itself, we are not defining a particular instance of the function in terms of itself. In other words, evaluating f(5) by computing f(5) would be circular. Evaluating f(5) by computing f(4) is not circular—unless, of course, f(4) is evaluated by eventually computing f(5). If f is called with the value of 4, then line 6 requires the computation of 2 ∗ f(3) + 4 ∗ 4. Thus, a call is made to compute f(3). This requires the computation of 2∗f(2)+3∗3. Therefore, another call is made to compute f(2). This means that 2 ∗ f(1) + 2 ∗ 2 must be evaluated. To do so, f(1) is computed as 2∗f(0)+1∗1. Now, f(0) must be evaluated. Since this is a base case, we know a priori that f(0) = 0. This enables the completion of the calculation for f(1), which is now seen to be 1. Then f(2), f(3), and finally f(4) can be determined. Recursive calls will keep on being made until a base case is reached *FINDING DEFINITION OF WORDS RECURSIVELY* Our recursive strategy to understand words is as follows: If we know the meaning of a word, then we are done; otherwise, we look the word up in the dictionary. If we understand all the words in the definition, we are done; otherwise, we figure out what the definition means by recursively looking up the words we don't know. This procedure will terminate if the dictionary is well defined but can loop indefinitely if a word is either not defined or circularly defined. *RECURSIVE ROUTINE TO PRINT AN INTEGER* void printOut( int n ) {// Print nonnegative n if( n >= 10 ) { printOut( n / 10 ); } printDigit( n % 10 ); //if n is less than 10, printDigit will be called; n = 3 -> 3 % 10 = 3 } RECURSION AND INDUCTION = proving (somewhat) rigorously that the recursive number-printing program works. To do so, we'll use a proof by induction.
3.1 Abstract Data Types (ADT)
*Abstract Data Type* = is a set of objects together with a set of operations - Objects such as lists, sets, and graphs, along with their operations, just as integers, reals, and booleans are data types - we might have such operations as add, remove, size, and contains - the programs that use them will not necessarily need to know which implementation was used
7.1 Preliminaries
*Comparison Based Sorting* = the existence of the "<" and ">" operators, which can be used to place a consistent ordering on the input. Besides the assignment operator, these are the only operations allowed on the input data. - This interface is not the same as in the STL sorting algorithms - The iterators must support random access
7.8 A General Lower Bound of Sorting
*Decision Tree* = an abstraction used to prove lower bounds. In our context, a decision tree is a binary tree. Each node represents a set of possible orderings, consistent with comparisons that have been made, among the elements. The results of the comparisons are the tree edges.
3.2 The List ADT
*Empty List* = special list of size 0 Some popular operations are printList and makeEmpty, which do the obvious things; 1) find, which returns the position of the first occurrence of an item; ex: If the list is 34, 12, 52, 16, 12, then find(52) might return 2; 2) insert and remove, which generally insert and remove some element from some position in the list ex: insert(x,2) might make the list into 34, 12, x, 52, 16, 12 (if we insert into the position given); and remove(52) might turn that list into 34, 12, x, 16, 12. 3) findKth, which returns the element in some position (specified as an argument). **3.2.1 SIMPLE ARRAY IMPLEMENTATION OF LISTS** - An array implementation allows printList to be carried out in linear time, and the findKth operation takes constant time, which is as good as can be expected. However, insertion and deletion are potentially expensive, depending on where the insertions and deletions occur. In the worst case, inserting into position 0 (in other words, at the front of the list) requires pushing the entire array down one spot to make room, and deleting the first element requires shifting all the elements in the list up one spot, so the worst case for these operations is O(N) - if insertions and deletions occur throughout the list and, in particular, at the front of the list, then the array is not a good option. The next section deals with the alternative: the linked list. **3.2.2 SIMPLE LINKED LISTS** - In order to avoid the linear cost of insertion and deletion, we need to ensure that the list is NOT stored ADJACENTLY, since otherwise entire parts of the list will need to be moved *Linked List* = consists of a series of nodes, which are not necessarily adjacent in memory - each node contains the element and a link to a node containing its successor. We call this the next link - the last cell's next link points to nullptr. - involves only a constant number of changes to node links when you want to insert or remove elements *Doubly-Linked List* = we have every node maintain a link to its previous node in the list and to the next node in the list
7.5 Heapsort O(N log N)
*Heapsort* = - The main problem with this algorithm is that it uses an extra array. Thus, the memory requirement is doubled. The problem is space. How To Avoid Using a Second Array - after each deleteMin, the heap shrinks by 1 - Thus the cell that was last in the heap can be used to store the element that was just deleted. As an example, suppose we have a heap with six elements. The first deleteMin produces a1. Now the heap has only five elements, so we can place a1 in position 6. The next deleteMin produces a2. Since the heap will now only have four elements, we can place a2 in position 5. => array will have elements in decreasing sorted order *unlike the binary heap, where the data begin at array index 1, the array for heapsort contains data in position 0. Thus the code is a little different from the binary heap code. The changes are minor.* LOOK AT NOTES ON HEAPSORT IN OTHER TAB
7.2 Insertion Sort O(N^2)
*Insertion Sort* = one of the simplest sorting algorithms **7.2.1 THE ALGORITHM** - Insertion sort consists of N - 1 passes - Insertion sort makes use of the fact that elements in positions 0 through p−1 are already known to be in sorted order. 1) Start with unsorted vector of N elements, say a[0], ..., a[N-1] For i-th iteration (i=1, ..., N-1) 2) Loop Invariant: a[0], ..., a[i-1] are already sorted, but a[i], ..., a[N-1] are unknown 3) Create a ``hole'' by removing a[i] from array by storing it in temp temp = a[i]; 4) Inner Loop to shift the ``hole'' towards the left into the correct position j=i; while (a[j-1]>temp) j--; 5) Finish the iteration by filling the ``hole'' using a[j]=temp; NOTE: Inner loop is just a linear search for correct ``hole'' position Still works if initial ``hole'' position was on the left (instead of the right) => TIGHT Just shift ``hole'' in the opposite direction, Inner loop is still O(N) **7.2.2 STL IMPLEMENTATION OF INSERTION SORT** - the sort routines receive a pair of iterators that represent the start and endmarker of a range
7.6 Mergesort O(N log N)
*Mergesort* = The fundamental operation in this algorithm is merging two sorted lists. Because the lists are sorted, this can be done in one pass through the input, if the output is put in a third list. - The basic merging algorithm takes two input arrays A and B, an output array C, and three counters, Actr, Bctr, and Cctr, which are initially set to the beginning of their respective arrays. - The smaller of A[Actr] and B[Bctr] is copied to the next entry in C, and the appropriate counters are advanced. - When either input list is exhausted, the remainder of the other list is copied to C - recursively mergesort the first half and the second half. This gives two sorted halves, which can then be merged together using the merging algorithm described above. - For instance, to sort the eight-element array: 24, 13, 26, 1, 2, 27, 38, 15, we recursively sort the first four and last four elements, obtaining: 1, 13, 24, 26, 2, 15, 27, 38. - Then we merge the two halves as above, obtaining the final list: 1, 2, 13, 15, 24, 26, 27, 38. This algorithm is a classic divide-and-conquer strategy. The problem is *divided* into smaller problems and solved recursively. The *conquering* phase consists of patching together the answers *Telescoping a Sum* = all the terms appear on both sides and cancel
CHAPTER 6: PRIORITY QUEUES (HEAPS)
*Priority Queue* = an abstract data type which is like a regular queue or stack data structure, but where additionally each element has a "priority" associated with it. In a priority queue, an element with high priority is served before an element with low priority - priority queues can be used to sort in O(N logN) time
3.7 The Queue ADT
*Queues* = like stacks, queues are lists and insertion is done at one end whereas deletion is performed at the other end **3.7.1 QUEUE MODEL** - Enqueue = inserts an element at the end of the list (called the rear) - Dequeue = deletes (and returns) the element at the start of the list (known as the front) **3.7.2 ARRAY IMPLEMENTATION OF QUEUES** - As with stacks, any list implementation is legal for queues - Like stacks, both the linked list and array implementations give fast O(1) running times for every operation - To enqueue an element x, we increment currentSize and back, then set theArray[back] = x. - To dequeue an element, we set the return value to theArray[front], decrement currentSize, and then increment front. *Circular Array Implementation* = whenever front or back gets to the end of the array, it is wrapped around to the beginning **3.7.3 APPLICATIONS OF QUEUES** - Calls to large companies are generally placed on a queue when all operators are busy. - In large universities, where resources are limited, students must sign a waiting list if all computers are occupied. The student who has been at a computer the longest is forced off first, and the student who has been waiting the longest is the next user to be allowed on. *The Queueing Theory* = deals with computing, probabilistically, how long users expect to wait on a line, how long the line gets, and other such questions
7.7 Quicksort O(N^2) WC and O(N log N) AC
*Quicksort* 1) Arbitrarily choose any item, and then form three groups: those smaller than the chosen item, those equal to the chosen item, and those larger than the chosen item. 2) Recursively sort the first and third groups, and then link the three groups. - Like mergesort, quicksort is a divide-andconquer recursive algorithm *Classic Quicksort* (has an array of S) 1) If the number of elements in S is 0 or 1, then return. 2) Pick any element v in S. This is called the pivot. 3) Partition S − {v} (the remaining elements in S) into two disjoint groups: S1 = {x ∈ S − {v}|x ≤ v}, and S2 = {x ∈ S − {v}|x ≥ v}. 4) Return {quicksort(S1) followed by v followed by quicksort(S2)} *Like mergesort, it recursively solves two subproblems and requires linear additional work (step 3), but, unlike mergesort, the subproblems are not guaranteed to be of equal size, which is potentially bad. *The reason that quicksort is faster is that the partitioning step can actually be performed in place and very efficiently. This efficiency more than makes up for the lack of equal-sized recursive calls. **7.7.1 PICKING THE PIVOT** - Although the algorithm as described works no matter which element is chosen as pivot, some choices are obviously better than others *Horrible Ideas for How to Choose Pivots:* 1) DO NOT CHOOSE FIRST ELEMENT TO BE PIVOT SINCE IT WILL DIVIDE THE ELEMENTS WHERE ONE PART HAS MANY MORE ELEMENTS THAN THE OTHER 2) DO NOT CHOOSE THE LARGER OF THE FIRST TWO DISTINCT ELEMENTS AS PIVOTS *A Safe Way for How to Choose Pivots:* 1) A safe course is merely to choose the pivot randomly *Median-of-Three Partitioning* - (BAD) The median of a group of N numbers is the N/2th largest number. The best choice of pivot would be the median of the array. Unfortunately, this is hard to calculate and would slow down quicksort considerably. - (GOOD) the common course is to use as pivot the median of the left, right, and center elements (For instance, with input 8, 1, 4, 9, 6, 3, 5, 2, 7, 0 as before, the left element is 8, the right element is 0, and the center (in position (left + right)/2 ) element is 6. Thus, the pivot would be v = 6.) **7.7.2 PARTITIONING STRATEGY** 1) The first step is to get the pivot element out of the way by swapping it with the last element. i starts at the first element and j starts at the next-to-last element 2) What our partitioning stage wants to do is to move all the small elements to the left part of the array and all the large elements to the right part. "Small" and "large" are, of course, relative to the pivot. 3) While i is to the left of j, we move i right, skipping over elements that are smaller than the pivot. We move j left, skipping over elements that are larger than the pivot. When i and j have stopped, i is pointing at a large element and j is pointing at a small element. 4) If i is to the left of j, those elements are swapped. 5) The effect is to push a large element to the right and a small element to the left.
3.3 Vector and List in the STL
*Standard Template Library (STL)* - the List ADT is one of the data structures implemented in the STL (these data structures are called collections or containers) - The list provides a doubly linked list implementation of the List ADT. - the advantage of using the list is that insertion of new items and removal of existing items is cheap, provided that the position of the changes is known - the disadvantage is that the list is not easily indexable. Both vector and list are inefficient for searches - list refers to the doubly linked list in the STL **3.3.1 ITERATORS** - In the STL, a position is represented by a nested type, iterator. In particular, for a list<string>, the position is represented by the type list<string>::iterator; for a vector<int>, the position is represented by a class vector<int>::iterator, and so on Three Main Issues to Address: 1) Getting Iterators (how one gets an iterator) - iterator begin( ): returns an appropriate iterator representing the first item in the container - iterator end( ): returns an appropriate iterator representing the endmarker in the container (i.e., the position after the last item in the container) for( vector<int>::iterator itr = v.begin( ); itr != v.end( ); ++itr ) { cout << *itr << endl; } 2) Iterator Methods (what operations the iterators themselves can perform) - *itr: returns a reference to the object stored at iterator itr's location. The reference returned may or may not be modifiable (we discuss these details shortly). - itr1==itr2: returns true if iterators itr1 and itr2 refer to the same location and false otherwise. - itr1!=itr2: returns true if iterators itr1 and itr2 refer to a different location and false otherwise. 3) Container Operations that Require Iterators (which List ADT methods require iterators as parameters) - iterator insert( iterator pos, const Object & x ): adds x into the list, prior to the position given by the iterator pos. This is a constant-time operation for list, but not for vector. The return value is an iterator representing the position of the inserted item. - iterator erase( iterator pos ): removes the object at the position given by the iterator. This is a constant-time operation for list, but not for vector. The return value is the position of the element that followed pos prior to the call. This operation invalidates pos, which is now stale, since the container item it was viewing has been removed. - iterator erase( iterator start, iterator end ): removes all items beginning at position start, up to, but not including end. Observe that the entire list can be erased by the call c.erase( c.begin( ), c.end( ) ). **3.3.2 EXAMPLE USING ERASE ON A LIST** 1 template <typename Container> 2 void removeEveryOtherItem( Container & lst ) 3 { 4 auto itr = lst.begin( ); // itr is a Container::iterator 5 6 while( itr != lst.end( ) ) 7 { 8 itr = lst.erase( itr ); 9 if( itr != lst.end( ) ) 10 ++itr; 11 } 12 } *Using iterators to remove every other item in a List (either a vector or list). Efficient for a list, but not for a vector. *3.3.3 CONST_ITERATORS* - every collection contains not only an iterator nested type but also a const_iterator nested type - The main difference between an iterator and a const_iterator is that operator* for const_iterator returns a constant reference, and thus *itr for a const_iterator cannot appear on the left-hand side of an assignment statement ex: iterator begin( ) const_iterator begin( ) const iterator end( ) const_iterator end( ) const
2.1 Mathematical Background
- Relative Rates of Growth *Big-O Notation* = allows the possibility that the growth rates are the same - it is very bad style to include constants or low-order terms inside a Big-Oh - Do not say T(N) = O(2N^2) or T(N) = O(N^2+N). In both cases, the correct form is T(N) = O(N^2). *Typical Growth Rates* c => Constant logN => Logarithmic log^2 N => Log-squared N => Linear N logN N^2 => Quadratic N^3 => Cubic 2^N => Exponential
CHAPTER 7: SORTING
- we discuss the problem of sorting an array of elements - For most of this chapter, we will also assume that the entire sort can be done in main memory, so that the number of elements is relatively small (less than a few million) - There are several easy algorithms to sort in O(N2), such as insertion sort. - There are slightly more complicated O(N logN) sorting algorithms. - Any general-purpose sorting algorithm requires (N logN) comparisons.
Selection Sort
1 Start with unsorted vector of N elements, say a[1], ..., a[N-1] For i-th iteration (i=0, ..., N-2) 2) Loop Invariant: a[0], ..., a[i-1] already contain the smallest value, second-smallest value, etc, from entire array a[i], ..., a[N-1] contain values that are at least as large as the above values, in no particular order 3) Inner Loop to find next-smallest value: nextSmallest = i for j=i, ..., N-1 if (a[j] < a[nextSmallest]) nextSmallest=j; 4) Finish the iteration by placing the next-smallest value in the i-th position of the array: swap (a[i], a[nextSmallest]) NOTE: Outer loop is clearly O(N) because we search repeatedly for next-smallest value Inner loop is also O(N) because we must look at evey value not-yet placed in its final position
3.5 Implementation List
1. The List class itself, which contains links to both ends, the size of the list, and a host of methods. 2. The Node class, which is likely to be a private nested class. A node contains the data and pointers to the previous and next nodes, along with appropriate constructors. 3. The const_iterator class, which abstracts the notion of a position, and is a public nested class. The const_iterator stores a pointer to "current" node, and provides implementation of the basic iterator operations, all in the form of overloaded operators such as =, ==, !=, and ++. 4. The iterator class, which abstracts the notion of a position, and is a public nested class. The iterator has the same functionality as const_iterator, except that operator* returns a reference to the item being viewed, rather than a constant reference to the item - Header Node = the node at the front of the list - Tail Node = the node at the back of the list - Friend Declaration = grants the List class access to const_iterator's nonpublic members
1.2 Mathematics Review
1.2.1 Exponents (X^A)*(X^B) = X^(A+B) (X^A)/(X^B) = X^(A−B) (X^A)^B = X^(AB) (X^N) + (X^N) = (2X)^N ≠ X^(2N) 2^N + 2^N = 2^(N+1) 1.2.2 Logarithms In computer science, all logarithms are to the base 2 unless specified otherwise. 1.2.3 Series 1) Arithmetic Series 2) Euler's Constant = approximates errors 1.2.4 Modular Arithmetic We say that A is congruent to B modulo N, written A ≡ B (mod N), if N divides A − B. Intuitively, this means that the remainder is the same when either A or B is divided by N. Thus, 81 ≡ 61 ≡ 1 (mod 10). As with equality, if A ≡ B (mod N), then A + C ≡ B + C (mod N) and AD ≡ BD (mod N). 1.2.5 The P Word (The best way of proving that a theorem is false is by exhibiting a counterexample) 1) Proof By Induction - Base Case = establishing that a theorem is true for some small (usually degenerate) value(s); this step is almost always unimportant - Inductive Hypothesis = the theorem is assumed to be true for all cases up to some limit k (Using this assumption, the theorem is then shown to be true for the next value, which is typically k + 1. This proves the theorem (as long as k is finite).) 2) Proof By Contradiction **REFER BACK TO TEXTBOOK**
6.1 Model
A *priority queue* is a data structure that allows at least the following two operations: 1) insert = which inserts an element in the priority queue - The insert operation is the equivalent of enqueue 2) deleteMin = finds, returns, and removes the minimum element in the priority queue. - deleteMin operation is the priority queue equivalent of the queue's dequeue operation. *Greedy Algorithms* = which operate by repeatedly finding a minimum
6.4 Applications of Priority Queues
Here we will show how to use priority queues to obtain solutions to two problems **6.4.1 THE SELECTION PROBLEM** - Algorithm 6A = we are interested in finding the kth smallest element. The algorithm is simple. We read the N elements into an array. We then apply the buildHeap algorithm to this array. Finally, we perform k deleteMin operations. The last element extracted from the heap is our answer - Algorithm 6B = we return to the original problem and find the kth largest element. We use the idea from algorithm 1B. At any point in time we will maintain a set S of the k largest elements. After the first k elements are read, when a new element is read it is compared with the kth largest element, which we denote by Sk. Notice that Sk is the smallest element in S. If the new element is larger, then it replaces Sk in S. S will then have a new smallest element, which may or may not be the newly added element. At the end of the input, we find the smallest element in S and return it as the answer.
2.2 Model
In order to analyze algorithms in a formal framework, we need a model of computation. Our model is basically a normal computer in which instructions are executed sequentially.
1.1 What's This Book About?
Selection Problem = you have a group of N numbers and would like to determine the kth largest - One way to solve this problem would be to read the N numbers into an array, sort the array in decreasing order by some simple algorithm such as bubble sort, and then return the element in position k - A somewhat better algorithm might be to read the first k elements into an array and sort them (in decreasing order). Next, each remaining element is read one by one. As a new element arrives, it is ignored if it is smaller than the kth element in the array. Otherwise, it is placed in its correct spot in the array, bumping one element out of the array. When the algorithm ends, the element in the kth position is returned as the answer *The natural questions, then, are: Which algorithm is better? And, more important, Is either algorithm good enough? *A simulation using a random file of 30 million elements and k = 15,000,000 will show that neither algorithm finishes in a reasonable amount of time; each requires several days of computer processing to terminate (albeit eventually with a correct answer). An alternative method, discussed in Chapter 7, gives a solution in about a second. *THEREFORE, ALL ALGORITHMS ABOVE CANNOT BE CONSIDERED GOOD ALGORITHMS *It is relatively easy to code up either method of solution and solve many of the real-life puzzles commonly published in magazines. An important concept is that, in many problems, writing a working program is not good enough. If the program is to be run on a large data set, then the RUNNING TIME becomes an issue. Throughout this book we will see how to estimate the running time of a program for large inputs and, more important, how to compare the running times of two programs without actually coding them. We will see techniques for drastically improving the speed of a program and for determining program bottlenecks. These techniques will enable us to find the section of the code on which to concentrate our optimization efforts.
2.4 Running-Time Calculations
we throw away leading constants. We will also throw away low-order terms, so what we are essentially doing is computing a Big-Oh running time. N = 100 O(N^3) = 0.000159 O(N^2) = 0.000006 O(N log N) = 0.000005 O(N) = 0.000002 Since Big-Oh is an upper bound, we must be careful never to underestimate the running time of the program **2.4.2 GENERAL RULES** Rule 1 (FOR Loops) = The running time of a for loop is at most the running time of the statements inside the for loop (including tests) times the number of iterations. Rule 2 (Nested Loops) = The total running time of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the sizes of all the loops Example of O(N^2) because we are multiplying the two sizes n, together for( i = 0; i < n; ++i ) { for( j = 0; j < n; ++j ) { ++k; } } Rule 3 (Consecutive Statements) = the maximum is the one that counts Example of O(N^2) because even though there are 3 for loops, the first for loop is O(N) and the nested loop afterwards is O(N^2); therefore, O(N) + O(N^2) still equals O(N^2): for( i = 0; i < n; ++i ) a[ i ] = 0; for( i = 0; i < n; ++i ) for( j = 0; j < n; ++j ) a[ i ] += a[ j ] + i + j; Rule 4 (If/Else) if( condition ) { S1 } else { S2 } the running time of an if/else statement is never more than the running time of the test plus the larger of the running times of S1 and S2. Online Algorithms = if the array is on a disk or is being transmitted over the Internet, it can be read sequentially, and there is no need to store any part of it in main memory - requiring only constant space and runs in linear time is just about as good as possible. **2.4.4 LOGARITHMS IN THE RUNNING TIME** - An algorithm is *O(logN)* if it takes constant (O(1)) time to cut the problem size by a fraction (which is usually 1/2) • when we talk about O(logN) algorithms for these kinds of problems, we usually presume that the input is pre-read - if constant time is required to merely reduce the problem by a constant amount (such as to make the problem smaller by 1), then the algorithm is *O(N)* *Binary Search* (O(logN)) = binary search, also known as half-interval search or logarithmic search, is a search algorithm that finds the position of a target value within a sorted array. It compares the target value to the middle element of the array; if they are unequal, the half in which the target cannot lie is eliminated and the search continues on the remaining half until it is successful *Euclid's Algorithm* (O(logN)) = computing the greatest common divisor - The greatest common divisor (gcd) of two integers is the largest integer that divides both. Thus, gcd(50, 15) = 5. The algorithm in Figure 2.10 computes gcd(M,N), assuming M ≥ N. (If N > M, the first iteration of the loop swaps them.) The algorithm works by continually computing remainders until 0 is reached. The last nonzero remainder is the answer.