Dijkstra's Algorithm
The logical foundation for proof by contradiction is anapplication of modus tollens:
( A → B ∧ ¬B ) → ¬A A: algorithm does not produce a "best" result B: algorithm makes a first non-"best" decision We assume ( A ∧ ¬B ) and show contradiction
Binary trees are inherently recursive
-base case when you reach and empty node -scan left/right subtree
Let G be a graph. Then the following statements are equivalent:
1) G is a tree 2) G is connected and acyclic 3) Between any two vertices of G there is precisouly one path
three tree traversal operations:
1. N - visit node and do some action 2. R - recursive descent to right subtree 3. L - recursive descent to left subtree
post order scan lrn
1. Traverse the left subtree ("go left") 2. Traverse the right subtree ("go right") 3. Visit the node
in order lnr
1. Traverse the left subtree ("go left") 2. Visit the node 3. Traverse the right subtree ("go right") (only when left is empty to you visit a node, then try visiting right of that base node)
Tree
A connected graph that contains no circuits.
Bipartite Corollary
A graph G is bipartite iff it contains no odd length cycle
Weighted Graph
A graph G(V, E) together with a function w: [0, infinity). If e is an edge the nonnegative real number w(e) is called the weight of e. The weight of a subgraph of G (often a path or a trail) is the sum of the weights of the edges of the subgraph.
What is a graph?
A graph is an abstract data type used to model a set of connections. Each node is a node and each connection between nodes is an edge.
strongly connected (directed graph)
A graph is strongly connected if every pair of nodes is mutually reachable
Spanning Tree
A spanning tree of a connected graph G is a subgraph that is a tree and that includes every vertex of G. A minimum spanning tree of a weigthed graph is a spanning tree of least weight, that is, a spanning tree for which the sum of the weights of all its edges os least among all the spanning trees
Minimum spanning tree
A spanning tree such that the total length of its arcs is as small as possible
Leaf
A vertex of degree 1.
Checking Optimality for the shortest path
Add a toll to each node equal to the shortest path to that node; c'(i, j) = c(i, j) + p(i) - p(j)
What are vertices connected by an edge in a graph called?
Adjacent vertices
Dijkstra's algorithm
An algorithm for finding the shortest path between two vertices in a weighted graph
What's the last structure you need to Dijsktra's algorithm?
An array to track the nodes you already checked completely! (i.e. checked every edge connected to it).
What data structure do you need to build to track the costs of going to each node?
An object to keep track of the cost of each node const costs = { A: 5, B: 2, finish: Infinity };
Counting leaves best traversal?
Any scan - just increment a counter for each node
How to retrieve a path?
At each visited node store a node back to predecessor; once you reach the goal, you can traverse back using pointers; source predecessor is NIL
BFS adjacency list theorem
BFS runs in O(m + n) time if the graph is given by its adjacency list representation
For weighted graphs
BFS still finds shortest path first (DFS does not). Can expand order of min distance ( keep track of length of each path so far)
BFS
Breadth First Search Goal: find everything findable from source vertex v Alg: 1) mark all vertices as unexplored 2) mark s as explored, put to queue Q 3) while Q non-empty - pop vertex v from Q - for all (v, w) if w is unexplored mark as explored, add it to Q Applications: a) finding shortest paths in undirected graph use same algorithm, just save path(w) = path(v) + 1 when marking w as explored b) finding connected components of undirected graph connected components - equivalence classes of relation: u relates to v <=> exists path from u to v equivalence relation: reflexive, symmetric, transitive Alg: for all v if v unexplored - run BFS from v.
strong connectivity of directed graph theorem
Can determine if G is strongly connected in O(m + n) time
Flloyd-Warshall Algorithm
Can either solve All-Pairs shortest path or reachability problems.
Prim's algorithm
Choose any vertex to start tree and select arc of least weight that joins vertex in tree to one not in tree; repeat until all vertices are connected
Tree Definition
Connected Acyclic graph
What's the difference between BFS and DFS alogrithms in a graph?
DFS is stored in a stack, BFS in a queue. In DFS, vertices are explored by going down a path and visiting a new adjacent vertex if there's one available. In BFS, the oldest unexplored vertices are explored first.
DFS algorithm
DFS(u): Mark u as explored and add u to R For each edge(u,v) incident to u if v is not marked explored then recursively invoke DFS(v) Endif Endfor
What is the algorithm for the unweighted shortest path problem?
Determine distances from a vertex by starting with the vertex and finding all adjacent vertices (these have length 1). Move to each of the adjacent vertices and find all of their adjacent vertices not already visited. These have length 2. Continue until all vertices are visited. (BREADTH-FIRST SEARCH)
undirected graph with +edge weights
Dijkstra O( |V|^2) Bellman-Ford O( |V|^3) <- sloppy O(|V| . |E|)
How many edges?
E = V-1 (since there is one edge connected to each node)
Minimum Spanning Tree Solution (Unweighted)
Execute depth first search, save the edges.
Edmonds-Karp algorithm
Find shortest unweighted path at each iteration (or the path with the fewest number of edges). Do this with BFS.
Prim's algorithm
Finds a minimum spanning tree by maintaining a "current spanning tree." Start by setting current spanning tree to an arbitrary node then iterate outwards, adding the minimum weight edge from a node in the current spanning tree to a node not in the tree until all nodes are accounted for. Use a multimap keyed on the weights of the edges. May need to delete edges as you might improve a node's distance to the current spanning tree.
What is a single-source shortest path problem?
Fine the shortest weighted path from a vertex, s, to every other vertex in the graph.
Finding a Path Using BFS
First path found is the shortest -may have memory issues
Step 1
Give a start vertex (s) a permanent label of 0
Step 2
Give each vertex connected to s a working value by recording it's distance from s
Directed reachability
Given a node s, find all nodes reachable from s.
encoding
Given some alphabet of characters, we need to assign to each character a "codeword" (fixed sequence of bits)
what is the s-t connectivity problem?
Given two nodes s and t, is there a path between s and t?
s-t shortest path problem
Given two nodes s and t, what is the length of the shortest path between s and t?
D Step 5
Go backwards through the network, retracing the path of shortest length from the destination node to the start node.
Correctness of Algorithm
How do we know that, at termination (when the min-priority queue is empty because every vertex has been dequeued), v.dist is in fact the shortest-path weight for every vertex v? We rely on a loop invariant argument. We state a property that pertains to a loop, which we call the loop invariant, and we have to show three things about it: 1. The loop invariant is true before we enter the loop. 2. If the loop invariant is true upon starting an iteration of the loop, it remains true upon starting the next iteration. 3. Theloopinvariant,combinedwiththereasonthatweexittheloop,yieldsthepropertythatwe want to show.
What are the degrees of a vertex?
How many edges it has.
negative edge
If a graph has a negative edge weight (but not a negative cycle), there is a (finite) shortest path between each pair of nodes (that is, provided there's a path between those two nodes to begin with), but the presence of a negative edge means that Dijkstra's algorithm isn't guaranteed to work; it won't necessarily find that shortest path.
6 simple recursive formulas for tree traversal
In-order: RNL, LNR Post-order: RLN, LRN Pre-order: NRL, NLR
How is the shortest path table set up?
Known is set to true after a vertex is processed, dv is distance from v3, start all vertices at infinite distance except first one (0), pv is the previous vertex used to reach this vertex
D Step 1
Label the start node with zero and box this label.
BFS tree property
Let (x,y) be an edge of G. then the level of x and y differ by at most 1.
tree theorem
Let G be an undirected graph on n nodes. Any two of the following statements imply the third. -G is connected -G does not contain a cycle -G has n-1 edges
Dijkstra's algorithm
Modification of breadth first search. Used to find the shortest path from a given node to all other nodes, given the edges may have non-negative weights. Uses a multimap instead of the queue. The multimap uses the distance from the starting node to the node as a key, and the node itself as a val.
mutually reachable (directed graph)
Node u and v are mutually reachable if there is a path from u to v and also a path from v to u
Dijkstra runtime
O (|E| + |V| log |V|) usually expressed O(|V|^2) Certainly, |E| < |V|^2 and |V| log |V| < |V|^2, so an algorithm that is O(|E| + |V| log |V|) must also be O(|V^|2).)
Kruskal - Priority Queue Sorted Array (Remove Minimum)
O(1)
time to check if (u,v) has an edge in an Adjacency matrix
O(1) time
Prim's runtime:
O(E + V log V)
Kruskal's runtime:
O(E log V)
Bellman Ford complexity
O(V*E) ...(V.E)
Dijkastra's Complexity
O(V*Vlog(v)
time to check if (u,v) is an edge in an Adjacency list
O(deg(u))
Kruskal - Priority Queue Heap (Remove Minimum)
O(lg(n))
time to identify all edges within an adjacency list
O(m + n) time
Kruskal - Priority Queue Heap (Build)
O(m)
Finding a Path using DFS
Path found might not be shortest, therefore you need an exhaustive search - also leads to memory issues
Shortest Path Output
Path from s to t
which tree traversal is best for deleting a node?
Post Order: LRN ensures that the parent is deleted last to avoid memory leaks. template <typename T> void DeleteTree(tnode<T> *t) { if(t != NULL) { DeleteTree(t->left); DeleteTree(t->right); delete t; } }
find all nodes reachable from s, a general algorithm
R will consist of nodes to which s has a path Initially R = {s} While there is an edge (u,v) where u is in set R and v is not in set R add v to R Endwhile
D Step 4
Repeat Steps 2 and 3 until the destination node has a permanent label.
Dijkstra's Algorithm
Solution to the single-source shortest path problem in graph theory ¤ Both directed and undirected graphs ¤ All edges must have nonnegative weights ¤ Graph must be connected
Main difference A* Vs Dijkstra
The key in the PQ for A* is always *distance so far plus the estimate of the distance remaining *
main idea in apply greedy method
The main idea in applying the greedy-method pattern to the single-source shortestpath problem is to perform a "weighted" breadth-first search starting at the sourcevertex s.
What is a weighted path length?
The sum of the weights on the path.
Dijkstra's Algorithm
To solve shortest path from one vertex to another. Start with a point, record the distance to all other points. Pick the shortest distance, travel to that point. From that point, explore the distance to other vertices, entering the total weight from the original to those vertices if that weight is smaller. Travel to the next smallest from the original. Repeat until all points have been visited.
Isomorphic
Trees T1 and T2 whose vertices are labeled with the same set of labels are isomorphic if and only if, for each pair of labels v and w, vertices v and w are adjacent in T1, if and only if they are adjaent in T2
Run time depends on
Type of data structure we use for priority queue: - unsorted array - min-heap -Fibonacci heap
Step 4
Update working values at any unlabelled vertices that can be reached from v
How do we write code for the unweighted shortest path problem?
Use a table to keep track of each vertex as it is visited.
Depth first search
Uses a stack, pushes nodes onto the stack.
What does it mean for a graph to be *dense*?
We say that a graph is dense if m = Θ(n^2), that is if on average every vertex n has Θ(n) neighbors.
What does it mean that graphs can be weighed?
Weight on edges can represent the cost from getting from point to point. For example, if node a's edge with node b is 2 and node b's edge with node c is 5, the trip from node b to node c is more expensive than the trip from node a to node b.
Min-heap for min-PQ
What if we use a min-heap for the min-priority queue? Now each insert, extractMin, and decreaseKey operation takes O(lg n) time, and so the total time for Dijkstra's algorithm is O((n + m) lg n). If we make the reasonable assumption that m ≥ n − 1 (otherwise, we don't even have a connected graph), then that running time is O(m lg n). That's always better than using an unsorted array, right? No. We say that a graph is dense if m = Θ(n2), that is if on average every vertex has Θ(n) neighbors. In a dense graph, Dijkstra's algorithm runs in time O(n2 lg n), which is worse than using an unsorted array. It is somewhat embarrassing that the "clever" heap is beaten by the "naive" unsorted array.
rooted tree
given a tree T, choose a root node r and orient each edge away from r
random-restart hill climbing
iteratate the algorithm, beginning each from random point
Adjacency matrix space
n^2
Do shortest paths contain shortest paths between nodes in path
no?
connected component theorem
upon termination, R is the connect component containing s
2 greedy algorithms for finding the minimum spanning tree?
- Kruskal's - Prim's
checking for strong connectivity in a directed graph
- Pick any node s. - Run BFS from s in G - Run BFS from s in G^rev - return true iff all nodes reached in both BFS executions. - correctness follows immediately from previous lemma
proof 2 of Let G be a connected graph and let L0, ...., Lk be the layers produced by BFS starting a node s. Exactly one of the following holds. An edge of G joins two nodes of the same layer, and G contains an odd-length cycle (and hence is not bipartite)
- Suppose (x,y) is an edge with x,y in same level Lj. - Let z = lca(x,y) = lowest common ancestor. - Let Li be level containing Z. - Consider cycle that takes edge from x to y, then pass from y to z, the pass from z to x - its length is 1 + (j-i) + (j-i), which is odd
BFS adjacency list proof of O(n^2) running time.
- at most n lists in L[i] - each node occurs on at least one list; <=n iterations - when we consider node u, there are <= n incident edges (u,v), and we spend O(1) processing each edge
hill-climbing variants:
- basic hill climbing - steepest hill climbing - stochastic hill climbing - random-restart hill climbing
In greedy algorithms, each decision must be:
- feasible (satisfy problem constraints) - locally optimal (best available step) - irrevocable (never reviewed or replaced)
Greedy approaches for knapsack problem?
- pick by highest value - pick by lowest weight - pick by highest value/weight ratio
BFS adjacency list proof of O(m + n) run time
- when we consider a node u, there are deg(u) incident edges (u,v) - total time processing edges is sigma(u in set V) deg(u) = 2m, m = number of edges
Breadth First Search algorithm
-L0 = {s} -L1 = all neighbors of L0 -L2 = all nodes that do not belong to L0 and L1, and that have an edge from a node in L1 -Li + 1 = all nodes that do not belong to an earlier layer, and that have an edge from a node in Li - mark discovered nodes - stop when no new nodes are found
Dijkastra's Algorithm
-Requires all positive weights, but has lower complexity than Bellman-Ford -also a single source shortest path algorithm
Dijkastra's Algorithm (works?)
-has a list of all vertices whose shortest path has been found, and a min-priority queue with shortest path estimates -close off a node once shortest path has been found, then see if you can shorten other paths by going through the closed node
Iterative Level Order Scan
-scans the tree by level rather than going up or down -uses a queue
depth of a tree best traversal?
-since it is recursive (based on subtrees), use Post-Order LRN -depth = 1 + maxDepth of subtrees -template <typename T> void Depth(tnode<T> *t) { int depthVal, depthLeft, depthRight; if(t == NULL) depthVal = -1; // depth of an empty tree else { depthLeft = Depth(t->left); depthRight = Depth(t->right); depthVal = 1 + (depthLeft > depthRight ? depthLeft: depthRight); } return depthVal; }
Bellman-Ford Algorithm
-single source shortest paths -Finds shortest path of weighted graph by starting at source node and 1st finding shortest NON-WEIGHTED path. It then iterates and updates to shorter known paths each time -handles negative weights -initialize source to 0 and paths to infinity, then relax them - returns false if it contains reachable negative cycles
Let G be a connected graph and let L0, ...., Lk be the layers produced by BFS starting a node s. Exactly one of the following holds.
1. No edge G joins two nodes of the same layer and G is bipartite 2. An edge of G joins two nodes of the same layer and G contains an odd-length cycle and is not bipartite
Rooted
A tree is rooted if it comes with a specified vertex, called the root
bipartite definition
An undirected graph G = (V,E) is bipartite if the nodes can be colored red or blue such that every edge has one red and one blue end
tree
An undirected graph is a tree if it is connected and does not contain a cycle
How to solve an unweighted shortest path problem?
An unweighted shortest path problem can be solved by treating all edges as having weight = 1. Therefore, it can be solved as a special case of the weighted shortest path problem.
Kruskal's algorithm
Builds the minimum spanning tree edge-by-edge. Start with current spanning tree as just the unconnected nodes, then sort the edges from smallest to largest and process them. Determine if the edge spans two different connected components in the current spanning tree, if it doesn't you ignore it. If it does, add the edge to the spanning tree, and the two connected components become one. Repeat until there's one connected component. To implement, sort edges in a multimap and use disjoint sets to identify connected components.
Minimum Spanning Tree Solution (Weighted)
Choose between Kruskal's or Prim's algorithm
D Step 3
Choose the least of all temporary labels on the network. Make this label permanent by boxing it.
Prim's algorithm in a distance matrix
Choose vertex to start tree, delete row for chosen vertex and number column; ring lowest undeleted entry in numbered column to become next arc; repeat until all rows deleted
Iterative Deepening Search
Combines DFS and BFS to get shortest path
D Step 2
Consider the node with the most recently boxed label. Suppose this node to be X and let D be its permanent label. Then, in turn, consider each node directly joined to X but not yet permanently boxed. For each node, Y say, temporarily label it with the lesser of D+(the weight of arc XY) and its existing label (if any.)
Shortest Path Solution
Dijkstra's algorithm (using nodes, total lengths, and previous nodes
Shortest Path Input
Directed Graph, G=(N,A) (a tuple of nodes and arcs) lengths l(i, j)>=0
compression ratio
Expected number of bits per character calculated as summation of frequency * number of bits
Finding the minimum cut
First find all nodes that are reachable from the source in the residual graph. The minimum cut is composed of all edges in the original graph that connect the reachable nodes to the non-reachable nodes.
Breadth First Search algorithm theorem
For each i, Li consists of all nodes at distance exactly i from s. There is a path from s to t iff t appears in some layer.
short path algorithm
It is probably not immediately clear why Dijkstra's algorithm correctly finds the shortest path from the start vertex s to each other vertex u in the graph. Why is it that the distance from s to u is equal to the value of the label D[u] at the time vertex u is removed from the priority queue Q and added to the cloud C? The answer to this question depends on there being no negative-weight edges in the graph, for it allows the greedy method to work correctly, as we show in the proposition that follows.
Single Source Shortest Path
Keep track of shortest path from a given source to every other node
Dijkstra idea
Keep track of the vertex we take every time we update a dist[I] value. Then follow the vertices backward to the source to reconnect the path.
Kirchhof's Theorem
Let M be the matrix obtained from the adjacency matrix of a connected graph G by changing all 1's to -1's and each diagonal 0 to the degree of the corresponding vertex. Then the number of spanning trees of G is the value of any cofactor of M.
update the dist[j] values for all adjacent vertices j
MIN{ dist[i] + weight[I,j], dist[j] }
Ford-Fulkerson (augmenting paths) algorithm
Maintain three graphs: copy of the original, a "flow" graph (keeps track of flow discovered through the graph) and a "residual" graph (keeps track of potential flow left in the graph). 1) Flow graph starts empty, residual graph starts out equaling the original. 2) At each step, start by finding a path from the source to the sink that has flow. If you can't find a path, you're done. 3) Add flow along path to the flow graph. If there's already flow in the reverse direction, you can subtract from that flow first, before adding additional flow to the graph. 4) Subtract the flow along the path in the residual graph. 5) Traverse the path backward, from sink to source, adding the flow along that path in the backward direction.
Kruskal - Priority Queue Sorted Array (Build)
O(mlg(n))
Breadth First Search (BFS)
O(n+m)
Depth First Search (DFS)
O(n+m)
Kruskal - Priority Queue Sorted Array (Total)
O(n+m+mlg(n)) - upper bound is O(mlg(n))
Dijkstra - Adjacency List/Unsorted Array
O(n^2)
Dijkstra - Adjacency Matrix/Unsorted Array
O(n^2)
Prim - Adjacency List/Unsorted Array
O(n^2)
Prim - Adjacency Matrix/Unsorted Array
O(n^2)
time to identify all edges in Adjacency matrix
O(n^2)
Dijkstra - Adjacency Matrix/Heap
O(n^2+mlg(n))
Prim - Adjacency Matrix/Heap
O(n^2+mlg(n))
Dijkstra - Going through all pairs - O(n(m+nlg(n))) (Sparse - m~n)
O(n^2+n^2lg(n))
Breadth first complexity
O(|V| + |E|)
Depth first complexity
O(|V| + |E|)
All Pair Shortest Path
Shortest path from each node to every other node
why does A*'s heuristic function need to be admissible?
Some paths may not be explored. If the heuristic overestimates the likely cost for some path, it may not be explored even though it's really the best choice.
Kruskal's algorithm
Sort the arcs until ascending order of weight and use arc of least weight to start tree; add arcs in order of ascending weight unless cycle formed
Topological sort
Sorting of vertices of a directed, acyclic graph, where if there is an edge from a to be then a comes before b in the sorting. To implement, maintain a list of nodes with no incoming edges, then until the list is empty, remove a node from the list and append it to the topological sort. For each edge coming out of the node, remove the edge from the graph. If that edge was a node that now has no more incoming edges, put that node on the list.
What is an unweighted path length?
The number of edges in the path, N-1
Theorem 12.2.3
The number of labeled trees with n>=3 vertices is n^(n-2).
Dijkstra's vs. topological sort
The tradeoff is number of edges processed vs. time to process edges. Topological sort must process all of the edges of the graph; however, it takes O(1) time to process each edge. Dijkstra's algorithm quits when it finds the shortest path, and it may do so well before processing all of the edges of the graph, as in the example above. However, because it manages a multimap, it takes O(|V|) times to process each edge. So, if Dijkstra's algorithm processes a fraction of the total number of edges in finding the shortest path, then it will be faster. If the two algorithms must process roughly the same number of edges, the topological sort will be faster.
Kruskal's Algorithm
To solve MST. Gather all the edges in a pq. One by one, pick the smallest edge in the PQ and remove it. That edge is its own tree until it connects to another one. Continue until it's spanning.
Prim's Algorithm
To solve MST. Pick a vertex, record all its edges weights to vertices that aren't in the tree in a pq. Pick the edge that has the lowest weight in the pq, add its destination vertex to the tree, put its edged in the pq. Repeat until done.
Floyd-Warshall Algorithm
Used to find the shortest distance between all pairs of vertices in a weighted graph where the are v1, v2,..., vn.
Dijkstra's Algorithm
Used to the find the shortest path from vertex A to vertex E in a weighted graph.
Breadth first search
Uses a queue and pushes nodes onto the queue. Used to find the shortest path from a starting node to all the other nodes in the graph by storing a back-edge (the edge that first put n onto the queue). Can also store n's distance to the starting node.
Bellman-Ford Algorithm (iterations?)
V-1
How can shortest path be used to model planes?
Vertices represent airplanes or other traffic routes, the shortest path represents the best route between two points.
How can shortest path be used to model computer communication costs?
Vertices represent computers, edges represent links, weights represent communication costs. Shortest path is the cheapest route.
huffman trees
[1] Initialize one-node trees, one for each character ➤ Each tree's "weight" is frequency of that character [2] Find two trees with smallest weight (ties broken arbitrarily) ➤ Make these the left and right subtrees of a new tree ➤ New tree's weight is sum of their weights [3] Repeat #2 until all nodes in one tree [4] Trace paths in this tree to generate "Huffman codes"
Dijkstra's Search Algorithm
[1] Mark best-known costs for all nodes ‣ source node has cost 0 and is fixed at start ‣ nodes one step away have that edge cost ‣ all other nodes initially have infinite cost [2] Select an unfixed node with the lowest cost - Fix that node/cost (won't be changed later) and update any estimated costs where we can now see a better (lower) cost [3] Repeat #2 until all node costs are fixed
Prim's Algorithm
[1] Select any arbitrary node in the graph, use to initialize a set of connected nodes [2] Pick any minimum weight edge connecting any node A in your set of connected nodes with any node B not in the set ➤ Add node B to set of connected nodes ➤ Add node B & edge to min spanning tree ➤ Add edge weight to total cost of MST [3] Repeat [2] until all nodes in connected set
Kruskal's Algorithm
[1] Sort edges by weight (nondescending) [2] Create N subgraphs, each with one node [3] Pick any minimum weight edge connecting any two nodes A and B that are not already joined in the same subgraph ➤ Join subgraphs with A and B ➤ Add subgraph weights and edge weight [4] Repeat [3] until all nodes appear in the same subgraph
simple path
a path is simple if all nodes are distinct
minimum spanning tree
acyclic subgraph that connects all vertices in an undirected graph
A* algorithm guarantees an optimal answer if the heuristic function is ___________ - it never overestimates the distance to the goal.
admissible
complete binary tree
all levels, expect for the last level, are full
full binary tree
all nodes except leaves have exactly two children (no node with only one leaf)
A* ("A star")
an adaptation of Dijkstra's that uses an informed search or "best first" search approach to find the shortest path to some goal.
a connected undirected graph
an undirected graph is connected if for every pair of nodes u and v, there is a path between u and v
A hill-climbing algorithm starts with an __________ ___________, then tries to __________ __________ that by incrementally changing one element.
arbitrary solution iteratively improve
HC algorithms are commonly used in ________ __________ systems, in _________-_______ searches, and for ______ optimization.
artificial intelligence infinite-space pure
Undirected graph
capture pairwise relationship between objects
Why toll method is valid
changes the length of every s to i path by same amount
making change
classic problem that asks for a collection of coins from a given set of coin values that sums to some amount
variable-length encoding
codewords have varying lengths example: morse code - 2/3/4 encoding
steepest hill climbing
compare all immediately available options, choose the best (not first)
Prim's algorithm is faster than Kruskal's when the graph is...
dense
Directed graph
edge (u,v) goes from node u to node v
heuristic
enabling a person to discover or learn something for themselves
binary tree
every node has 0, 1, or 2 children
fixed-length encoding
example: ASCII - 7 bit encoding
Depth First intuition
explore by going as deep as possible
how does BFS explore a connected component starting at s
explore in order of distance from s
Breadth First Search intuition
explore outward from s in all possible directions adding nodes one "layer" at a time
simulated annealing begins with a _______ solution (e.g., a set of knapsack items within limit W) and then ________ and ___________ random pieces (e.g., removing some item and adding another).
feasible picks modifies
connected component
find all nodes reachable from s
Single Destination Shortest Path
find shortest path from given destination to eery other node
What does Dijkstra's Search Algorithm do?
finds a single-source shortest path, from one node to each of the other nodes.
Some greedy algorithms are always optimalbut greedy algorithms are commonly used to find a "___________" answer
good enough
hill-climbing algorithms family of _________ algorithms that seek and return a _______ _________ result (local maximum).
greedy locally optimal
Any application of A* needs a _________ function specific to the context.
heuristic
Bipartite theorem
if a graph is bipartite, it cannot contain an odd length cycle
cycle
is a path v1, v2, ... vk-1, vk in which va = vk, k > 2 and the first k-1 nodes are all distinct
path in an undirected graph G
is a sequence of p nodes v1, v2, ..., vk-1, vk with the property that each consecutive pair vi, vi+1 is joined by an edge E
Where Dijkstra's algorithm considers only the ______ cost to some node V, A* considers the known cost to V as well as an ________ of the distance from V to the goal node.
known estimate
Hill-climbing algorithms often find reasonable solutions in ______ problem spaces and in ______ time.
large short
Adjacency list space
m + n
Hill-climbing algorithms use very little __________, typically tracking only the _______ __________ and maybe a ______-known solution.
memory current solution best
If the graph is weighted (edges have cost), a ________ _________ tree is a spanning tree of smallest possible cumulative weight
minimum spanning
At each step, A* identifies and explores the node _____ likely to lead to the goal with least cost.
most
Adjacency matrix
n X n matrix with Auv = 1 if (u,v) is an edge
Adjacency list
node indexed array of lists, ex, in array index 1, it will contain a list of nodes it is connected to
Dijkstra's algorithm finds optimal paths as long as all edge weights are ______-___________.
non-negative
simulated annealing
nondeterministic method for approximating optimization problems, modeling the physical process of annealing.
Many greedy algorithms will often find an ________ answer and almost always find at least a decent approximation of the "______" answer.
optimal best
Greedy Algorithms are proved by.....
proof by contradiction
Obviously a better heuristic estimate is going to _______ the number of steps (sub-paths) in comparison to a worse estimate.
reduce
basic hill climbing
search available changes, choose the first option that is better
stochastic hill climbing
select neighbor state at random, jump there if it's "enough" better
Kruskal's algorithm is faster than Prim's when the graph is...
sparse
We can consider Dijkstra's algorithm to be a ________ case of A* with all estimates equal.
special
proof by contradiction Assume that the algorithm produces a ________ answer for some input so at some point a decision was made that diverged from the "________" answer. But if we can use the algorithm to show that this "_______" decision cannot have happened, then the algorithm must be _________.
suboptimal best bad optimal
proof 1 of Let G be a connected graph and let L0, ...., Lk be the layers produced by BFS starting a node s. Exactly one of the following holds. No edge of G joins two nodes of the same layer, and G is bipartite
suppose no edge joins two nodes in the same layer. Bipartition: red = nodes on odd length, blue = nodes on even nodes
simulated annealing - If the new solution is less optimal, it is accepted probabilistically based on the "_________," which begins high and decreases over time. This means bigger random jumps early, with small oscillations later around the maximum.
temperature
implementation in C++
template <typename T> class tnode { public: T nodevalue; //data tnode<T> *left, *right; //pointers to children tnode() {}; tnode(const T &item, tnode<T> *lptr=null,tnode<T> *rptr=null) :nodeValue(item), left(lptr), right(rptr){}; } //parent node points to children, first allocate room for children
traversal?
trees are graphs, so we can traverse them, but there are specialized algorithms that are optimized for trees
There must be an _________ translation for the encoded text. We must create _________-________ codewords(so that no codeword is a prefix of any other)
unambiguous prefix-free
dynamic runtime for making change
value * # of coins V*C because you are constantly looking back
greedy runtime for making change
value + # of coins V+C
preorder scan NLR
visit node, go left, go right
Step 5
Repeat steps 3 and 4 until the destination vertex has been given a permanent label
What is the idea behind A*?
Sometimes we are looking for the shortest path to a *known goal*. In these cases, we can use a generalization of Dijkstra's algorithm called A* search to find the shortest path without taking as much time as normal Dijkstra's
Topological Ordering
Such ordering of the vertices in which every edge goes forward To have it graph must be directed acyclic Intuitive algorithm: - Find a sync vertex, assigh it highest number - Remove that vertex, reverse on the remaining graph DFS solution: 1) DFS-loop: for all vertices: if it's not explored, start DFS 2) DFS: - mark v explored - for all edges going out of v perform DFS - assign v score s - s-- Proof: take edge (u, v) from E. Need to prove that f(u) < f(v). Have 2 cases: 1) if DFS was started from u and v was found, then f(v)>f(u) by design of this algorithm 2) if DFS from u did not find v, then it means that it was already found by some earlier DFS (otherwise it would be found). This means that f(v)>f(u)
In Dijkstra's algorithm, whenever a vertex v is pulled into the cloud, the label D[v] is equal to d(s,v), the length of a shortest path from s to v.
Suppose that D[v] > d(s,v) for some vertex v in V, and let z be the first vertex the algorithm pulled into the cloud C (that is, removed from Q) such that D[z] > d(s, z). There is a shortest path P from s to z (for otherwise d(s, z) = ∞ = D[z]). Let us therefore consider the moment when z is pulled into C, and let y be the first vertex of P (when going from s to z) that is not in C at this moment. Let x be the predecessor of y in path P (note that we could have x = s). (See Figure 14.18.) We know, by our choice of y, that x is already in C at this point.
main ides of algorith in greedy algorithms
The algorithm terminates when no more vertices are outside the cloud (or when those outside the cloud are not connected to those within the cloud), at which point we have a shortest path from s to every vertex of G that is reachable from s. This approach is a simple, but nevertheless powerful, example of the greedy-method design pattern. Applying the greedy method to the single-source, shortest-path problem, results in an algorithm known as Dijkstra's algorithm
update the cost of getting from the source vertex to every other vertex
MIN{ weight[ source, I ] , dist [I] }
Floyd-Warshall (Dense - m~n^2)
O(n^3)
Floyd-Warshall (Sparse - m~n)
O(n^3)
Dijkstra - Going through all pairs - O(n(m+nlg(n))) (Dense - m~n^2)
O(n^3+n^2lg(n)) = O(n^3)
Dijkstra - Adjacency List/Fibonacci Heap
O(nlg(n) + m) w/ O(1)* - amortized for decrease-key operation
Prim - Adjacency List/Fibonacci Heap
O(nlg(n) + m) w/ O(1)* - amortized for decrease-key operation
Dijkstra - Adjacency List/Heap
O(nlg(n)+mlg(n)) - upper bound is O(mlg(n))
Prim - Adjacency List/Heap
O(nlg(n)+mlg(n)) - upper bound is O(mlg(n))
Big O of |E|
O(|V|^2) - worst case the # E in Kn = O(|V|^2) - its over estimated. Most of the time |E| is less than that
Since you don't just need to know how much it costs to reach the finish node---you want to know the path we need to take to get there---you're required to use another data structure to keep track of...
... the parent node in each node. If a node has many parents, just keep the parent node that leads to the cheapest cost. This is how we retrace the cheapest path it takes to get from start to finish: const parents = { A: 'start', B: 'start', finish: null }; So now there's two objects besides the graph object: costs and parents
Programming Dijkstra's Algorithm in Java
1 /** Computes shortest-path distances from src vertex to all reachable vertices of g. */ 2 public static <V> Map<Vertex<V>, Integer> 3 shortestPathLengths(Graph<V,Integer> g, Vertex<V> src) { 4 // d.get(v) is upper bound on distance from src to v 5 Map<Vertex<V>, Integer> d = new ProbeHashMap<>( ); 6 // map reachable v to its d value 7 Map<Vertex<V>, Integer> cloud = new ProbeHashMap<>( ); 8 // pq will have vertices as elements, with d.get(v) as key 9 AdaptablePriorityQueue<Integer, Vertex<V>> pq; 10 pq = new HeapAdaptablePriorityQueue<>( ); 11 // maps from vertex to its pq locator 12 Map<Vertex<V>, Entry<Integer,Vertex<V>>> pqTokens; 13 pqTokens = new ProbeHashMap<>( ); 14 15 // for each vertex v of the graph, add an entry to the priority queue, with 16 // the source having distance 0 and all others having infinite distance 17 for (Vertex<V> v : g.vertices( )) { 18 if (v == src) 19 d.put(v,0); 20 else 21 d.put(v, Integer.MAX VALUE); 22 pqTokens.put(v, pq.insert(d.get(v), v)); // save entry for future updates 23 }
Dijkstra algorithm
1. mark all dist[I] to oo 2. Mark the starting point dist[source] to 0 3. while there are unvisited vertices: - find the unvisited vertex with minimum dist[I] value - visit the vertex -update the dist[I] for its unvisited neighbors Find the cheapest path (smallest dist[I] value and visit it
For this to find the shortest path we need two things.
1. the estimate of distance remaining must be *admissible*. This means that *we always underestimate the true remaining distance or get it exactly.* (i.e. "at least: it could be more but it certainly isn't less") Because of the triangle inequality the Euclidean distance is an underestimate of the true travel distance. In fact, the ultimate underestimate of the remaining distance is 0. This leads to normal Dijkstra, because then we take the shortest distance traveled next. 2. the cost estimate is monotone. That means that if we extend the path (thus increasing the distance travelled so far), the total estimate is never less that the estimate before we extended the path. This is the generalization of "no negative edges." Our Euclidean distance plus distance so far estimate satisfies this, because if moving to an adjacent vertex increases the distance so far by d it cannot reduce the Euclidean distance by more than d. (It would reduce by exactly d if you were moving directly toward the goal.) Thus the sum will not go down. Using just Euclidean distance (without distance so far) would fail to be monotone, so would not be not guaranteed to give shortest paths.
java for dijkstra algorithm
24 // now begin adding reachable vertices to the cloud 25 while (!pq.isEmpty( )) { 26 Entry<Integer, Vertex<V>> entry = pq.removeMin( ); 27 int key = entry.getKey( ); 28 Vertex<V> u = entry.getValue( ); 29 cloud.put(u, key); // this is actual distance to u 30 pqTokens.remove(u); // u is no longer in pq 31 for (Edge<Integer> e : g.outgoingEdges(u)) { 32 Vertex<V> v = g.opposite(u,e); 33 if (cloud.get(v) == null) { 34 // perform relaxation step on edge (u,v) 35 int wgt = e.getElement( ); 36 if (d.get(u) + wgt < d.get(v)) { // better path to v? 37 d.put(v, d.get(u) + wgt); // update the distance 38 pq.replaceKey(pqTokens.get(v), d.get(v)); // update the pq entry 39 } 40 } 41 } 42 } 43 return cloud; // this only includes reachable vertices 44}
What's a simple path vs a cycle?
A simple path doesn't contain repeated vertices. A cycle is the same thing as a simple path, except for the last vertex, which is the same as the first: A->D->C->A is a cycle, for example
Algorithm Description and Example
Algorithm ShortestPath(G, s): Input: A directed or undirected graph G with nonnegative edge weights, and a distinguished vertex s of G. Output: The length of a shortest path from s to v for each vertex v of G. Initialize D[s] = 0 and D[v] = ∞for each vertex v 6= s. Let a priority queue Q contain all the vertices of G using the D labels as keys. while Q is not empty do {pull a new vertex u into the cloud} u = value returned by Q.removeMin( ) for each edge (u,v) such that v is in Q do {perform the relaxation procedure on edge (u,v)} if D[u]+w(u,v) < D[v] then D[v] = D[u]+w(u,v) Change the key of vertex v in Q to D[v]. return the label D[v] of each vertex v
Kosaraju's Two-Pass Algorithm for finding Connected Components in directed graph
Connected Components - equivalence classes of relation: u rel. v <=> there is a path (u,v) and there is a path (v,u) Equivalence relation: reflexive, symmetric, transitive Alg: 1. numbering a) Set current_label= 0 b) DFS-Loop on G-rev (all edges reversed): for each vertex if it is not explored DFS(this vertex) c) DFS(v) set v explored for all edges (v,w) if w unexplored DFS(w) set f(w) = current_label current_label ++ 2. finding SCC's a) DFS-Loop for i from n to 1 get vertex v with f(v)=i if v unexplored DFS(v, v) b) DFS(vertex v, parent p) set v explored parent(v) = p for all edges(v,w) if w unexplored DFS(w,v) Why this works? SCC's of graph induce a directed acyclic metagraph Also, SCCs of G-rev and G are the same. Lemma: consider 2 adjacent SCC's: there is an edge from SCC1 to SCC2. Claim that: max(f(v)) from SCC1 < max(f(v)) from SCC2 If this lemma is true, then in second step we will start from sink SCC, explore it, then go to next sink SCC, which is exactly what we need. Proof of lemma: take SCC1 and SCC2, there is and edge i->j (i from SCC1, j from SCC2). In G-rev there is and edge j->i let u from SCC1&SCC2 be the first vertex explored in numbering DFS-Loop 1) if u from SCC1, then all SCC1 was explored before SCC2, and thus for all v from SCC1 and w from SCC2 f(v) < f(w) 2) if u from SCC2, then f(u) = max( f from SCC1&SCC2). Thus for all v from SCC1 f(v) < f(u)
DFS
Depth First Search Motivation: find everything findable from source vertex s As opposite to BFS, acts "greedily", like when exploring a maze Impl1: 1) mark all vertices as unexplored 2) mark s explored, add to stack S 3) while S non-empty - pop vertex v from S - for all (v,w) if w unexplored mark as explored, add to S Impl2: DFS(v) for all (v,w) if w unexplored DFS(w) Applications: a) topological ordering in directed graph b) connected components in directed graph
What does Dijkstra's Algorithm do?
Dijkstra's algorithm finds a shortest path from a source vertex s to all other vertices. It keeps track of 2 things for each vertex v in a graph: --> the weight of a shortest path from the source to v (v.dist) --> the back-pointer for v (predecessor v.pred) on each shortest path.
Implementing Dijkstra's algorithm: What key data structure is needed?
Dijkstra's algorithm maintains a *min-priority queue* of vertices, with their dist values as the keys.t repeatedly extracts from the min-priority queue the vertex u with the minimum dist value of all those in the queue, and then it examines all edges leaving u. If v is adjacent to u and taking the edge (u, v) can decrease v's dist value, then we put edge (u, v) into the shortest-path tree (for now, at least), and adjust v.dist and v.pred. Let's denote the weight of edge (u, v) by w(u,v). We can encapsulate what we do for each edge in a relaxation step, with the following pseudocode (See crib sheet)
Sending runners Out vs. Dijkstra's
Dijkstra's algorithm works slightly differently. It treats all edges the same, so that when it considers the edges leaving a vertex, it processes the adjacent vertices together, and in no particular order.For example, when Dijkstra's algorithm processes the edges leaving vertex s, it declares that y.dist = 4, t.dist = 6, and y.pred and t.pred are both s—*so far* (please see Week7#2 for images). When Dijkstra's algorithm later considers the edge (y, t), it decreases the weight of the shortest path to vertex t that it has found so far, so that t.dist goes from 6 to 5 and t.pred switches from s to y.
Precondition for Dijkstra's
Dijkstra's algorithm, is guaranteed to find shortest paths *only when all edge weights are nonnegative*When edge weights are required to be nonnegative, Dijkstra's algorithm is often the algorithm of choice.
What does connected/unconnected mean in the context of graphs?
Directed means the each connection is one-way; undirected means that connections can run both ways.
What's an adjacency matrix? What kind of graph is it good for?
Each node is associated with an integer, which is the array index. We represent the connectivity between vertices using a two dimensional array, like array[i][j] = 1 if there's an edge from the node with index i to index j and 0 if there isn't A B C D A 0 0 0 1 B 0 0 1 1 C 0 1 0 0 D 1 1 0 0 In this, A and D are connected, and B is connected with C and D The adjacency matrix is good for dense graphs, because the same amount of memory is used no matter how many edges are in the graph
What does a weighted graph look like?
Each node is represented by the keys in the graph object. Each key has an object for its value, which represents the immediate neighbors and the cost of reaching that neighbor. For example, node A is connected to nodes C and D. Node A is the "parent" to nodes C and D, and they are the "children" of node A. const graph = { start: {A: 5, B: 2}, A: {C: 4, D: 2}, B: {A: 8, D: 7}, C: {D: 6, finish: 3}, D: {finish: 1}, finish: {} };
Step 3
Find the smallest working value and give the corresponding vertex (v) this value as a permanent label
Graph Primitives
Graph - combination of two sets: V - vertices E - edges Edges can be directed and undirected |V| = n, |E| = m for graphs where each 2 vertices can be connected by 0 or 1 edge, number of edges in a connected graph lies in range [n-1, n*(n-1)/2] Graphs can be sparse and Dense Graph representations: 1) Adjacency Matrix - theta(n^2) space 2) Adjacency Lists - list of vertices - theta(n) - list of edges - theta(m) - for each edge - pointers to head and tail - theta(2*m) = O(m) - for each vertex - pointers to adjacent edges - same size as previous - theta(2*m) = O(m) So the whole space required = O(n + m) = O(max(n,m)) = O(m)
Minimum Cuts
Graph Cut - division of vertices into two non-empty sets A and B Number of possible minimum cuts: 2^|V| Crossing edges: 1) for undirected graph - such edges that one vertex is from A and another from B 2) for directed graph - such edges that have tail in A and head in B Minimum Cut - such cut that has minimum number of crossing edges Randomized contraction algorithm: 1. while |V| > 2 a) take random edge (u,v) b) contract u and v to one vertex u c) remove self-loops 2. output cut formed by 2 vertices This algorithm yields just some min cut, not necessarily the minimum one! By probability review, n^2 repeated trials needed to get min cut with probability 1/n. (probability review in the picture)
Dijkstra Heap Implementation
Heap data structure: maintains extract-min and delete in O(n* logn) time. Let X be the set of vertices that are already processed Maintain 2 invariants: 1) V-X vertices are stored in heap 2) key(v) = smallest Dijkstra greedy score of all edges (u,v) with u from X by invariants, extract-min yields the next vertex to be added to X Maintaining invariant2: when the vertex v is added to X, only keys of vertices that have edge (v,w) and are still in V-X are changed. So for these vertices: extract vertex - recompute key - put vertex back Running time: 1) initial heapify - O(n) 2) n-e Extract-Min, each O(log(n)) 3) each edge triggers at most 1 Delete-Insert <= m*log(n) All together: O(m * logn) Note that algorithm requires deletions from the heap. Thus should somehow maintain some mapping between the vertex and it's location in the heap, to allow it to be found in logarithmic time. Deleting itself also takes log(n).
BFS Vs. Dijkstra's
If you want to find paths with the minimum number of edges, then breadth-first search is the way to go. In other words, BFS is great for unweighted graphs. Dijkstra's algorithm generalizes BFS, but with weighted edges.
How is traversing a graph different than traversing a tree?
In a graph, you need to keep track of which vertices haven't been visited yet and which have been completely explored, as well as which will be the first vertex we visit
key aspect of while-loop
In the simulation with runners, once a vertex receives dist and pred values, they cannot change, but here a vertex could receive dist and pred values in one iteration of NFINITY the while-loop, and a later iteration of the while-loop for some other vertex u could change these values. For example, after edge (y, x) is relaxed in part (c) of the figure, the value of x.dist decreases from ∞ to 13, and x.pred becomes y. The next iteration of the while-loop in (part (d)) relaxes edge (t, x), and x.dist decreases further, to 8, and x.pred becomes t. In the next iteration (part (e)), edge (z, x) is relaxed, but this time x.dist does not change, because its value, 8, is less than z.dist + w(z,x), which equals 12.
The Running Time of Dijkstra's Algorithm
In this section, we analyze the time complexity of Dijkstra's algorithm. We denote with n and m the number of vertices and edges of the input graph G, respectively. We assume that the edge weights can be added and compared in constant time. Because of the high level of the description we gave for Dijkstra's algorithm in Code Fragment 14.12, analyzing its running time requires that we give more details on its implementation. Specifically, we should indicate the data structures used and how they are implemented
Edge Relaxation
Let us define a label D[v] for each vertex v in V, which we use to approximate the distance in G from s to v. The meaning of these labels is that D[v] will always store the length of the best path we have found so far from s to v. Initially, D[s] = 0 and D[v] =∞for each v 6= s, and we define the set C, which is our "cloud" of vertices, to initially be the empty set. At each iteration of the algorithm, we select a vertex u not in C with smallest D[u] label, and we pull u into C. (In general, we will use a priority queue to select among the vertices outside the cloud.) In the very first iteration we will, of course, pull s into C. Once a new vertex u is pulled into C, we update the label D[v] of each vertex v that is adjacent to u and is outside of C, to reflect the fact that there may be a new and better way to get to v via u. This update operation is known as a relaxation procedure, for it takes an old estimate and checks if it can be improved to get closer to its true value. The specific edge relaxation operation is as follows: Edge Relaxation: if D[u]+w(u,v) < D[v] then D[v] = D[u]+w(u,v
Kruskal - Priority Queue Heap (Total)
O(n+m+mlg(n)) - upper bound is O(mlg(n))
Why It Works
The interesting aspect of the Dijkstra algorithm is that, at the moment a vertex u is pulled into C, its label D[u] stores the correct length of a shortest path from v to u. Thus, when the algorithm terminates, it will have computed the shortest-path distance from s to every vertex of G. That is, it will have solved the single-source shortest-path problem
Why are negative weights a problem?
There is a situation that we have to watch out for when edge weights can be negative. Suppose that there is a cycle whose total weight is negative. Then you can keep going around and around that cycle, decreasing the cost each time, and getting a path weight of − ∞. Shortest paths are undefined when the graph contains a negative-weight cycle.
What is Dijkstra's Algorithm?
This algorithm uses a directed, weighted graph to determine the "cheapest" path to reach a node.
Fibonacci-heap for min-PQ
This embarrassment lasted for decades, until the invention of a data structure called a Fibonacci heap that implements extractMin in O(lg n) amortized time (amortized over all the operations) and insert and decreaseKey in Θ(1) amortized time. With a Fibonacci heap, Dijkstra's algorithm runs in time O(n lg n + m), which is at least as good as using either an unsorted array or a min-heap. Fibonacci heaps are a little tricky to implement, and their hidden constant factors are a little worse than those for binary heaps, but they're not as hard to implement as some people seem to think.
Shortest path
We call a path from vertex u to vertex v whose weight is minimum over all paths from u to v a shortest path, since if weights were distances, a minimum- weight path would indeed be the shortest of all paths u to v.
Why
We can use an unsorted array for the min-priority queue. Each insert and decreaseKey operation takes Θ(1) time. Each extractMin operation takes time O(q), where q is the number of vertices in the min-priority queue at the time. We know that q ≤ n always, and so we can say that each extractMin operation takes O(n) time. But wait—what about when q is small? The way to think about it is to look at the time for all n extractMin operations altogether. You can do the analysis that results in an arithmetic series in n, or you can do it in a simpler way. We have n operations, each taking O(n) time, and so the total time for all extractMin operations is O(n2). If we look at just the first n/2 extractMin operations, each of them has to examine at least n/2 vertices to find the one with the smallest dist value, and so just the first half of the extractMin operations take Ω(n2) time. Hence the total time for the n extractMin operations is Θ(n2), as is the total time for Dijkstra's algorithm.
Abstraction*
You might wonder, since the min-priority queue is going to be implemented by a min-heap, why not just have the Dijkstra's algorithm code keep track of the index of each vertex's index in the array that implements the heap? The problem is that the premise violates our goal of abstraction. Who is to say that a min-heap implements the min-priority queue? In fact, we could just use an unsorted array. Or any other implementation of a min-priority queue. The Dijkstra's algorithm code should know only that it's using a min-priority queue, and not how the min-priority queue is implemented.
Dijkstra Algorithm
computes shortest paths from source vertex s to all other vertices in a directed graph with edge weights >=0. Define X - subset of processed vertices, W = V-X. Initialize X with s. Let A[v] = size of shortest path, B[v] = actual shortest path (just for curiosity) While(X != V){ through all edges (u,v) with u in x and v in V-X find v which minimizes value S = A[u] + weight(u,v) add v to x, A[v] = S, B[v] = B[u] + (u,v) } Why this works? Proof by induction: 1) for i = 0 A[s] = 0 - true 2) assume that B[u] = shortest path to u, A[u] = it's size 3) prove that B[v] is shortest path to v, A[v] = shortest path. Any path to v has to somewhere cross the frontier X / W. Let's assume that there is another path that has size smaller then A[V]. This another path consists of: .... -> x0 -> w0 -> .... -> v size of this path S1 = A[x0] + weight(x0,w0) + (sum of weights of edges in W) BUT: A[x0] + weight(x0, w0) >= A[v] by definition of algorithm, and (sum of weights)>=0 because all weights are >= 0. So we have contradiction!
shortest path?
sum of the edge weights along each path , not the number of edges in each path
What are the main steps in Dijkstra's algorithm?
the "cheapest" node. Update the costs of the immediate neighbors of this node. Repeat steps 1 and 2 until you've done this for every node. Return the lowest cost to reach the node, and the optimal path to do so.
what is dist[I]
the lowest cost path
bellman-ford
does not work on negative cycles because it can bounce back and forth across undirected edges great for negative edges
Bellman-Ford
easy to code works on negative edges
Dijkstra goal
find the lowest-cost path from some start vertex(source) to every other vertex in the graph No negative edges!!!
negative cycle
has a negative sum of edge weights if there is no restriction of cycle and the graph is undirected w/ single negative edge will be a negative cycle
What's an adjacency list?
his consists of a list of adjacent vertices for every vertex of the graph. There are a few different ways we can represent this data structure. To represent the list of adjacent vertices, we can use a list (array), a linked list, or even a hash map or dictionary
When plementing Dijkstra's algorithm, you must take care that...
the min-priority queue is updated whenever a dist value decreases. For example, the implementation in the textbook uses a HeapAdaptablePriorityQueue, which has a method replaceKey. This method updates the min-priority queue's internal state to reflect the change in a vertex's key. The textbook also relies on a map from Vertex objects to Entry objects, which locate the vertex in the min-priority queue. It's a bit complicated.
negative cycle
there are two nodes in the graph for which there is no (finite) shortest path
restriction of cycle
use each edge once.
development of greedy method
we can use the greedy method to develop an algorithm that iteratively grows a "cloud" of vertices out of s, with the vertices entering the cloud in order of their distances froms
Unsorted array for min-PQ run time
Θ(n ^2)