Artificial intelligence

What is the basic idea of tree searching algorithms?

Offline, simulated exploration of state space Expand states by generating successors of already-explored states

What is a problem solving agent?

A problem-solving agent is a goal-based agent that will determine sequences of actions that lead to desirable states

What are the different problem types?

Deterministic, fully observable → single-state problem I sensors tell agent current state and it knows exactly what its actions do I means that it knows exactly what state it will be in after any action Deterministic, partially observable → multiple-state problem I limited access to state, but knows what its actions do I can determine set of states resulting from an action I instead of single states, agent must manipulate sets of states Stochastic, partially observable → contingency problem I don't know current state, and don't know what state will result from action I must use sensors during execution I solution is a tree with branches for contingencies I often interleave search and execution Unknown state space → exploration problem (online) I agent doesn't know what its actions will do I example: don't have a map, get from Coventry to London

How can we define a problem?

A problem is defined by four things: Initial state e.g. in Coventry Operators e.g. drive from Coventry to Warwick (Alternative notation: successor function S(x), where S(x) returns the states reachable from x using a single action.) Goal test explicit set of states, e.g. "at London", or implicit abstract property, e.g. noDirt(x) Path cost e.g. number/cost of operators in path Solution is a sequence of operators from initial state to a goal state

How can we implement search?

A state represents a physical configuration A node is a data structure comprising part of search tree Node: state, parent, children, depth, path cost 2 3 4 5 8 6 7 state node parent children depth = 3 g = 3 state 1 Nodes waiting to be expanded called the fringe Represent fringe as a queue.

How is a strategy defined? 1

A strategy is defined by selecting the order of node expansion

How do agents interact within an enviornment?

Agents are only useful in some environment Agents interact with their environment through sensors and effectors — perceptions and actions ? sensors effectors percepts actions agent environment We are primarily concerned with designing the agent program (the "?" in the diagram) to engender rational behaviour

What is SMA?

Avoids repeated states as far as memory allows Complete if enough memory to store shallowest solution Optimal if enough memory to store shallowest optimal solution If enough memory for whole tree, search is optimally efficient

How do we specify an agent's setting?

Before considering design of an agent program, we must specify its setting Consider, for example, designing an automated taxi: I Performance measure? I Environment? I Actuators (effectors)? I Sensors? Taken together, these characterise an agent's setting → PEAS.

What are the different types of agent?

Broadly speaking, there are five fundamental types of agent, with increasing generality: Simple reflex agents → condition-action rules Reflex agents with state → retain knowledge about world Goal-based agents → have representation of what states are desirable. Utility-based agents → ability to discern some useful measure (e.g. cost/quality/?) between possible means of achieving some state Learning agents → able to modify their behaviour based on their performance, or in the light of new information

What is Stochastic Hill-climbing?

Choose a random uphill move. Probability of selection determined by steepness of move Typically slower than steepest ascent approach, but better in some state landscapes

Properties of greedy searching?

Complete? No. Can get stuck in loops. Time? O(b m). A good heuristic can give significant improvement Space? O(b^m). Same as time since all nodes kept in memory Optimal? No.

How does breath first search work?

Complete? Yes (always finds solution if b is finite) Time? 1 + b + b 2 + b 3 + . . . + b d + nodes at depth d = O(b d+1) Space? Each leaf node kept in memory: O(b d−1 ) explored + O(b d ) in frontier = O(b d ) Optimal? Yes, if cost nondecreasing function of depth (e.g. operators equal cost); not optimal in general. Space is the main problem for breadth-first search. Expand shallowest unexpanded node QueueingFn = put successors at end of queue (FIFO)

What are the properties of A* searching?

Complete? Yes, unless infinitely many nodes with f ≤ f (goal) I nodes with infinite branching factor I path with finite cost, but infinite number of steps Time? Exponential in relative error in h × length of solution (i.e. number of nodes in goal contour exponential in length of solution) Space? Keeps all nodes in memory (A*'s main problem) Optimal? Yes. Cannot expand fi+1 until fi is finished.

How do genetic algorithms work?

Each individual in the population is evaluated by a fitness function Fitness function should return higher values for better states, e.g. number of non-attacking pairs of queens for 8-queens problem Fitness function determines probability of being chosen for reproduction Pairs of individuals chosen according to these probabilities (possibly with individuals below a certain threshold being discarded or culled). For each chosen pair, a random crossover point is chosen (from positions in the string representation) Offspring generated by crossing over parent strings at chosen crossover point First child gets first part of string from first parent and remainder from second parent, visa versa for second child E.g. parents "12341234" and "56785678" with crossover at position three, will give "12385678" and "56741234". Finally, each location in the newly created children is subject to a random mutation with a small probability E.g. for 8-queens, there is a small probability of taking a random queen and moving it to some random position (in its column) E.g. child "12385678" may mutate to "32385678".

How does depth first search work?

Expand deepest unexpanded node QueueingFn = insert successors at front of queue (LIFO) [b branching factor; d depth of least-cost solution; m maximum depth] Complete? No. Fails in infinite-depth spaces, spaces with loops. Modify to avoid repeated states along path ⇒ complete in finite spaces. Time? O(b m) terrible if m much larger than d. If solutions are dense may be much faster than breadth-first search Space? O(bm), i.e. linear space (store path from root to leaf and unexpanded siblings of nodes on path) Optimal? No

How does uniform informed search work?

Expand least-cost unexpanded node QueueingFn = insert in order of increasing path cost Example: route-finding (s: start, g: goal). [C∗ = cost of optimal solution, = minimum action cost, g = path cost] Complete? Yes (assuming non-zero step cost) Time? number of nodes with g ≤ cost of optimal solution, O(b 1+bC∗/c ) (if steps have equal cost b 1+bC∗/c = b d+1) NB b 1+bC∗/c can be much greater than b d+1 — explore lots of small steps before paths involving potentially useful large steps Space? also O(b 1+bC∗/c ) Optimal? Yes, if nondecreasing path cost, i.e. g(Successor(n)) ≥ g(n)

What are some properties of an environment?

Fully observable (vs. partially observable): sensors give access to complete state of environment Deterministic (vs. stochastic vs. strategic): next state of environment determined (solely) by current state and agent's actions. If deterministic except for another's actions then environment is strategic Episodic (vs. sequential): actions depend on a single "episode", subsequent episodes not dependent on previous ones Static (vs. dynamic): if the environment can not change during deliberation Discrete (vs. continuous): a limited number of percepts and actions, e.g. chess Single agent (vs. multi-agent): environment contains a single agent. Multi-agent environments can be competitive or cooperative

What are the ideal environemntal properties for a agent?

Fully observable → no need to maintain internal state to track environment Deterministic → no need to worry about uncertainty Episodic → no need to think ahead Static → no need to perceive while deliberating (less concern about deliberation time) Discrete → easier to manipulate discrete values

what are Genetic Algorithms?

Genetic algorithms (GAs) related to stochastic beam search Successor states obtained from two parents Starts with population of k randomly generated individuals, i.e. states Each individual represented as a string over a finite alphabet, e.g. string of 0s and 1s 8-queens problem might be represented by a string of 8 integers from 1 to 8 representing positions.

What are the four steps that a problem solving agent must take?

Goal formulation: identify goal given current situation Problem formulation: identify permissible actions (or operators), and states to consider Search: find sequence of actions to achieve goal Execution: perform the actions in the solution

What are the implications of using a greedy search?

Greedy search always takes the biggest bite out of remaining cost to reach goal (. . . hence the name) Does not consider whether best choice in the long run Tends to find solutions quickly, although not always optimal In practise, consideration of long-term options may take more time than it is worth Trade-off: solution quality versus speed, space, and simplicity

What is Simulated annealing?

Hill-climbing that never moves downhill is incomplete due to local maxima Combine hill-climbing with random element Simulated annealing algorithm: analogy with annealing — cooling a metal into a low energy crystalline state Instead of picking best move, picks a random successor of current node.

What are the different types of Memory bounded heuristic search?

IDA* (iterative deepening A*) I RBFS (recursive best-first search) I SMA* (simplified memory-bounded A*)

What are the properties of IDA?

IDA* is optimal and complete (for locally finite graphs, where each operator has positive cost) Space is proportional to longest path Time is dependent on the number of different values the heuristic can take (one iteration per contour) Unlike A*, no priority queue, so cost per node is less, and IDA* may actually be quicker. IDA* has problems in complex domains where heuristic can take many values: one iteration per contour (many contours = many iterations) Worst case: if A* expands N nodes, IDA* does N iterations and expands O(N 2 ) nodes Workaround: increase f-cost by fixed each iteration — reduces search cost, but returns suboptimal solutions (worse than optimal by at most ), called -admissible IDA* was the only optimal, memory-bounded heuristic algorithm for several years (c.1985).

What is A* search?

Idea: avoid expanding already expensive paths Greedy search minimises estimated cost to goal I not optimal and not complete I tends to minimise search cost Uniform-cost search minimises cost of path so far I optimal and complete I can be inefficient Solution: Combine greedy search with uniform-cost search. f (n) = g(n) + h(n) g(n) - path cost to n h(n) - estimated cost to goal from n f (n) - estimated total cost of path to goal through n This approach is complete and optimal if we restrict h h must be optimistic and never over estimate cost to goal, i.e. h(n) ≤ h 0 (n) where h 0 (n) is true cost from n Such a heuristic is called admissible A* = best-first search using f as evaluation function, with admissible h Note: hSLD(n) is admissible — distance never over estimated. Along any path, f never decreases — it is monotonic (also called consistent) Monotonic: for node n and successor n 0 , estimated cost to goal from n is no greater than step cost to n 0 plus estimated cost to goal from n 0 Most admissible heuristics are monotonic — those that are not can be made so with pathmax modification A* is optimally efficient for a given heuristic compared to other optimal algorithms that extend paths from the root: no other optimal algorithm is guaranteed to expand fewer nodes than A*

What is Local Beam Search?

Idea: keep more than 1 node in memory Local Beam Search keeps track of k states (initially randomly generated) Each iteration: generate all successors of k states If found goal, halt Otherwise, select k best states and repeat Not equivalent to k concurrent random restart hill-climbing searches — searches not independent Suffers from lack of diversity — use stochastic beam search: choose k successors at random with probability an increasing function of value.

What is best-first search?

Idea: use an evaluation function for each node (state) — estimate "desirability" of the nodes Best-first search = expand most desirable unexpanded node QueueingFn = insert successors in decreasing order of desirability Many different evaluation functions: consider two cases: greedy search and A* search Measure must incorporate some estimate of path cost from a state to closest goal state.

What is the ideal rational agent?

Ideal rational agent: does whatever action is expected to maximise performance measure on basis of percept sequence and built-in knowledge

How does graph search avoid the problems of repeated states?

If algorithm remembers visited states, can be viewed as exploring state-space graph directly. Modify Tree Search to include closed set which stores every expanded node Fringe of unexpanded nodes called open set If current node matches one in closed set then discard rather than expand.

What are learning agents?

If the agent designer has incomplete information about the environment, an agent needs to be able to learn Learning provides an agent with a degree of autonomy or independence (in the sense that its behaviour is determined by its own experience) Idea: percepts can be used to improve agent's ability to act in the future (as well as for choosing an action now) Learning results from interaction between the agent and the world, and observation by agent of its decision making.

What are genetic algorithms useful for?

If two parents are quite different, crossover can produce state that is very different from both parents GAs take large steps early in the search, smaller steps as population converges (c.f. simulated annealing) Primary advantage is the ability to crossover large blocks that have evolved independently to perform useful functions Choice of representation is fundamental — GAs work best if representation corresponds to meaningful components of solution Useful for optimisation problems (e.g. scheduling).

How does depth limited search work?

Impose cutoff on depth of path Same as depth-first with depth limit l Implementation: nodes at depth l have no successors Complete? Yes for appropriate l (if know maximum path length of solution, guaranteed to find it) Time? O(b l ) Space? O(bl) Optimal? no.

What is IDA?

Iterative deepening search good for reducing number of nodes in uninformed search Apply the same trick to A* Each iteration does depth-first search, but modify DFS — use f cost limit (not depth limit) Each iteration expands nodes within contour of current limit Algorithm "peeps over contour" to find next contour Once finished search within contour, use next f cost.

What are goal based agents?

Knowing current environment state not generally enough to choose action: need to know what the agent is trying to achieve Can combine information about environment, goals, and effects of actions to choose what to do Relatively simple if goal is achievable in single action But, typically goal achievement needs a sequence of actions — use search or planning A goal-based agent is much more flexible than a reflex agent.

What implementation do we need for a rational agent?

Lookup table suggests a notional "ideal mapping" Our aim: find a way to implement the rational agent function Need to implement rational agent function concisely Our implementation must: I be relatively efficient I exhibit autonomy if required I get as close as possible to the ideal mapping.

How does graph search work?

NB algorithm discards newly discovered path: if the new path is shorter then might miss an optimal path As for Tree Search can do breadth-first, depth-first, uniform cost, iterative deepening graph search Breadth-first and uniform cost with constant step cost are optimal graph search strategies Iterative deepening search uses depth-first expansion, and so iterative deepening graph search must check whether new path is better, and if so revise depths and path costs of that node's descendants Closed list means that depth-first and iterative deepening search no longer have linear space.

How does iterative deepening search work?

Problem with depth-limited search is picking depth limit Alternative: try all possible depth limits; 0, 1, 2, . . . Combines breadth-first and depth-first search Complete? Yes Time? bottom nodes expanded once, next level twice, . . . , root node d + 1 times (d + 1)b 0 + db1 + (d − 1)b 2 + . . . + (1)b d = O(b d ) NB: breadth-first time = 1 + b + b 2 + b 3 + . . . + b d−1 + b d Nodes at lowest level d expanded once; nodes at next level expanded twice, and so on, until root expanded d + 1 times. Plug these things together. . . (d + 1)1 + (d)b + (d − 1)b 2 + . . . + 3b d−2 + 2b d−1 + 1b d which gives (d + 1)b 0 + db1 + (d − 1)b 2 + . . . + b d = O(b d ) Space? O(bd) Optimal? Yes (if step cost = 1) IDS is a practical and useful search — generally preferred method for large search spaces and unknown solution depth (If b = 10, d = 5, IDS generates 123,450 nodes compared to 1,111,100 for BFS.)

How can we avoid repeated states?

Problem: waste time expanding already considered states Many problems have repeated states (e.g. where operators are reversible) Repeated states lead to infinite search trees — but we can prune repeated states to make tree finite For finite trees, avoiding repeated states can exponentially reduce search cost. Do not return to parent state: prevent Expand from generating a parent node as a successor Do not create cyclic paths: prevent Expand from generating an ancestor node as a successor Do not generate a previously considered state: every previous state kept in memory, checked by Expand. I Potential space complexity O(b d ) I Can implement as a hash table Best option = trade-off between computational overhead v. cost of searching cycles.

How do reflex agent with state work?

Pure reflex agents only work if the "correct" decision of what to do can be made on current percept We typically need some internal state to track the environment The update state function typically uses knowledge about how the world evolves and how actions affect the world Agent can use such information to track unseen parts of the world.

What are the properties of RBFS?

RBFS optimal and complete if h admissible Space complexity O(bd) Time complexity dependent on accuracy of h and frequency with which best path changes RBFS suffers from excessive node regeneration (as for IDA*) — too little memory is used. IDA* does not remember its history. . . so it repeats it; RBFS remembers a little. . . but still repeats itself Can modify IDA* to detect repeated states in current path, but not repetitions from other paths Alternative is SMA* (simplified memory-bounded A*) - use all available memory for search When out of space for queue drops node that looks unpromising - high f cost Keeps track in ancestor nodes of best path in forgotten subtree - i.e. how worthwhile it is to expand subtree again Only re-expands if all other paths worse than forgotten one

How does Simulated annealing work?

Random move always executed if improves situation Otherwise executed with some probability which decreases exponentially with badness of move - amount (∆E) by which evaluation worsened Temperature (T) also used to determine probability schedule determines rate at which T lowered If schedule lowers T slowly enough will find global optimum with prob → 1 Widely used in VLSI layout, airport scheduling etc

What are the properties of hill climbing?

Random restart hill-climbing — once algorithm stops making progress, restart at random location Repeat for fixed number of iterations, or, no more improvement Return best so far Not failsafe, but an improvement on a single iteration If only few local maxima, random restart hill-climbing quite good Real world state space often looks like a porcupine so may take exponential time In practise, random restart hill-climbing is often useful

How can we select a state space?

Real world extremely complex — state space must be abstracted for problem solving. Abstract state ≈ set of real states Abstract operators ≈ combination of real actions, e.g. Coventry → Warwick comprises all possible routes, detours, fuel stops etc. (must ensure validity: reliable for all real states in abstract state) Abstract solution ≈ set of real paths that are solutions in real world.

What is RBFS?

Recursive best-first search is similar to recursive depth-first search Keep track of f-cost of best alternative path from an ancestor of current node If current node exceeds this, recursion unwinds back to alternative path, replacing f-cost of nodes along the path with the best f-cost of its children Thus, RBFS remembers f-cost of best leaf in forgotten subtree — can decide whether to re-expand later.

What are reflex agents?

Reflex agents are based on a set of condition-action rules We can view these rules as summarising the notional lookup table Although simple, reflex agents can still achieve relatively complex behaviour We can also add the learning of new rules, etc.

What are the applications of search?

Route finding — have a set of locations and links between them, find the best route from a to b. Used in travel planning systems, network management etc. Travelling salesman problem — visit every city at least once starting and ending in same location. Used in travel planning and industrial robotics applications. VLSI layout — positioning of logic gates and making appropriate connections between them in silicon chip design. Navigation — route finding in the real world i.e. a continuous environment.

What is greedy search?

Simple approach: minimise estimated cost to reach goal Can often estimate cost of reaching goal from given state, although cannot determine exactly Heuristic: A function that calculates such estimates (denoted by h) Evaluation function h(n) = estimated cost of cheapest path from node n to goal Greedy search = best-first search using h as evaluation function Thus, greedy search expands node that appears to be closest to goal, according to h h can be any function such that h(n) = 0 if n is a goal Heuristic h is problem specific (e.g. consider route-finding example). Use straight-line distance from goal as our heuristic hSLD(n) = straight-line distance from n to Bucharest Can only determine hSLD if have coordinates of cites Furthermore, hSLD(n) only useful since roads tend to head in right direction, i.e. if a is closer to goal than b, according to hSLD, then road from a likely to be shorter than from b Effectiveness of hSLD situation dependent — consider mountainous road

Why does a lookup table fail for finding the ideal mapping of a rational agent?

So, in principle there is an ideal mapping of percept sequences to actions corresponding to ideal agent The simple approach is a lookup table. However, this is doomed to failure because: I size of table I time to build I agent has no autonomy.

How does bi-directional search work?

Start from both initial state and goal Motivation is that b d/2 + b d/2 is much less than b d One or both searches check before expansion whether a node is in the fringe of the other search tree; if so, have found solution. c Nathan Griffiths (University of Warwick) CS255 Artificial Intelligence Problem-Solving and Search 39 / 48 Bi-directional Search Good time complexity, since membership check is constant time using a hash function, giving O(b d/2 ) For membership check one (or both) search trees must be in memory, so space requirement also O(b d/2 ) (which is problematic) Complete and optimal (for uniform step cost) if both searches are breadth-first Problem: have to generate predecessors — tricky Problem: what if many goals states? also tricky.

What are the different search strategies?

Strategy defined by order of node expansion Evaluation along several dimensions: I completeness: always find solution (if it exists)? I optimality: always find a least-cost solution? I time complexity: number of nodes expanded F maximum branching factor b F depth of least cost solution d F maximum depth of state space m (may be ∞) I space complexity: maximum number of nodes in memory (also measured in terms of b, d, m).

What are utility-based agents?

There are typically many ways to achieve a goal — some desirable, some less so: E.g. might achieve free space on hard drive by I deleting all files > 10MB I compressing all files > 10MB not used for 7 days no files = low utility(!); all files plus space = high utility Can use a utility function to judge resulting states of world Utility function also enables a choice about which goals to achieve — select the one with highest utility If achievement is uncertain we can measure the importance of goal against likelihood of achievement.

What are hill climbing algorithms?

Two main classes of iterative improvement algorithm I hill-climbing (or greedy local search) - always try to improve state I simulated annealing - sometimes move to make things worse (temporarily) Hill-climbing: each iteration move in direction of increasing value No search tree, just keep current state and its cost If several alternatives with equal value, choose one at random.

What are the different types of uninformed search strategies?

Uninformed search only uses information from problem definition (i.e. we have no measure of the best node to expand). Five key types of uninformed search: Breadth-first Uniform-cost Depth-first Depth-limited Iterative deepening

What is rational action?

Without loss of generality, "goals" can be specified by some performance measure defining a numerical value for a given environment history Rational action is whichever action maximises the expected value of the performance measure given the percept sequence to date (i.e. doing the right thing) Previous perceptions are typically important

What is an agent specified?

agent is specified by the agent function mapping percept sequences to actions f : P → A

What are the two types of problem solving agents?

o types of problem-solving: offline — complete knowledge of problem and solution online — involves acting without complete knowledge of problem and solution.

What are some common misconceptions abuot rational agents?

rational 6= omniscient I An omniscient agent would know the actual outcome of its actions — agents are rarely omniscient since unexpected situations occur in dynamic environments; a rational agent needs only to do its best given the current percepts rational 6= clairvoyant I An agent is not expected to foresee future changes in its environment rational 6= successful I Rational action is defined in terms of expected value, rather than actual value; a failed action can still be rational.

What are the components of a learning agent?

ypical learning agent has four conceptual components: Learning element: responsible for making improvements — takes knowledge of the performance element and feedback on agent's success, and suggests improvements to the performance element Performance element: responsible for acting — takes percepts and returns actions (in non-learning agents, this is often considered to be the whole agent) Design of learning element dependent on design of performance element — learning algorithms for improving logical inference different to those for improving decision-theoretic agents. Critic: tells the learning element how well the agent is doing, using some performance standard Performance standard should be a fixed measure, external to the agent Problem generator: responsible for suggesting actions in pursuit of new (and informative) experiences. Without the problem generator the performance element would always do what it thinks is best. But performing a "suboptimal" action may lead to discovery of much better actions in future (e.g. route finding)