Ch. 04

Incremental Belief-State Search

an algorithm that builds up the solution one physical state at a time.

Contingency Plan

also known as strategy, this a solution to a problem that specifies what to do depending on what percepts are received.

Recursive State Estimator

a function that computes the new belief state from the previous one rather than by examining the entire percept sequence. b' = UPDATE(PREDICT(b, a), o)

Culling

a method of selection in which all individuals below a given threshold are discarded. Can be shown to converge faster than the random version

Sensorless (Conformant) Problem

a problem in which the agent's percepts provide no information at all.

Optimization Problems

a problem in which the aim is to find the best state according to an objective function.

Mutation

a random alteration to a single digit in the individual's string after crossover.

Crossover Point

a random position in a pair of individuals' strings chosen after selection that determines the composition of their children's strings.

Convex Set

a set of points S is convex if the line joining any two points in S is also contained in S. A convex function is one for which the space "above" it forms a convex set; by definition, convex functions have no local (as opposed to global) minima.

Population

a set of states. In Beam Searches and Genetic Algorithms, the population is generated randomly.

Individual

a single state of a population. In GAs, an individual is represented as a string over a finite alphabet--most commonly, a string of 0s and 1s.

Step Size

a small constant (alpha) used to update the current state in a steepest-ascent hill climbing algorithm. x <- x + (alpha)(del)f(x)

Schema

a substring in which some of the positions can be left unspecified.

Online Search

a type of search algorithm that interleaves computation and action: first it takes an action, then it observes the environment and computes the next action.

Offline Search

a type of search algorithm that makes no action until a sequence of actions yielding a solution is found.

Gradient of the Objective Function

a vector (del)f that gives the magnitude and direction of the steepest slope.

Local Search

algorithms that operate using a single current node (rather than multiple paths) and generally move only to neighbors of that node. Two advantages: (1) they use very little memory (usually a constant amount) (2) they can often find reasonable solutions in large or infinite (continuous) state spaces for which systematic algorithms are unsuitable

Constrained Optimization

an optimization problem in which solutions must satisfy some hard constraints on the values of the variables.

Competitive Ratio

comparing the total path cost of the path that the agent actually travels to the actual shortest path (or shortest complete exploration). This ratio should be as small as possible.

Linear Programming

describes a problem in which constraints must be linear inequalities forming a convex set and the objective function is also linear. The time complexity of linear programming is polynomial in the number of variables.

Convex Optimization

describes a problem which allows the constraint region to be any convex region and the objective to be any function that is convex within the constraint region.

Dead-End State

describes a state from which no goal state is reachable.

Safely Explorable

describes a state space in which some goal state is reachable from every reachable state. State spaces with reversible actions can be viewed as undirected graphs and are clearly safely explorable.

Irreversible

describes an action that leads to a state from which no action leads back to the previous state

Discretize

focusing on a subset of the state space.

State-Space Landscape

has both "location" (defined by the state) and "elevation" (defined by the value of the heuristic cost function or objective function). If elevation corresponds to cost, then the aim is to find the lowest valley--a global minimum If elevation corresponds to an objective function, then the aim is to find the highest peak--a global maximum

Cyclic Solution

in environments where actions are nondeterministic, this solution method repeats a step until the subsequent state is what is expected.

Empirical Gradient Search

same as steepest-ascent hill climbing in a discretized version of the state space.

Belief State

the agent's current belief about the possible physical states it might be in, given the sequence of actions and percepts up to that point.

Localization

the process of determining one's location given a map of the world and a sequence of percepts and actions.

Prediction

the process of generating a new belief state after an action.

Monitoring (also called Filtering and State Estimation)

the process of maintaining one's belief state.