Machine Learning Mid-Term

Summarize the multi-layer Perceptron learning algorithm: The 1)___ layer applies the given information to the first weight vector. The 2)___ ___ (or sigmoid) determines whether the neurons in the first layer should fire. The neurons in the hidden layer receive input from the first layer of neurons, compute a weighted sum, and decide whether to fire towards the output layer. The 3)___ is computed as the sum-of-squares difference between the network outputs and the targets. This sum propagates 4)___ to update the weights in reverse order of their computation.

1) Input. 2) Activation function. 3) Error. 4) Backwards.

Fill in the numbered blanks: A multi-layer Perceptron (when viewed "going forward") consists of a/an 1)___ layer, followed by multiple 2)___ layers, and finally a/an 3)___ layer.

1) Input. 2) Hidden. 3) Output.

What are the four main approaches to machine learning?

1. Supervised 2. Unsupervised 3. Reinforcement 4. Evolutionary

What does a linear discriminant do?

A linear discriminant partitions the data of a classification problem into its own classes.

How does the artificial neuron (McColluch-Pitts) learn?

A neuron compares the difference between the expected output and the actual output for given training data, and alters the weights to improve the performance of that neuron for future inputs.

A biological neural network holds billions of neurons, along with trillions of connections for communication. How does the artificial neural network model this behavior?

Artificial neural networks loosely model biological neural networks. Inputs are received, neurons compute the sum of their weighted signals to determine whether to fire, and these fired neurons signal other neurons.

A multi-layer Perceptron computes the error of the hidden-layer neurons using ___

Back-propagation of Error.

True/False: The action space is the set of possible inputs, and the state space is the set of possible outputs.

False. State space -> Inputs. Action space -> Outputs.

True/False: AI is a subset of Machine Learning.

False. Machine Learning is just one area of Artificial Intelligence.

What is generalization, and why is it important?

Generalization is a model's ability to give correct outputs for inputs it has not trained on. Models cannot train all possible inputs (there are far too many), so good algorithms can learn to give correct answers on real-world data.

List the three most common methods of choosing an action for a given policy.

Greedy, epsilon-greedy, and soft-max.

Reinforcement learning, at its most basic level, searches through all possible ___/___, and chooses the one with the best ___.

Inputs/outputs, reward.

What are the differences between the Q-learning algorithm and the Sarsa algorithm?

Q-learning: chooses the highest-value future action when updating (known as off-policy), which "streamlines" the agent to pursue the most optimal solution. Sarsa: computes the reward function for future actions (known as on-policy), which can bias the solution against actions with large negative rewards.

(Symbolic/Subsymbolic) Which uses mathematical symbols, as opposed to logical manipulation? Hint: Machine Learning is this.

Subsymbolic.

Classification/Regression problems are typically solved using which of the four machine learning approaches?

Supervised.

How does the Perceptron model a discriminant function?

The Perceptron attempts to find a straight line (plane in 3D, hyperplane in 4D, etc.) that divides the examples where each neuron fires from those where it does not.

A biological neuron receives electrical pulses from other neurons via axons and synapses. These pulses determine whether the neuron will fire its pulse. How does the artifical neuron (McColluch-Pitts) model this behavior?

The artificial neuron receives weighted input signals, computes the sum of those weighted signals, and fires if and only if that sum is greater than a certain threshold.

Where are the reward values stored in the Q-learning algorithm?

The reward values are stored in an S x A Q-table, and updated after each action is taken.

What is Machine Learning?

The study of techniques for enabling a computer to change its actions so as to improve its performance

What is the formula for updating weights in the single-layer Perceptron?

w <- w - ╖(y - t)*x w: weight ╖: learning rate y: actual output t: target (expected) output x: input