Neural Networks

Perceptron

A mathematical Neuron that is similar to logistic regression.

Overfitting

A small value for the objective function, when calculated on training data, need not imply a small value for the function on validation data.

Objective Function

How Predictions are Compared to the Actual Values of the target.

Neural Networks are used for

Classification, Regression, Time Series analysis, and Clustering.

Validation Data

Data that has been cleaned and is both correct and useful

Advantages of Neural Network

Easy to implement, Can be trained to address very complex problems, Can handle non-linear and non-normal data.

Neural Networks Combine

Human generalization from experience and Computer following instructions repetitively

Example of an Objective Function

Mean Squared Error

Neural Network Process

Model "learns" the structure of the data from a representative sample, Uses a series of weights and hidden neurons to detect complex relationships.

Neural Networks

Mysterious and Powerful Modeling Technique, uses machine learning or Perceptron

Disadvantages of Neural Network

Neural networks are "black boxes", Training can be slow, May require very large training data set.

maximum validation profit

The neural network training iteration where the final parameter estimates for the model are taken.

Universal Approximation Theorem

The standard multilayer feed-forward network with a single hidden layer, which contains finite number of hidden neurons, is a universal approximator among continuous functions on compact subsets of Rn, under mild assumptions on the activation function.

network diagram

The structure of a multi-layer perceptron that lends itself to a graphical representation

Overall Average Profit

Used to prevent overfitting on a neural network, computed on validation data.

gradient decent

Weights are iteratively adjusted to descend down the error function. Usually employing a back propagation algorithm.

Perceptron

a machine learning algorithm that helps provide classified outcomes for computing.

Neural Networks Work with

complicated, noisy, and/or imprecise data

Hidden Layer

connects to a final layer called the output layer.

Input Layer

connects to a layer of neurons called a hidden layer,

Neural Networks are Useful

data mining and decision-support applications

Artificial Neural Networks

indicates that neural networks are a (poor) approximation of real, biological neural networks

Multi-layer perceptron models

originally inspired by neurophysiology and the interconnections between neurons. The basic model form arranges neurons in layers.

At Convergence

the model is likely to be highly overfit and the values of the objective function computed on training and validation data may be quite different.