Neural Networks
Perceptron
A mathematical Neuron that is similar to logistic regression.
Overfitting
A small value for the objective function, when calculated on training data, need not imply a small value for the function on validation data.
Objective Function
How Predictions are Compared to the Actual Values of the target.
Neural Networks are used for
Classification, Regression, Time Series analysis, and Clustering.
Validation Data
Data that has been cleaned and is both correct and useful
Advantages of Neural Network
Easy to implement, Can be trained to address very complex problems, Can handle non-linear and non-normal data.
Neural Networks Combine
Human generalization from experience and Computer following instructions repetitively
Example of an Objective Function
Mean Squared Error
Neural Network Process
Model "learns" the structure of the data from a representative sample, Uses a series of weights and hidden neurons to detect complex relationships.
Neural Networks
Mysterious and Powerful Modeling Technique, uses machine learning or Perceptron
Disadvantages of Neural Network
Neural networks are "black boxes", Training can be slow, May require very large training data set.
maximum validation profit
The neural network training iteration where the final parameter estimates for the model are taken.
Universal Approximation Theorem
The standard multilayer feed-forward network with a single hidden layer, which contains finite number of hidden neurons, is a universal approximator among continuous functions on compact subsets of Rn, under mild assumptions on the activation function.
network diagram
The structure of a multi-layer perceptron that lends itself to a graphical representation
Overall Average Profit
Used to prevent overfitting on a neural network, computed on validation data.
gradient decent
Weights are iteratively adjusted to descend down the error function. Usually employing a back propagation algorithm.
Perceptron
a machine learning algorithm that helps provide classified outcomes for computing.
Neural Networks Work with
complicated, noisy, and/or imprecise data
Hidden Layer
connects to a final layer called the output layer.
Input Layer
connects to a layer of neurons called a hidden layer,
Neural Networks are Useful
data mining and decision-support applications
Artificial Neural Networks
indicates that neural networks are a (poor) approximation of real, biological neural networks
Multi-layer perceptron models
originally inspired by neurophysiology and the interconnections between neurons. The basic model form arranges neurons in layers.
At Convergence
the model is likely to be highly overfit and the values of the objective function computed on training and validation data may be quite different.
