Neural networks and Deep learning
Define deep learning
A blanket term for machine learning techniques that attempt to find interesting properties of data through multiple 'layered' graphs, usually with hidden layers(corresponding to unseen but useful features)
neural network
A neural network is a supervised machine learning algorithm that is usually graphicallly represented using nodes and branches. There a number of inputs and output nodes, and sometimes intermediate nodes with weights that characterise how the incoming values (depends on the branch) contribute to the weight at a given node( output or intermediate).
How to determine the WEIGHTS on the branches (machine learning step of the neural network)
BACK PROPAGATION ALGORITHM - feed forward, calculate the error, and then propagates errors back through the network and adjusts weights. (use gradient descent method over the partial derivatives of the error function(with respect to various weights) to estimate the optimal weight iteratively)
Training phase of neural network
Try to determine what the weights on the branches are so as to minimise the calculated ERROR function between our predicted value at the output node and the actual value across the data.
Once the weights are determined, what is done
make predictions for the test instances by, 1. Multiply the input nodes by the corresponding weights to find the PREACTIVATION values of the hidden nodes. 2. Applying the activation function at the hidden nodes 3. iterating through each hidden layer 4. Finally applying the activation function on the output node to observe the predicted value.
why is deep learning so popular now?
It gives good results - improved processing power - availability of huge data sets that allow for interesting results. - more sophisticated algorithms
conclusion, how does it work (simple)
- edge weights determine how the incoming values contribute to node weight - perform a weighted sum of the inputs and then apply an ACTIVATION function, to determine the activation that feeds forward
