Neural Networks Overview
Transfer learning is a machine learning technique in which ____
a model trained on one task is repurposed on a second, related task
Each neuron receives ____, which is then added with a static bias value that is unique to each neuron layer.
a multiplied version of inputs and random weights
ML is a framework to ____ to get better at ____
automate statistical models, making predictions
What is another term for DNN?
deep net
The vectors of weights and biases are called ____ and represent ____
filters, particular features of the input (e.g., a particular shape)
Convolutional neural networks leverage principles from _____, specifically ____, to identify patterns within an image.
linear algebra, matrix multiplication
Perceptron is a ____ Machine Learning algorithm used for ____ learning for various ____ classifiers
linear, supervised, binary
The "full connectivity" of multilayer perceptrons makes them prone to ____
overfitting
Artificial Intelligence is a machine learning technique that ____
teaches computers and devices logical functioning
In a fully connected layer, the receptive field is ____
the entire previous layer
T/F Perceptron neural networks are capable of separating data non-linearly
FALSE
T/F Like a true perceptron, MLPs perform only binary classification.
FALSE - an MLP neuron is free to perform either regression or classification
T/F In CNN, multiple neurons cannot share the same filter
FALSE. In CNN, many neurons can share the same filter
In a Feed Forward Neural Network, there are no ____ or ____
cycles, loops
Name the 9 neural network models.
1. Perceptron 2. Feed Forward Neural Network 3. Multilayer Perceptron 4. Convolutional Neural Network 5. Radial Basis Functional Neural Network 6. Recurrent Neural Network 7. LSTM - Long Short-Term Memory 8. Sequence to Sequence Models 9. Modular Neural Network
What are the purposes of the hidden layer?
1. store information regarding the input's importance 2. make associations between the importance of combinations of inputs
A fully connected layer for an image size of 100 x 100 has ____ weights for each neuron in its second layer.
10,000
How many types of neural network models are there?
9
In a neural network, what is a neuron?
A node
In a CNN, at what point does the image become abstracted to a feature map?
After passing through a convolutional layer
What is ImageNet?
An image database with over 14 million images for the purposes of advancing computer vision and deep learning research.
Deep Learning is a subset of ____
Artificial Intelligence
What does ANN stand for?
Artificial Neural Network
___ utilize the hidden layer as a place to store and evaluate how significant one of the inputs is to the output.
Artificial Neural Networks
What is the biological inspiration for the term 'neural networks'?
Biological neurons activate under certain circumstances, resulting in a related action performed by the body.
SIANN is another term for ____
CNN
____ neurons process data only for their receptive field
Convolutional
CNN stands for...
Convolutional Neural Network
DNN stands for...
Deep Neural Network
What hardware is necessary when training a model using CNN?
GPU - graphics processing unit. This is a specialized electronic circuit.
How does a Feed Forward Neural Network differ from a Recurrent Neural Network?
In a Feed Forward Neural Network, information only moves in one direction - forward. Information never moves backward.
What is the image database with over 14 million images for the purposes of advancing computer vision and deep learning research?
ImageNet
Perceptron is also known as a ____
Linear Binary Classifier
MLP stands for ____
Multilayer Perceptron
What happens if a node is NOT activated in a neural network?
No data is sent from that node to the next layer.
What are the pros and cons of using DNN rather than a neural network with a single hidden layer?
PRO: performance is improved CON: explainability is reduced
____ reduce the dimensions of data by combining the outputs of neuron clusters at one layer into a single neuron in the next layer.
Pooling layers
Compared to other image classification algorithms, how much pre-processing is necessary for CNNs?
Relatively little - the network learns to optimize the filters (or kernels) through automated learning
SIANN stands for ____ or ____
Shift Invariant Artificial Neural Network, Space Invariant Artificial Neural Network
CNNs are also known as ____ or ____
Shift Invariant Artificial Neural Networks, Space Invariant Artificial Neural Networks (SIANNs)
What is the simplest kind of neural network?
Single-layer perceptron network
T/F Feed Forward Neural Networks are capable of processing non-linear data
TRUE
What happens when the output of a node is above the specified threshold value?
That node is activated
What aspect of multilayer perceptrons makes them prone to overfitting?
The "full connectivity"
What is happening when a model 'learns'? (in layman's terms)
The model takes all its bad predictions and tweaks the weights inside the model so that it makes fewer mistakes.
What is the black box problem with regard to deep nets?
The model's explainability is reduced.
What makes a neural network 'deep'? (DNNs)
The neural network must have some level of complexity - usually at least two layers.
What must happen for a node to be activated in a neural network?
The node's output must be above its threshold value.
What is a drawback of CNNs?
They can be computationally demanding, requiring graphical processing units (GPUs) to train models.
What is the benefit of many neurons sharing the same filter in CNN?
This reduces the memory footprint (i.e. the process uses less memory)
What is the purpose of regularization?
To prevent overfitting
T/F Most convolutional neural networks are not invariant to translation.
True - This is due to the downsampling operation they apply on the input.
In a convolutional layer, each neuron receives input from only ____ called the neuron's receptive field.
a restricted area of the previous layer
In fitting a neural network, backpropagation computes the gradient of the loss function with respect to the weights of the network for ____
a single input-output example
Pooling layers reduce the dimensions of data by combining the outputs of neuron clusters at one layer into ____
a single neuron in the next layer
What is another term for feature map?
activation map
Neural networks are comprised of node layers, containing ____
an input layer, one or more hidden layers, an output layer
Each node in the hidden layer makes both ____ and grades ____
associations, importance of the input to determining the output
BP stands for ____
back-propagation
Convolutional neural networks are commonly used for...
classification and computer vision tasks
Perceptron is generally used to ____
classify the data into two parts
CNNs use ____ in place of general matrix multiplication in at least one of their layers
convolution
Artificial Intelligence is the application of tools and techniques to ____ that ____
design an algorithm, helps machines to solve problems
Pooling layers reduce the ____ by combining the outputs of neuron clusters at one layer into a single neuron in the next layer.
dimensions of data
A fully connected layer for an image size of 100 x 100 has 10,000 weights for ____
each neuron in its second layer.
A highly sensitive test means that there are few ____
false negatives
A test with high specificity means that there are few ____
false positives
Convolution reduces the number of ____, allowing the network to be deeper.
free parameters
Multilayer perceptrons usually indicate ____ connected networks, which means...
fully, each neuron in one layer is connected to all neurons in the next layer
CNNs use convolution in place of ____ in at least one of their layers
general matrix multiplication
In fitting a neural network, backpropagation computes the ______ with respect to the weights of the network for a single input-output example.
gradient of the loss function
ANNs utilize the ____ as a place to store and evaluate how significant one of the inputs is to the output.
hidden layer
The deep net has multiple ____
hidden layers
ANNs utilize the hidden layer as a place to store and evaluate ____
how significant one of the inputs is to the output
Deep Learning relies on ____ methods to teach machines to imitate human intelligence
iterative
Recurrent neural networks are commonly used for...
language processing and speech recognition
CNNs are regularized versions of ____
multilayer perceptrons
In a neural network, a node is also known as a ____ or a ____
neuron, Perceptron
Each ____ in the hidden layer makes both associations and grades importance of the input to determining the output.
node
Neural networks are comprised of ____, containing an input layer, one or more hidden layers, and an output layer
node layers
On Caffe Model Zoo, ____ are shared.
pre-trained models
Each convolutional neuron processes data only for its ____
receptive field
In a convolutional layer, each neuron receives input from only a restricted area of the previous layer called the neuron's ____
receptive field
CNNs are ____ versions of multilayer perceptrons
regularized
Each neuron receives a multiplied version of inputs and random weights, which is then added with a ____
static bias value that is unique to each neuron layer.
The hidden layer stores information regarding ____ and makes associations between ____
the input's importance, the importance of combinations of inputs
Pooling layers reduce the dimensions of data by combining ____ into a single neuron in the next layer.
the outputs of neuron clusters at one layer
CNNs are often compared to ____ in living organisms.
the way the brain achieves visual processing
Once a node reaches the activation value, it calculates the entire amount and modifies it according to the ____
transfer function
High specificity is the ability to detect ____
true negatives
High sensitivity is the ability to detect ____
true positives
In mathematics, convolution is a mathematical operation on ____ that produces ____ that expresses ____
two functions, a third function, how the shape of one is modified by the other
In neural networks, each node has an associated ____ and ____
weight, threshold
In fitting a neural network, backpropagation computes the gradient of the loss function with respect to the ____ for a single input-output example.
weights of the network