Neural Networks Overview

Transfer learning is a machine learning technique in which ____

a model trained on one task is repurposed on a second, related task

Each neuron receives ____, which is then added with a static bias value that is unique to each neuron layer.

a multiplied version of inputs and random weights

ML is a framework to ____ to get better at ____

automate statistical models, making predictions

What is another term for DNN?

deep net

The vectors of weights and biases are called ____ and represent ____

filters, particular features of the input (e.g., a particular shape)

Convolutional neural networks leverage principles from _____, specifically ____, to identify patterns within an image.

linear algebra, matrix multiplication

Perceptron is a ____ Machine Learning algorithm used for ____ learning for various ____ classifiers

linear, supervised, binary

The "full connectivity" of multilayer perceptrons makes them prone to ____

overfitting

Artificial Intelligence is a machine learning technique that ____

teaches computers and devices logical functioning

In a fully connected layer, the receptive field is ____

the entire previous layer

T/F Perceptron neural networks are capable of separating data non-linearly

FALSE

T/F Like a true perceptron, MLPs perform only binary classification.

FALSE - an MLP neuron is free to perform either regression or classification

T/F In CNN, multiple neurons cannot share the same filter

FALSE. In CNN, many neurons can share the same filter

In a Feed Forward Neural Network, there are no ____ or ____

cycles, loops

Name the 9 neural network models.

1. Perceptron 2. Feed Forward Neural Network 3. Multilayer Perceptron 4. Convolutional Neural Network 5. Radial Basis Functional Neural Network 6. Recurrent Neural Network 7. LSTM - Long Short-Term Memory 8. Sequence to Sequence Models 9. Modular Neural Network

What are the purposes of the hidden layer?

1. store information regarding the input's importance 2. make associations between the importance of combinations of inputs

A fully connected layer for an image size of 100 x 100 has ____ weights for each neuron in its second layer.

10,000

How many types of neural network models are there?

9

In a neural network, what is a neuron?

A node

In a CNN, at what point does the image become abstracted to a feature map?

After passing through a convolutional layer

What is ImageNet?

An image database with over 14 million images for the purposes of advancing computer vision and deep learning research.

Deep Learning is a subset of ____

Artificial Intelligence

What does ANN stand for?

Artificial Neural Network

___ utilize the hidden layer as a place to store and evaluate how significant one of the inputs is to the output.

Artificial Neural Networks

What is the biological inspiration for the term 'neural networks'?

Biological neurons activate under certain circumstances, resulting in a related action performed by the body.

SIANN is another term for ____

CNN

____ neurons process data only for their receptive field

Convolutional

CNN stands for...

Convolutional Neural Network

DNN stands for...

Deep Neural Network

What hardware is necessary when training a model using CNN?

GPU - graphics processing unit. This is a specialized electronic circuit.

How does a Feed Forward Neural Network differ from a Recurrent Neural Network?

In a Feed Forward Neural Network, information only moves in one direction - forward. Information never moves backward.

What is the image database with over 14 million images for the purposes of advancing computer vision and deep learning research?

ImageNet

Perceptron is also known as a ____

Linear Binary Classifier

MLP stands for ____

Multilayer Perceptron

What happens if a node is NOT activated in a neural network?

No data is sent from that node to the next layer.

What are the pros and cons of using DNN rather than a neural network with a single hidden layer?

PRO: performance is improved CON: explainability is reduced

____ reduce the dimensions of data by combining the outputs of neuron clusters at one layer into a single neuron in the next layer.

Pooling layers

Compared to other image classification algorithms, how much pre-processing is necessary for CNNs?

Relatively little - the network learns to optimize the filters (or kernels) through automated learning

SIANN stands for ____ or ____

Shift Invariant Artificial Neural Network, Space Invariant Artificial Neural Network

CNNs are also known as ____ or ____

Shift Invariant Artificial Neural Networks, Space Invariant Artificial Neural Networks (SIANNs)

What is the simplest kind of neural network?

Single-layer perceptron network

T/F Feed Forward Neural Networks are capable of processing non-linear data

TRUE

What happens when the output of a node is above the specified threshold value?

That node is activated

What aspect of multilayer perceptrons makes them prone to overfitting?

The "full connectivity"

What is happening when a model 'learns'? (in layman's terms)

The model takes all its bad predictions and tweaks the weights inside the model so that it makes fewer mistakes.

What is the black box problem with regard to deep nets?

The model's explainability is reduced.

What makes a neural network 'deep'? (DNNs)

The neural network must have some level of complexity - usually at least two layers.

What must happen for a node to be activated in a neural network?

The node's output must be above its threshold value.

What is a drawback of CNNs?

They can be computationally demanding, requiring graphical processing units (GPUs) to train models.

What is the benefit of many neurons sharing the same filter in CNN?

This reduces the memory footprint (i.e. the process uses less memory)

What is the purpose of regularization?

To prevent overfitting

T/F Most convolutional neural networks are not invariant to translation.

True - This is due to the downsampling operation they apply on the input.

In a convolutional layer, each neuron receives input from only ____ called the neuron's receptive field.

a restricted area of the previous layer

In fitting a neural network, backpropagation computes the gradient of the loss function with respect to the weights of the network for ____

a single input-output example

Pooling layers reduce the dimensions of data by combining the outputs of neuron clusters at one layer into ____

a single neuron in the next layer

What is another term for feature map?

activation map

Neural networks are comprised of node layers, containing ____

an input layer, one or more hidden layers, an output layer

Each node in the hidden layer makes both ____ and grades ____

associations, importance of the input to determining the output

BP stands for ____

back-propagation

Convolutional neural networks are commonly used for...

classification and computer vision tasks

Perceptron is generally used to ____

classify the data into two parts

CNNs use ____ in place of general matrix multiplication in at least one of their layers

convolution

Artificial Intelligence is the application of tools and techniques to ____ that ____

design an algorithm, helps machines to solve problems

Pooling layers reduce the ____ by combining the outputs of neuron clusters at one layer into a single neuron in the next layer.

dimensions of data

A fully connected layer for an image size of 100 x 100 has 10,000 weights for ____

each neuron in its second layer.

A highly sensitive test means that there are few ____

false negatives

A test with high specificity means that there are few ____

false positives

Convolution reduces the number of ____, allowing the network to be deeper.

free parameters

Multilayer perceptrons usually indicate ____ connected networks, which means...

fully, each neuron in one layer is connected to all neurons in the next layer

CNNs use convolution in place of ____ in at least one of their layers

general matrix multiplication

In fitting a neural network, backpropagation computes the ______ with respect to the weights of the network for a single input-output example.

gradient of the loss function

ANNs utilize the ____ as a place to store and evaluate how significant one of the inputs is to the output.

hidden layer

The deep net has multiple ____

hidden layers

ANNs utilize the hidden layer as a place to store and evaluate ____

how significant one of the inputs is to the output

Deep Learning relies on ____ methods to teach machines to imitate human intelligence

iterative

Recurrent neural networks are commonly used for...

language processing and speech recognition

CNNs are regularized versions of ____

multilayer perceptrons

In a neural network, a node is also known as a ____ or a ____

neuron, Perceptron

Each ____ in the hidden layer makes both associations and grades importance of the input to determining the output.

node

Neural networks are comprised of ____, containing an input layer, one or more hidden layers, and an output layer

node layers

On Caffe Model Zoo, ____ are shared.

pre-trained models

Each convolutional neuron processes data only for its ____

receptive field

In a convolutional layer, each neuron receives input from only a restricted area of the previous layer called the neuron's ____

receptive field

CNNs are ____ versions of multilayer perceptrons

regularized

Each neuron receives a multiplied version of inputs and random weights, which is then added with a ____

static bias value that is unique to each neuron layer.

The hidden layer stores information regarding ____ and makes associations between ____

the input's importance, the importance of combinations of inputs

Pooling layers reduce the dimensions of data by combining ____ into a single neuron in the next layer.

the outputs of neuron clusters at one layer

CNNs are often compared to ____ in living organisms.

the way the brain achieves visual processing

Once a node reaches the activation value, it calculates the entire amount and modifies it according to the ____

transfer function

High specificity is the ability to detect ____

true negatives

High sensitivity is the ability to detect ____

true positives

In mathematics, convolution is a mathematical operation on ____ that produces ____ that expresses ____

two functions, a third function, how the shape of one is modified by the other

In neural networks, each node has an associated ____ and ____

weight, threshold

In fitting a neural network, backpropagation computes the gradient of the loss function with respect to the ____ for a single input-output example.

weights of the network