Deep Learning Notes

What does the analogy "AI is the new electricity" refer to?

Similar to electricity starting about 100 years ago, AI is transforming multiple industries.

Consider the following code: a = np.random.randn(3, 3) b = np.random.randn(3, 1) c = a * b What will be c?

This will invoke broadcasting, so b is copied three times to become (3,3), and ∗ is an element-wise product so c.shape = (3, 3).

Which of these are reasons for Deep Learning recently taking off? (Check the two options that apply.)

We have access to a lot more computational power, We have access to a lot more data, Deep learning has resulted in significant improvements in important applications such as online advertising, speech recognition, and image recognition.

Consider the two following random arrays "a" and "b": a = np.random.randn(2, 3) # a.shape = (2, 3) b = np.random.randn(2, 1) # b.shape = (2, 1) c = a + b What will be the shape of "c"?

b (column vector) is copied 3 times so that it can be summed to each column of a. Therefore, c.shape = (2, 3).

Consider the following code snippet: # a.shape = (3,4) # b.shape = (4,1) for i in range(3): for j in range(4): c[i][j] = a[i][j] + b[j] How do you vectorize this?

c = a + b.T

Recall that np.dot(a,b) performs a matrix multiplication on a and b, whereas a*b performs an element-wise multiplication. Consider the two following random arrays "a" and "b": a = np.random.randn(12288, 150) # a.shape = (12288, 150) b = np.random.randn(150, 45) # b.shape = (150, 45) c = np.dot(a, b) What is the shape of c?

c.shape = (12288, 45), this is a simple matrix multiplication example.

When an experienced deep learning engineer works on a new problem, they

can't usually use insight from previous problems to train a good model on the first try, without needing to iterate multiple times through different models.

Images for cat recognition is an example of "structured" data, because it is represented as a structured array in a computer.

false

A demographic dataset with statistics on different cities' population, GDP per capita, economic growth is an example of "unstructured" data because it contains data coming from different sources

faslse

Suppose img is a (32,32,3) array, representing a 32x32 image with 3 color channels red, green and blue. How do you reshape this into a column vector?

x = img.reshape((32 * 32 * 3, 1))

In this diagram which we hand-drew in lecture, what do the horizontal axis (x-axis) and vertical axis (y-axis) represent? x-axis is the amount of data y-axis (vertical axis) is the performance of the algorithm.

x-axis is the amount of data y-axis (vertical axis) is the performance of the algorithm.

What does a neuron compute?

A neuron computes a linear function (z = Wx + b) followed by an activation function

Suppose you have n_x input features per example. Recall that X=[x^(1), x^(2)...x^(m)]. What is the dimension of X? (n_x, m) Note: A stupid way to validate this is use the formula Z^(l) = W^(l)A^(l) when l = 1, then we have

A^(1) = X X.shape = (n_x, m) Z^(1).shape = (n^(1), m) W^(1).shape = (n^(1), n_x)

Why is an RNN (Recurrent Neural Network) used for machine translation, say translating English to French? (Check all that apply.)

It can be trained as a supervised learning problem. It is applicable when the input/output is a sequence (e.g., a sequence of words).

Recall this diagram of iterating over different ML ideas. Which of the statements below are true?

Being able to try out ideas quickly allows deep learning engineers to iterate more quickly. Faster computation can help speed up how long a team takes to iterate to a good idea. Recent progress in deep learning algorithms has allowed us to train good models faster (even without changing the CPU/GPU hardware).

Assuming the trends described in the previous question's figure are accurate (and hoping you got the axis labels right), which of the following are true? (Check all that apply.)

Increasing the training set size generally does not hurt an algorithm's performance, and it may help significantly. Increasing the size of a neural network generally does not hurt an algorithm's performance, and it may help significantly.

Consider the two following random arrays "a" and "b": a = np.random.randn(4, 3) # a.shape = (4, 3) b = np.random.randn(3, 2) # b.shape = (3, 2) c = a * b What will be the shape of "c"?

"*" operator indicates element-wise multiplication. Element-wise multiplication requires same dimension between two matrices. It's going to be an error.

Consider the following computation graph. J = u + v - w = a * b + a * c - (b + c) = a * (b + c) - (b + c) = (a - 1) * (b + c)

(a - 1) * (b + c)