Loss Functions — determines how good or how bad the model’s prediction was

• ‘mean_squared_error’ — measures the average of the squares of the errors—that is, the average squared difference between the estimated values and the actual value
• ‘sparse_categorical_crossentropy’ — measure the dissimilarity between the distribution of observed class labels and the predicted probabilities of class membership. Categorical refers to the possibility of having more than two classes (instead of binary, which refers to two classes). Sparse refers to using a single integer from zero to the number of classes minus one
• ‘binary_crossentropy’ —
• ‘categorical_crossentropy’ —
• Huber — less sensitive to outliers than squared error loss functions

Optimizers — makes a new prediction based on the data from the loss function

• stg (stochastic gradient descent) —
• adam —
• RMSProp(lr=0.001) — if gradient descent is not working (if J-theta is increasing instead of decreasing), then use smaller learning rate, but if it’s too small, then it’ll take a very long time to converge! Here’ s a video by Andrew Ng to demonstrate how to choose the proper learning rate.

Layers — defines the output of that node given an input or set of inputs

• Dense — the most common, regular, connected layer: does the below operation on the input and return the output
• Conv2D — 2-dimensional convolution layer (e.g. spatial convolution over images) to filter the most important features that determine the output
• MaxPooling2D — max pooling operation (feature emphasis compression) for 2-dimensional spatial data
• Conv1D —
• SimpleRNN —
• Lambda —

Activation Functions —

• tf.nn.relu — effectively means “If X>0 return X, else return 0” — so what it does it it only passes values 0 or greater to the next layer in the network
• tf.nn.softmax — measures probabilities of all possibilities, all of which sum up to one… takes a set of values, and effectively picks the biggest one, so, for example, if the output of the last layer looks like [0.1, 0.1, 0.05, 0.1, 9.5, 0.1, 0.05, 0.05, 0.05], it saves you from fishing through it looking for the biggest value, and turns it into [0,0,0,0,1,0,0,0,0] — The goal is to save a lot of coding!
• ‘sigmoid’ — produces a result from 0 to 1, which is great for binary classifiers

QUIZ

Question 1 — The diagram for traditional programming had Rules and Data In, but what came out?

Answers

Question 2 — The diagram for Machine Learning had Answers and Data In, but what came out?

Rules

Question 3 — When I tell a computer what the data represents (i.e. this data is for walking, this data is for running), what is that process called?

Labelling

Question 4 — What is a Dense?

A layer of connected neurons

Question 5 — What does a Loss function do?

Measures how good the current ‘guess’ is

Question 6 — What does the optimizer do?

Generates a new and improved guess

Question 7 — What is Convergence?

The process of getting very close to the correct answer

Question 8 — What does model.fit do?

It trains the neural network to fit one set of values to another

QUIZ

Question 1 — What’s the name of the dataset of Fashion images used in this week’s code?

Fashion MNIST

Question 2 — What do the above mentioned Images look like?

28×28 Greyscale

Question 3 — How many images are in the Fashion MNIST dataset?

70,000

Question 4 — Why are there 10 output neurons?

There are 10 different labels

Question 5 — What does Relu do?

It only returns x if x is greater than zero

Question 6 — Why do you split data into training and test sets?

To test a network with previously unseen data

Question 7 — What method gets called when an epoch finishes?

on_epoch_end

Question 8 — What parameter to you set in your fit function to tell it to use callbacks?

callbacks=

QUIZ

Question 1 — What is a Convolution?

A technique to isolate features in images

Question 2 — What is a Pooling?

A technique to reduce the information in an image while maintaining features

Question 3 — How do Convolutions improve image recognition?

They isolate features in images

Question 4 — After passing a 3×3 filter over a 28×28 image, how big will the output be?

26×26

Question 5 — After max pooling a 26×26 image with a 2×2 filter, how big will the output be?

13×13

Question 6 — Applying Convolutions on top of our Deep neural network will make training:

It depends on many factors. It might make your training faster or slower, and a poorly designed Convolutional layer may even be less efficient than a plain DNN!

QUIZ

Question 1 — Using Image Generator, how do you label images?

It’s based on the directory the image is contained in

Question 2 — What method on the Image Generator is used to normalize the image?

rescale

Question 3 — How did we specify the training size for the images?

The target_size parameter on the training generator

Question 4 — When we specify the input_shape to be (300, 300, 3), what does that mean?

Every Image will be 300×300 pixels, with 3 bytes to define color

Question 5 — If your training data is close to 1.000 accuracy, but your validation data isn’t, what’s the risk here?

You’re overfitting on your training data

Question 6 — Convolutional Neural Networks are better for classifying images like horses and humans because:

All of the above (The distinguishable features may be in different parts of the frame, There’s a wide variety of horses, There’s a wide variety of humans)

Question 7 — After reducing the size of the images, the training results were different. Why?

We removed some convolutions to handle the smaller images