We are going to use TensorFlow and create CNN model step by step. Then we will train the model with training data and evaluate the model with test data.
import tensorflow as tf
tf.__version__
The MNIST database of handwritten digits, has a training set of 60,000 examples, and a test set of 10,000 examples.
It is a good database for people who want to try learning techniques and pattern recognition methods on real-world data while spending minimal efforts on preprocessing and formatting.
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
print(x_train[0])
print(x_train.shape)
print(y_train)
print(y_train.shape)
import matplotlib.pyplot as plt
plt.imshow(x_train[0], cmap='gray')
plt.title('%i' % y_train[0])
plt.show()
num = 26
imgs = x_train[:num]
lbls = y_train[:num]
figure = plt.figure(figsize=(10, 8))
cols, rows = 5, 5
for i in range(1, cols * rows + 1):
figure.add_subplot(rows, cols, i)
plt.title(lbls[i])
plt.axis("off")
plt.imshow(imgs[i], cmap='gray')
plt.show()
x_train, x_test = x_train / 255.0, x_test / 255.0
print(x_train.shape)
print(x_test.shape)
Conv2D expects 4 dimensions (BatchSize, Channel, Width, Height).
Convolutional layers require a number of input channels. For an RBG image, that number is 3 and for greyscale images, the number of channels is 1.
As we are creating a convlution neural network we have to add channels.
# Add a channels dimension
x_train = x_train[..., tf.newaxis].astype("float32")
x_test = x_test[..., tf.newaxis].astype("float32")
print(x_train.shape)
print(x_test.shape)
The tf.data API enables you to build complex input pipelines from simple, reusable pieces. It's makes it possible to handle large amounts of data, read from different data formats, and perform complex transformations.
The tf.data API introduces a tf.data.Dataset abstraction that represents a sequence of elements, in which each element consists of one or more components.
To create an input pipeline, you must start with a data source. For example, to construct a Dataset from data in memory, you can use tf.data.Dataset.from_tensors() or tf.data.Dataset.from_tensor_slices().
from_tensor_slices can be used to combine different elements into one dataset, e.g., combine x_train, y_train into one dataset.
train_ds = tf.data.Dataset.from_tensor_slices(
(x_train, y_train)).shuffle(10000).batch(32)
test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(32)
print(train_ds)
print(test_ds)
A Convolutional Neural Network is type of neural network that is used mainly in image processing applications.
As input, a CNN takes tensors of shape (image_height, image_width, color_channels), ignoring the batch size. If you are new to these dimensions, color_channels refers to (R,G,B). In this example, you will configure our CNN to process inputs of shape (28, 28, 1), which is the format of MNIST images. You can do this by passing the argument input_shape to our first layer.
tf.keras.Model:
Model groups layers into an object with training and inference features.
There are two ways to instantiate a Model:
1) With the "Functional API", where we start from Input, you chain layer calls to specify the model's forward pass, and finally you create your model from inputs and outputs:
2) By subclassing the Model class: in that case, you have to define layers in __init__ and we have to implement the model's forward pass in call.
We are using second opttion.
from tensorflow.keras.layers import Dense, Flatten, Conv2D
from tensorflow.keras import Model
class CNN(Model):
def __init__(self):
super(CNN, self).__init__()
self.conv1 = Conv2D(32, 3, activation='relu')
self.flatten = Flatten()
self.d1 = Dense(128, activation='relu')
self.d2 = Dense(10)
def call(self, x):
x = self.conv1(x)
x = self.flatten(x)
x = self.d1(x)
return self.d2(x)
# Create an instance of the model
cnn = CNN()
print(cnn)
loss_func = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
loss_func
lr(Learning Rate): Rate at which our model updates the weights in the cells each time back-propagation is done.
optimizer = tf.keras.optimizers.Adam()
optimizer
train_loss = tf.keras.metrics.Mean(name='train_loss')
train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='train_accuracy')
test_loss = tf.keras.metrics.Mean(name='test_loss')
test_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='test_accuracy')
Create a function called train() and pass images and labels as inputs.
num_epochs: Number of times our model will go through the entire training dataset
num_epochs = 10
def train(images, labels):
with tf.GradientTape() as tape:
# training=True is only needed if there are layers with different
# behavior during training versus inference (e.g. Dropout).
predictions = cnn(images, training=True)
loss = loss_func(labels, predictions)
gradients = tape.gradient(loss, cnn.trainable_variables)
optimizer.apply_gradients(zip(gradients, cnn.trainable_variables))
train_loss(loss)
train_accuracy(labels, predictions)
pass
for epoch in range(num_epochs):
# Reset the metrics at the start of the next epoch
train_loss.reset_states()
train_accuracy.reset_states()
for images, labels in train_ds:
train(images, labels)
print(
f'Epoch {epoch + 1}, '
f'Loss: {train_loss.result()}, '
f'Accuracy: {train_accuracy.result() * 100}, '
)
def test(images, labels):
# training=False is only needed if there are layers with different
# behavior during training versus inference (e.g. Dropout).
predictions = cnn(images, training=False)
t_loss = loss_func(labels, predictions)
test_loss(t_loss)
test_accuracy(labels, predictions)
pass
for epoch in range(num_epochs):
test_loss.reset_states()
test_accuracy.reset_states()
for test_images, test_labels in test_ds:
test(test_images, test_labels)
print(
f'Epoch {epoch + 1}, '
f'Test Loss: {test_loss.result()}, '
f'Test Accuracy: {test_accuracy.result() * 100}'
)