We are going to learn about sequence prediction withLSTM model. We will pass an input sequence, predict the next value in the sequence.
Long short-term memory (LSTM) is an artificial recurrent neural network (RNN) architecture used in the field of deep learning. Unlike standard feedforward neural networks, LSTM has feedback connections. It can not only process single data points (such as images), but also entire sequences of data (such as speech or video).
For example, LSTM is applicable to tasks such as unsegmented, connected handwriting recognition, speech recognition and anomaly detection in network traffic or IDSs (intrusion detection systems).
LSTM networks are well-suited to classifying, processing and making predictions based on time series data, since there can be lags of unknown duration between important events in a time series.
A common LSTM unit is composed of a cell, an input gate, an output gate and a forget gate. The cell remembers values over arbitrary time intervals and the three gates regulate the flow of information into and out of the cell.
data = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200]
print(data)
We are going to split sequence like below:
X, y 10, 20, 30, 40, 50 60 20, 30, 40, 50, 60 70 30, 40, 50, 60, 70 80 40, 50, 60, 70, 80 90 50, 60, 70, 80, 90 100 60, 70, 80, 90, 100 110 70, 80, 90, 100, 110 120 ...
import numpy as np
def splitSequence(seq, n_steps):
#Declare X and y as empty list
X = []
y = []
for i in range(len(seq)):
#get the last index
lastIndex = i + n_steps
#if lastIndex is greater than length of sequence then break
if lastIndex > len(seq) - 1:
break
#Create input and output sequence
seq_X, seq_y = seq[i:lastIndex], seq[lastIndex]
#append seq_X, seq_y in X and y list
X.append(seq_X)
y.append(seq_y)
pass
#Convert X and y into numpy array
X = np.array(X)
y = np.array(y)
return X,y
pass
n_steps = 5
X, y = splitSequence(data, n_steps = 5)
print(X)
print(y)
for i in range(len(X)):
print(X[i], y[i])
For tf.keras.layers.LSTM model we need inputs with shape [batch, timesteps, feature]. Model required data in this format only. For that we need to reshape our X data.
# reshape from [samples, timesteps] into [samples, timesteps, features]
n_features = 1
X = X.reshape((X.shape[0], X.shape[1], n_features))
print(X[:2])
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
model = tf.keras.Sequential()
model.add(layers.LSTM(50, activation='relu', input_shape=(n_steps, n_features)))
model.add(layers.Dense(1))
model.layers
model.summary()
Once the model is created, we can config the model with losses and metrics with model.compile().
model.compile(optimizer=tf.keras.optimizers.Adam(0.01), loss=tf.keras.losses.MeanSquaredError(),
metrics=['accuracy'])
We can train the model with model.fit()
model.fit(X, y, epochs=200, verbose=1)
import pickle
filename = 'finalized_model.sav'
pickle.dump(model, open(filename, 'wb'))
loaded_model = pickle.load(open(filename, 'rb'))
test_data = np.array([90, 100, 110, 120, 130])
test_data = test_data.reshape((1, n_steps, n_features))
test_data
predictNextNumber = loaded_model.predict(test_data, verbose=1)
print(predictNextNumber)
predictNextNumber = model.predict(test_data, verbose=1)
print(predictNextNumber)
predictNextNumber = model.predict(test_data, verbose=1)
print(predictNextNumber)
test_data = np.array([160, 170, 180, 190, 200])
test_data = test_data.reshape((1, n_steps, n_features))
test_data
predictNextNumber = model.predict(test_data, verbose=1)
print(predictNextNumber)
test_data = np.array([200, 210, 220, 230, 240])
test_data = test_data.reshape((1, 5, 1))
test_data
predictNextNumber = model.predict(test_data, verbose=1)
print(predictNextNumber)