How To Do Multivariate Time Series Forecasting Using LSTM
This is the 21st century, and it has been revolutionary for the development of machines so far and enabled us to perform supposedly impossible tasks; predicting the future was one of them. But now, with the help of advanced computational power and a tremendous boost in the field of artificial intelligence, machine learning, the process of predicting the future has become quite simple and fast. Some of the major applications of this field are Image recognition, Speech recognition, Traffic prediction, Self-driving car, Virtual Personal assistance, and the list continues. Time series forecasting is also an important area in machine learning. However, it is neglected due to its complexity, and this complexity is due to the time components like trend, seasonality, base level of series, Noise. Time series forecasting involves fitting models on historical data and using the fitment to predict the future data the same as the other ML technique. The only major difference between the simple prediction based model and forecasting model is that here the forecasting is completely unavailable and must be only estimated with the help of what has already happened. This article will discuss deep learning techniques used to address forecasting using multiple dependent variables and one target variable. This technique is taken from the Book called ‘Hands on Time series analysis using Python’.  The author used a Bidirectional LSTM based network with customized data preparation, and the result is supposed to follow the trend. Let’s check the result practically by leveraging python. Code implementation Multivariate Time Series Forecasting Using LSTM Import all dependencies: import pandas as pd import numpy as np import matplotlib.pyplot as plt import plotly.express as px # to plot the time series plot from sklearn import metrics # for the evaluation from sklearn.preprocessing import LabelEncoder,MinMaxScaler import tensorflow as tf Dataset is about the Metro interstate traffic status comprising nine variables and the target variable, and the samples are taken for six years from 2012 to 2018. First, let’s have a look at the data frame. data = pd.read_csv('metro data.csv') data Check out the trend using Plotly w.r.to target variable and date; here target variable is nothing but the traffic_volume for one year. Some of the variables are categorical. So we have to use LabelEncoder to convert it into numbers and use MinMaxScaler to scale down the values. The neural network converges sooner when it exposes the same scaled features and gives better accuracy. for i in data.select_dtypes('object').columns: le = LabelEncoder().fit(data[i]) data[i] = le.transform(data[i]) X_scaler = MinMaxScaler() Y_scaler = MinMaxScaler() X_data = X_scaler.fit_transform(data[['holiday', 'temp', 'rain_1h', 'snow_1h', 'clouds_all', 'weather_main', 'weather_description','traffic_volume']]) Y_data = Y_scaler.fit_transform(data[['traffic_volume']]) Below is the user-defined function which preprocesses the data suitable for forecasting. def custom_ts_multi_data_prep(dataset, target, start, end, window, horizon): X = [] y = [] start = start + window if end is None: end = len(dataset) - horizon for i in range(start, end): indices = range(i-window, i) X.append(dataset[indices]) indicey = range(i+1, i+1+horizon) y.append(target[indicey]) return np.array(X), np.array(y) As we are doing multiple-step forecasting, let’s allow the model to see past 48 hours of data and forecast the 10 hrs after data; for that, we set the horizon to 10. hist_window = 48 horizon = 10 TRAIN_SPLIT = 30000 x_train, y_train = custom_ts_multi_data_prep(X_data, Y_data, 0, TRAIN_SPLIT, hist_window, horizon) x_vali, y_vali = custom_ts_multi_data_prep(X_data, Y_data, TRAIN_SPLIT, None, hist_window, horizon) The train window should contain eight variables and one target variable for about ten observations. print ('Multiple window of past history\n') print(x_train[0]) print ('\n Target horizon\n') print (y_train[0]) Output: Multiple window of past history [[0.63636364 0.92972555 0.         0.         0.4        0.1 0.7        0.76167582] [0.63636364 0.93320863 0.         0.         0.75       0.1 0.06666667 0.62032967] [0.63636364 0.93391815 0.         0.         0.9        0.1 0.56666667 0.65480769] [0.63636364 0.93569194 0.         0.         0.9        0.1 0.56666667 0.69038462] [0.63636364 0.93894927 0.         0.         0.75       0.1 0.06666667 0.67554945] [0.63636364 0.94081981 0.         0.         0.01       0. 0.73333333 0.71167582] [0.63636364 0.94549618 0.         0.         0.01       0. 0.73333333 0.76703297] [0.63636364 0.94772148 0.         0.         0.01       0. 0.73333333 0.82623626] [0.63636364 0.9486245  0.         0.         0.2        0.1 0.13333333 0.79546703] [0.63636364 0.94527042 0.         0.         0.2        0.1 0.13333333 0.65521978] [0.63636364 0.93840101 0.         0.         0.2        0.1 0.13333333 0.48612637] [0.63636364 0.93327313 0.         0.         0.01       0. 0.73333333 0.38241758] [0.63636364 0.93078982 0.         0.         0.01       0. 0.73333333 0.32431319] [0.63636364 0.92611346 0.         0.         0.01       0. 0.73333333 0.21002747]] Target horizon [[0.75563187] [0.78475275] [0.86428571] [0.83200549] [0.67403846] [0.48118132] [0.41717033] [0.38763736] [0.27362637] [0.16016484]] Prepare the training data and validation data using the TensorFlow data function, which faster and efficient way to feed data for training. batch_size = 256 buffer_size = 150 train_data = tf.data.Dataset.from_tensor_slices((x_train, y_train)) train_data = train_data.cache().shuffle(buffer_size).batch(batch_size).repeat() val_data = tf.data.Dataset.from_tensor_slices((x_vali, y_vali)) val_data = val_data.batch(batch_size).repeat() Build and compile the model lstm_model = tf.keras.models.Sequential([ tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(200, return_sequences=True), input_shape=x_train.shape[-2:]), tf.keras.layers.Dense(20, activation='tanh'), tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(150)), tf.keras.layers.Dense(20, activation='tanh'), tf.keras.layers.Dense(20, activation='tanh'), tf.keras.layers.Dropout(0.25), tf.keras.layers.Dense(units=horizon), ]) lstm_model.compile(optimizer='adam', loss='mse') lstm_model.summary() Configure the model and start training with early stopping and checkpoint. Early stopping stops training when monitored loss starts increasing above the patience, and checkpoint saves the model weight as it reaches the minimum loss. model_path = 'Bidirectional_LSTM_Multivariate.h5' early_stopings = tf.keras.callbacks.EarlyStopping(monitor='val_loss', min_delta=0, patience=10, verbose=1, mode='min') checkpoint =  tf.keras.callbacks.ModelCheckpoint(model_path, monitor='val_loss', save_best_only=True, mode='min', verbose=0) callbacks=[early_stopings,checkpoint] history = lstm_model.fit(train_data,epochs=150,steps_per_epoch=100,validation_data=val_data,validation_steps=50,verbose=1,callbacks=callbacks) Early stopping has done its job; out of 150 epochs model stopped training at 32 epochs. Check the loss curve for training and validation. plt.figure(figsize=(16,9)) plt.plot(history.history['loss']) plt.plot(history.history['val_loss']) plt.title('Model loss') plt.ylabel('loss') plt.xlabel('epoch') plt.legend(['train loss', 'validation loss']) plt.show() Prepare the testing data for the last 48 hrs and check the prediction against it by visualizing the actual and predicted values. Finally, evaluate the result with standard performance metrics. data_val = X_scaler.fit_transform(data[['holiday', 'temp', 'rain_1h', 'snow_1h', 'clouds_all', 'weather_main','weather_description', 'traffic_volume']].tail(48)) val_rescaled = data_val.reshape(1, data_val.shape[0], data_val.shape[1]) pred = lstm_model.predict(val_rescaled) pred_Inverse = Y_scaler.inverse_transform(pred) pred_Inverse def timeseries_evaluation_metrics_func(y_true, y_pred): def mean_absolute_percentage_error(y_true, y_pred): y_true, y_pred = np.array(y_true), np.array(y_pred) return np.mean(np.abs((y_true - y_pred) / y_true)) * 100 print('Evaluation metric results:-') print(f'MSE is : {metrics.mean_squared_error(y_true, y_pred)}') print(f'MAE is : {metrics.mean_absolute_error(y_true, y_pred)}') print(f'RMSE is : {np.sqrt(metrics.mean_squared_error(y_true, y_pred))}') print(f'MAPE is : {mean_absolute_percentage_error(y_true, y_pred)}') print(f'R2 is : {metrics.r2_score(y_true, y_pred)}',end='\n\n') timeseries_evaluation_metrics_func(validate['traffic_volume'],pred_Inverse[0]) plt.figure(figsize=(16,9)) plt.plot( list(validate['traffic_volume'])) plt.plot( list(pred_Inverse[0])) plt.title("Actual vs Predicted") plt.ylabel("Traffic volume") plt.legend(('Actual','predicted')) plt.show() Conclusion: We have seen how the time series forecasting differs from any other prediction technique and the component like a trend; seasonality affects the analysis. We have discussed only one technique from the book, where the author has covered many more techniques for single-step multi-step analysis. The discussed method is quite impressive and can be adept in real-time situations. References: Link for colab Notebook implementation Links for pre-trained model and datasetOfficial Git-Hub repository for book    Plotly dashboard
This is the 21st century, and it has been revolutionary for the development of machines so far and enabled us…
Time series forecasting is also an important area in machine learning. However, it is neglected due to its complexity, and this complexity.
["LSTM Network", "Neural Networks", "Time Series Forecasting", "time series prediction"]
["artificial intelligence", "machine learning", "Keras", "AI", "neural network", "ML", "Time Series Forecasting", "Colab", "Ray", "deep learning", "time series prediction", "LSTM Network", "TensorFlow", "Neural Networks"]
["AI", "artificial intelligence", "machine learning", "ML", "deep learning", "neural network", "Ray", "TensorFlow", "Keras", "Colab"]
Vijaysinh is an enthusiast in machine learning and deep learning. He is skilled in ML algorithms, data manipulation, handling and visualization, model building.