Time Series Forecasting using Artificial Neural Networks

In today’s world, everyone is excited to know what will happen in future. Researchers have been developing sophisticated algorithms to forecast weather, earthquake, stock prices and whatnot. Nowadays, we are using forecasting in every domain. In this article, I will explain how we can do time series forecasting using Neural Networks.

Artificial Neural Networks are used as an alternative technique for time series forecasting as it has gained more popularity in recent years. Neural Networks are universal approximators. They can detect non-linear patterns and hidden information inside the data. They are data-driven and self-adaptive in nature.

Unlike statistical methods, we need not make any statistical assumptions. In statistical methods like ARIMA, we have to make sure that the time series is stationary. If the time series has seasonality, we have to use SARIMA, an extended version of ARIMA. But, we need not worry about that if we are using Neural Networks for forecasting. But, there are some variants of Neural Networks that we use only for seasonal time series. We will discuss in later parts of the article.

In this article, I will focus on explaining three variants of Neural Networks that are mainly used for time series forecasting. They are:

1. Feedforward Neural Networks (FNN)
2. Time lagged Neural Networks (TLNN)
3. Seasonal Neural Networks (SANN)

Feedforward Neural Networks (FNN):

The most used Artificial Neural Networks (ANN) are Feedforward Neural Networks (FNN) or Multilayer Perceptrons(MLPs). The word itself says that there will be multiple layers of perceptrons. The network consists of an input layer, hidden layer and output layer. The hidden layers can be more than 1. You can see the network architecture below:

Our aim is to forecast the next data point. So, the number of neurons in the output layer will be one. The number of neurons in the first layer is “p”, where p is the number of past time steps that need to be considered to forecast in ‘t+1’ time stamp. The number of units in the first layer and hidden layer can be selected by hit and trial method. If we have domain knowledge of the data, we can select ‘p’ easily. Otherwise, we can try a different combination of ‘p’ and ‘q’ values and select based on model performance.

The output ‘x_t+1’ depends on the previous ‘p’ time steps (x_t-p+1, x_t-p+2, … x_t), which is the non-linear mapping from previous ‘p’ inputs to the output. Most commonly, sigmoid is used as the activation function at each neuron, which causes non-linearity in the network. The cost function used is Mean squared error (MSE).

Time lagged Neural Networks (TLNN):

This is one of the variants of FNN. In addition to previous ‘p’ time steps, we also use time steps at different lags. Let’s say, the seasonality is 12. The output x_t+1 depends on the previous ‘p’ time steps (x_t-p+1, x_t-p+2, … x_t) and x_t-s. This helps us to capture the information at the previous season too.

If s=12 and p=3, then x_t+1 depends on (x_t-12, x_t-2, x_t-1 and x_t). We can also use x_t-13, x_t-14,..etc. It completely depends on us which time steps to choose based on the data. We have to do a lot of experiments to come up with a good set of parameters.

Seasonal Neural Networks (SANN):

The Seasonal Neural Networks is used mainly for seasonal time series data. Unlike traditional approaches like SARIMA, we can discover the seasonal patterns in the time series data, without removing them.

If you observe the above architecture, we are forecasting the next ‘s’ time steps using the previous ‘s’ time steps as input. We should appropriately select the parameter ‘s’. If we are forecasting weather, we should ideally choose ‘s’ as 12. If the seasonality of the time series changes for each quarter, we should choose ‘s’ as 4.

We learnt about the different architectures that can be used for time series forecasting. These are some of the most used Neural Networks. Selecting the appropriate network will need a lot of data analysis and domain knowledge. We should also make sure that the network doesn’t overfit on the training data to avoid poor results. We can use regularization or dropout to avoid this. It also happens when we have less training data. So, spend some time in collecting a sufficient amount of data.

As the technology is growing, researchers have come up with different architectures like Recurrent Neural Networks (RNNs), Long Short Term Memory (LSTM). The RNNs and LSTM are mainly used for time-dependent data like time series, text, speech..etc. When compared with the above Neural Networks, researchers have proved that LSTM gives more accurate results.

I have created a detailed Kaggle kernel, in which I have used “Pune precipitation data” and compared the performance of FNN, TLNN, SANN and LSTM. LSTM outperformed all the other networks. I have also provided Python code, which automatically selects the number of neurons too. Do check the below mentioned kernel.

References:

1. Research paper: An Introductory Study on Time Series Modeling and Forecasting
2. Kaggle kernel: Time Series Analysis - Artificial Neural Networks

I hope you got some idea of time series forecasting using Artificial Neural Networks. If you have any queries, comment in the comments section below. I would be more than happy to answer your queries.

Thank you for reading my blog and supporting me. Stay tuned for my next article. If you want to receive email updates, don’t forget to subscribe to my blog. Keep learning and sharing!!

Follow me here:
GitHub: https://github.com/Abhishekmamidi123
LinkedIn: https://www.linkedin.com/in/abhishekmamidi
Kaggle: https://www.kaggle.com/abhishekmamidi
If you are looking for any specific blog, please do comment in the comment section below.