Deep Belief Network Regression (on financial time series)

Source: Deep Learning on Medium

Deep Belief Network Regression (on financial time series)

I wanted to experiment with Deep Belief Networks for univariate time series regression and found a Python library that runs on numpy and tensorflow and includes modules for supervised regression and classification. The time series that I used was the SPY exchange-traded fund (ETF) — which tracks the S&P500. The goal was to use 20 consecutive days’ closing price to create an autoregressive model that could predict the closing price 5 days into the future. I compared the model and the R2 score from the DBN with the results from Lasso and LSTM-based models. First off, here’s the time series.

I decomposed this series using Empirical Mode Decomposition, which separates nonlinear, nonstationary time series such as these into narrow-band oscillating components that operate on a few “physically meaningful” frequencies, as well as a monotonically increasing residue that represents the long-term trend of the data. Here’s what the IMFs look like — you can see that they act like a dyadic filter bank, meaning that the (average) frequencies differ by a factor of 2. Here’s what they look like:

IMFs and the Residue of the SPY time series.

Here are the means and standard deviations of each IMF and the residue:

You can see that most of them have a mean close to 0. The sum of the IMFs and the Residue equals the original series. I created a new series by taking out IMF1 and IMF2 (high-frequency components)as well as the residue (monotonic trend). If you sum what remains, you get a smoother series that oscillates around 0.

Sums of IMFs 2–6

The series was then differenced once (an I(1) series in ARIMA parlance) to get this:

Differenced Time Series

I divided this into a training and validation set, without shuffling, so that the validation represents the out of sample future. Firstly, I trained a LASSO model, and this is how it fared on the validation set:

Then came the Deep Belief Network Model, and here’s how it did on the validation set:

And finally, here’s how an LSTM + FCN model worked on the validation set:

In terms of the R2 score, the LSTM did the best. I’m going to test this on some other time-series data sets to get a better sense of what works best in what scenario, as well as work in some other RNN-variants.