The Future of Time Series Forecasting

Original article was published by O’Reilly Media on Deep Learning on Medium


The Future of Time Series Forecasting

Editor’s Note: Time series data analysis and forecasting have become increasingly important due to the massive production of time series data, and as continuous monitoring and collection of such data becomes more common, the need for more efficient analysis and forecasting will only increase. As a foremost expert on time series analysis and forecasting, Aileen Nielsen shares her thoughts on what’s on the horizon for time series forecasting, from enhanced methodologies to the integration of time series forecasting into everyday life. We’d love to hear from you about what you think about this piece.

There are many good quotes about the hopelessness of predicting the future, and yet I can’t help wanting to share some thoughts about what’s coming.

Forecasting as a Service

Because time series forecasting has fewer expert practitioners than other areas of data science, there has been a drive to develop time series analysis and forecasting as a service that can be easily packaged and rolled out in an efficient way. For example, Amazon recently rolled out a time series prediction service, and it’s not the only company to do so. The company’s model seems deliberately general, and it frames forecasting as just one step in a data pipeline.

These forecasting-as-a-service modeling endeavors aim for a good enough general model that can accommodate a variety of fields without making terribly inaccurate forecasts. Most of them describe their models as using a mix of deep learning and traditional statistical models. However, because the service is ultimately a black box, it will be difficult to understand what could make forecasts go wrong or even to retrospectively investigate how they can be improved. This means there is a reasonably high quality level for the forecasts but probably a performance ceiling as well.

This service can be valuable for companies that need many forecasts but do not have the personnel available to generate them individually. However, for companies that have substantial amounts of historical data where more general heuristics and “laws” of their data could be discovered, it’s likely that an analyst could outperform these algorithms given familiarity with the domain.

Notice that a fair amount of the product that is being sold in the area of forecasting-as-a-service also has to do with good visualizations of the forecasts and pipeline utilities to easily revise forecasts, change the forecasting frequency, and so on. Even if your organization will ultimately build out its own forecasting analyses, it can be helpful to see what is emerging as an industry standard.

Deep Learning Enhances Probabilistic Possibilities

In the past few years, many of the largest tech companies have made some information public about how they do their own forecasting for their most important services. In this case we are not talking about the need for many parallel forecasts of the large number of metrics that affect the company’s business, but rather core concerns. In these cases, where the quality of the forecast is paramount, companies are often giving indications that they are using deep learning with a probabilistic component.

For example, Uber has blogged about forecasting demand for car rides, and Amazon has developed a well-regarded autoregressive recurrent neural network, inspired by statistical thinking for making predictions about product demands. The more that researchers are able to integrate statistical methodologies, such as the injection of priors for domain knowledge and the quantification of uncertainty, the less reason there will be to seek out statistical models when a deep learning model can offer the strengths of both statistics and deep learning.

However, making reasonably interpretable deep learning models — so that we can know just how “wrong” or extreme a forecast can be — remains a difficult endeavor, so it’s unlikely that traditional statistical models, with greater theoretical understanding and mechanistic clarity, will be discarded. For critical forecasts, where health and safety may be at risk, people reasonably may continue to rely on what has worked for decades until more transparent and inspectable methods can be developed for machine learning forecasts.

Increasing Importance of Machine Learning Rather Than Statistics

Empirically there seems to be less and less use of proper statistics in modeling data and generating predictions. Do not despair: the field of statistics continues to thrive and answer interesting questions that are related to statistics. And yet — particularly for low-stakes forecasts that merely need to be good enough — machine learning techniques and results-oriented statistical methods, rather than fancy theories and closed-form solutions or proofs of convergence, are winning out in actual deployment and real-world use cases.

From a practitioner’s perspective, this is a good thing. If you happily left your problem sets behind long ago with no desire to prove things, you needn’t fear a return of proper proofs and the like. On the other hand, this is a worrying trend as these technologies make their way into more and more fundamental aspects of life. I don’t mind surfing a retailer’s website that uses machine learning to guesstimate my likely future actions as a buyer. But I’d like to know that the time series predictions that modeled my health outcomes or my child’s academic progressions were more thorough and were statistically validated, because a biased model could really hurt someone in these core areas.

For now, the leaders in time series thinking for industrial purposes are working in low-stakes areas. For problems such as predicting revenues from an advertising campaign or a social media product rollout, it’s not important whether the forecasts are fully validated. As more fundamental aspects of the care and feeding of human beings come into the modeling domain, let’s hope that statistics plays a more fundamental role in high-stakes forecasts.

Increasing Combination of Statistical and Machine Learning Methodologies

A number of indications point toward combining machine learning and statistical methodologies rather than simply searching for the “best” method for forecasting. This is an extension of increasing acceptance and use of ensemble methods for forecasting.

An example of an extraordinarily robust test with many real-world data sets is the recent M4 competition, a time series competition measuring forecasting accuracy on 100,00 time series data sets. The winning entry to this competition combined elements of a statistical model and a neural network. Likewise, the runner-up incorporated both machine learning and statistics, in this case by using an ensemble of statistical models but then using a gradient boosted tree (XGBoost) to choose the relative weightings of each model in the ensemble. In this example we see two distinctive ways machine learning and statistical approaches can be combined: either as alternative models assembled together (as in the case of the winning entry) or with one method determining how to set the metaparameters of the other method (as in the case of the runner-up). A comprehensive and highly accessible summary of the competition results was subsequently published in the International Journal of Forecasting.

As such combinations gain traction, we will likely see research develop in the area of determining what problems are most amenable to combining statistical and machine learning models, as well as best practices emerging for tuning the performance of these models and selecting architectures. We expect to see the same refinement as in other complex architectures, such as neural networks, whereby standard design paradigms emerge over time with known strengths, weaknesses, and training techniques.

More Forecasts for Everyday Life

More consumer-facing companies, such as mobile health and wellness applications, have rolled out or been asked to roll out, personalized predictions. As people grow more aware of just how much data their applications store about them and others, they are looking to take advantage by getting tailored forecasts for metrics such as health and fitness goals. Similarly, people often go looking for forecasts for everything from future real estate values (difficult to predict) to the likely arrival dates of migratory bird species.

More products will be explicitly driven by the demand to make forecasts, be it on esoteric subjects or for individual metrics. This means that more forecasting pipelines will be integrated into places where they have not been too likely before, such as mobile applications and websites for casual readers rather than industry specialists. The more common this becomes, the more likely that time series jargon will be part of everyday speech. Hopefully people will also acquire enough time series education to understand the limits and assumptions of forecasts so that they do not rely too much on these products.