Source: Deep Learning on Medium
Deep Stock Visualization with Dreams
Visualizing and understanding finance data is certainly important. It might just be our bread and butter in alpha research! What does that have to do with dreams, or rather, DeepDreams? 🙂
Believe it or not, the image above is actually Amazon’s stock price baselined with the S&P500 index. Readers who follow through the article will have a better sense of what the “deep” hype is all about, as well as several key observations about the stock market. Note that the post is not just about DeepDreams, but rather stock visualization in general. Deep Learning just happens to be a convenient tool.
Stock Visualisation Platform
At one point, one will realize there’s just too many codes and securities to keep track of that it becomes rather messy and infeasible. Hence, I built a stock visualization platform with the news data I’ve collected, and thanks to the BERT model (I have motivated the use of BERT in this post) that I have used to identify entities and DBPedia for Entity Linking for all the news in the database. The database has been proven to be quite useful in several use cases, which I will demonstrate soon.
I am able to search for all the news related to the companies in S&P500 (I only did the mapping for them) as well as several well-known person entities such as Donald Trump. We will see later see how influential Donald Trump is. The dashboard will show the closing prices of the company within the date of the searched time frame, and benchmarked against S&P500. I have started the post with Deep Learning, hence let’s get right to it. In this post, I would like to illustrate the use of Deep Learning to solve our problems, and in this case, Named Entity Recognition! For each of the news in the database, persons or organizations that are mentioned in the news are first identified by BERT and stored in the database. Which such information in the vault, it only makes sense to start making use of them to generate insight.
We will make use of Graph Theories to make connections between entities. Each edge (or connection) represents the frequency in which the entities co-occur together. Hence, the darker the edge, the more frequently the entities are mentioned together in the news.
Based just on our visual input, we could identify a few main players in the news coverage. Donald Trump, Putin, Xi Jinping, Kim Jong-un and so forth. I am sure it comes as no surprise that Donald Trump is in the center of the network, taking on a degree of centrality of 1. Not just that, Donald Trump is still going strong on other graph metrics such as Betweenness and Eigenvalue! What this basically means is that Donald Trump holds the most precious node in the network, having him removed will disconnect a huge portion of the network. Remember when I said we will examine how influential Donald Trump is? Checked.
Readers at this point might start to wonder, so what does this have to do with DeepDreams? It’s coming soon 🙂 Before that, let’s visualize the network of organizations.
Notice here that, Apple Inc, Google Inc, Facebook Inc, Netflix Inc, Huawei, Uber are among the organizations that often co-occur in the news. This could mean that they have a huge influence on one another. Hence, if there are several blue-chip companies we ought to look out for in 2019, these might be our best bet.
Now, let the charts begin. Let’s visualize some of the stock prices of the aforementioned companies.
Woooooow! Indeed, they look highly correlated with each other. Thanks, BERT! Since I have motivated the post with DeepDreams in the beginning, here is where I connect the dots. DeepDreams is actually nothing but a new and creative way of running neural networks, pre-trained on ImageNet, which is why we often see images of animals. Just out of curiosity, upon generating the charts above, as I was preparing for the spooky machine learning theme for Halloween Night, I thought I might as well feed the charts through the DeepDream model as well.
DeepDream at its core work based on recursion, where it makes a forward pass till a particular layer, and gradient descent is done on the image instead of the network. This cycle is done recursively. The deeper it goes, the more dreamy it gets! Computationally, it is considered cheap. Hence, this is the result.
The algorithm will try to identify patterns in the input image and recursively changing the content in those areas. Sometimes “patterns” may get people excited. What if the network is able to spit clues on the stock prices being “bearish” or “bullish”? : ) This may not be just for aesthetics, and may actually be useful to identify patterns in the market. As the spooky animal appearances coincide with the periods of extreme drawdown, I refuse to believe that this is all just a coincidence 😀
Anyway, back to stock prices. Another observation is that these stocks move closely with the market index. For a case in point,
Apple’s stock price has been constantly falling between October 2018 to Jan 2019. Hence, let’s try to understand what is going on.
Some of the major news regarding Apple Inc within this period includes:
- Apple’s infringement case with Qualcomm
- Apple’s release of new iPhones such as iPhone XR, new iPad, new Macbook Air
- Apple’s Takeover of Shazam
- Mentions of profits due to the new devices
By the way, clicking “View News” will bring us to the source of the news for further investigation! 😀 Aggregating and researching about a company has never been easier. Glad I spent all this time cleaning and dealing with data to create this platform.
Other than a lawsuit with Qualcomm, the rest of the news is generally pretty positive and should drive the stock prices up. What happened? Trump’s trade war with China happened. Apple will be much affected as “iPhone is assembled in China” and “a tariff of 10 percent which will be increased up to 25 percent by January will be imposed to $200 billion worth of Chinese goods. ” In fact, it has affected all of the companies as identified by our graph network.
The point I am trying to make here is that, even though the company may be doing very good themselves, the change in fiscal policies can really turn the market upside down, hence dragging the “good samaritans” or rather the “good corporate citizens” of the market along. Hence, the moral of the story is, modeling the market indices may be the key to predicting the stock prices of these companies.
Recently, there have been applications of graph embeddings in the Deep Learning space as well. I wonder if there is a deep way of converting the network graph into a feature space that is more friendly for training an AI model? I wonder :))
That is all for this post. For other series regarding my final year project on stock market analysis and prediction,
- Deep Learning the Stock Market | Feasibility Studies
- Deep Learning the Stock Market | Return Forecast
- Deep ESG Scoring with Embeddings
Stay tuned for more! 🙂