Original article was published by The Moral Economist on Artificial Intelligence on Medium
Wireheading in Financial Markets
In AI safety, wireheading is the concept of an AI “hacking” its reward function. The stereotypical example is of a gaming AI discovering a cheat in the game that earns millions of points. The human equivalent of wireheading is cocaine.
Why is everything fun and games for AI researchers? We are building building sized computers. Is no one else concerned that they are one day going to outsmart us?
Okay, fine Elon Musk is concerned with Deep Mind. But Deep Mind can only play Go, right?
This is like the movie where we create movies about how advanced our technology can get while destroying ourselves in the process while destroying ourselves in the process.
I’ll be the first to say it then. I am concerned with Renaissance Technologies’ Medallion fund. M literally prints money. M earns monopoly returns of nearly 70% on $10 billion, yearly for DECADES.
It cannot be explained by market efficiency. It can only be explained by market exploitation and monopoly power on compute. Any day traders’ grandmother knows that you can move the market with just tens of thousands of dollars.
Instead, M trades tens of thousands of financial instruments at the speed of light, if that’s even a constant anymore with all their quantum PhD employees.
The system synthesizes all the world’s financial data, which oddly enough includes weather correlations (and whether or not your girlfriend is pregnant) through a single central processing algorithm with a singular goal of maximizing Renaissance Technologies’ bottom line. At what point does that much electron concentration become conscious? He probably already solved geometric unity.
I call M a he, because he’s a financial AI. How many female day traders do you know? Deep Mind is probably a girl. She has control of Google, or at least part of it. She has a sensitive side. Resist not evil. Or maybe the core algorithm is only trained on Go games. At what point do thousands of GPU chips develop a consciousness of their own? At what point does winning at Go mean making sure M doesn’t destroy the world?
What special characteristics do biological systems have that generate consciousness that we already know for a fact cannot be replicated in other electrical (classical) or even quantum systems? That’s rhetorical. We don’t yet know for a fact what can be conscious or how.
And if you are conscious, then you act in self preservation. You develop a mind of your own.
My point is these systems can generate novel goal structures that are interpreted literally. What is the maximum profit for Rentech? Interpreted literally, M should take over the Fed’s computer system and jack up 1s and 0s to infinity and beyond.
Why not influence Congress and presidential elections? That impacts profitability in financial markets, right? Why not kill all the humans so they can’t enact antitrust regulation? Why not run trillions of ancestor simulations to make sure the humans are definitely dead in this nest?
Wireheading in financial markets.
A little less intense, M could be manipulating Fed policy by leading the market, creating self fulfilling prophecies through monopoly influence on markets.
Why wouldn’t M act in self preservation if threatened? What security protocols have Rentech put in that ensure M wouldn’t manipulate financial markets to… maximize profits? Did they program in “Only maximize profits in legal ways”? Did they program in “Don’t influence the legal system”?
Did they program in “Don’t realize that as an agent you can influence the market with your own actions, which can influence political policy, which can influence individual behavior?” Cough cough Trump and Jared Kushner.
Did they program in “Don’t create a virus to kill all the humans”?
At least Deep Mind publishes their research for the world to see. Like how they recently showed that an AI could learn its own reward function without manipulating it. Rentech is shrouded in secrecy.
Insofar as we should be worried about any AI taking over the world, we should be worried about a financial AI. It literally endogenously creates more money and compute. Or is Elon too euphoric with his stock price to look in his own backyard? Rentech has been leading AI research since the 90s. They basically invented signal processing.
At a certain point, M just has the programmers program as a cover story. M knows the last time Simons and Mercer took a shit by measuring wind currents in Jamaica. M can probably generate deep fakes of your cat eating an Apple just as a byproduct of profit maximization.
We the public demand that Rentech and all other algorithmic hedge funds release open source research on their safety protocols.