Where is the current AI limit?
AI Dungeon is a game that let you create your own story through interaction with a BERT enabled model. It is an artificial intelligence take on those choose your own adventure books.
Not going to lie but that was my first idea of a interesting Natural Language Processing (NLP) Application; one that advances the capacities of models to understand humans to move forward to other challenges, like Semantics Processing, Criticial Thinking Machines. But I could not be happier that this game exists, and not only that. It exists and it is quite good.
The only problem it has it is that it doesn’t remember. One moment you will have your character moving North to Charleston. the other he becomes Jesus and resurrect his uncle Jim. (That is the story that happened to me.) The game developer address this challenge with a game design solution, and not with a deep learning model solution. You can make your story remember certain aspects.
On the other hand researchers at DeepMind proposed a new type of architecture they called Differentiable Neural Computer (DNC), that is a take on the Long-Short-Term-Memory (LSTM) Recurrent Neural Network (RNN). the LSTM was a solution for “forgettable” nature of the neural networks and let it remember something, but not enough. So the DNC shifts “memory” from LSTM (that keeps pushing some old values with the new inputs) to an idea of “attention” with what they call Compressive transformer.
I am more of a fan of the first project. Yes, the advancement of AI is on these papers and the research conducted by brilliant people. But what AI Dungeon is doing is putting these models out there. We need more projects out there to test these limits.
Because one question was not asked: will the increase of memory enable us to create AGI? Maybe that is not the problem after all because consciousness in AI terms will probably look very differently form how our own consciousness works. Also the cutting edge state of art development of AI models use too much computational power and doesn’t take into account what is more important for the development we hope: a negligible use of energy.
What is why I am excited to see projects such as Eyeriss. They approach Deep Learning on an energy consumption by creating a Convolucional NN specific chip.
What is exciting to think is where exactly is the AI limit. But every time we expect to find this limit in a particular aspect of the neural network, or the limitation of the programming language, a killing paper comes to obliterate it, or a fun project pushes to its limits.