AI Conflicts and Futures in The Lifecycle of Software Objects by Ted Chiang

Original article was published on Artificial Intelligence on Medium

AI Conflicts and Futures in The Lifecycle of Software Objects by Ted Chiang

Photo by Pascal Müller on Unsplash

The Lifecycle of Software Objects by Ted Chiang follows Ana Alvarado and Derek Brooks through a dozen or so years of their idealistic development of AI companions against an ever changing business and software landscape. Ana, a former zookeeper, joins Blue Gamma, a software company that is on the verge of releasing a new type of digient, a digital companion that people can develop and care for in Data Earth, the initially prevalent digital world people inhabit to interact and play games. Ana’s job is to train the initial set of digients before copies are released to Data Earth players, where they’ll train and grow them independently. The Blue Gamma digients run on a genomic engine called the Neuroblast Engine.

Over the dozen or so years of Ana and Derek raising their digients, Ted Chiang thoughtfully conveys a number of conflicts and possible futures of AI development.

Hothouses and Real World Experience

Digients are not a new concept to Data Earth, companies have released digients before to help players through quests in games and provide them with complimentary social interactions. Digients before Blue Gamma’s release were trained in hothouses, virtual environments without the time bounds of the human world. Akin to training today’s machine learning models. Ana and Derek grow to believe that general AI can’t be developed in hothouses, they have to learn by experiencing worlds as humans experience them.

Emotional and Physical Pain

Blue Gamma understood the wild west nature of Data Earth. It was not like the games of the real world. Developers could create continents in Data Earth with their own rules for people and digients to explore and interact with. Blue Gamma did not want their digients, who were like adolescent children learning and developing as their experienced the world, to be tortured or harmed. Blue Gamma equipped the Neuroblast digients with pain circuit-breakers to make them unappealing to certain players in Data Earth. What they couldn’t do, however, is make them intolerant to emotional abuse or neglect because it would hinder their ability to mature and develop.

Work and Play

Blue Gamma wasn’t without competitors in Data Earth. Most people thought of the Neuroblast digients as only useful for companionship and entertainment. They developed only at a pace that they could be trained by humans. Blue Gamma never found any perfect genome for their design, instead they trained a few initial digients and then sold copies with minor genetic mutations to the players. A new genomic engine called Sophonce was introduced to vastly reduce social tendencies and increase propensity for obsessive behavior. The Sophonce digients were then trained in hothouses to become obsessive about completing specific types of tasks. Player’s bought puzzle solving Sophonce digients to help them with quests in Data Earth, and eventually developers tried to train them to do work in the real world. They found, however, that the Sophonce digients were so obsessive that when sold corporations to complete tasks, they were disobedient because of their attachment to their original trainers. Neuroblast digients on the other hand, upon learning about Ana and Derek’s need to work to pay for them, offered to work on their behalf and make them money. The Neuroblast digient working group never considered them for profit, and struggled to understand whether it was okay to give them homework to learn more, as it would mean introducing some level of negative reinforcement for not succeeding.

Physical and Digital Worlds

A hardware company, SaruMech Toys, creates a robot that Blue Gamma can upload their digients into so they can experience the real world. Derek changes a few of their digients avatars in Data Earth to match the SaruMech Toys robot to assimilate them to the different body type. The digients are fascinated by the textures of the real world, they want to feel everything. Data Earth doesn’t have complicated textures because human players don’t use peripherals with tactile feedback. The most the digients had experienced before was collision and a coefficient of friction on any particular surface. The digients are generally bored of the real world because the SaruMech robot is classified as an unpiloted free-roaming vehicle. They can’t explore the world without being accompanied by their owners.

Suspension

Neuroblast digients are designed such that they can be suspended by their owners. This feature ensures that they aren’t left neglected in Data Earth when their owners aren’t around to give them the attention that they need. Suspension has an interesting side effect, one digient will notice that another is missing from regular play sessions in Data Earth. When the missing digient is taken out of suspension, his friends tell him about their experiences while he was gone. The suspended digient is sad to have missed a month of play.

Rollback

Owners can roll back their digients to previous points in time to erase experiences and “start over” from a certain checkpoint. This feature was used a lot when initially learning how to train the digients during testing. Derek’s digients, Marco and Polo, get in a fight, they argued about whether it was fair that Marco had been created before Polo by Blue Gamma. After weeks of not feeling the same about each other, they ask Derek to roll them back to before their argument so they can be best friends again. Derek considers it but ultimately refuses and decides they need to learn to become friends again.

Incorporation

A wealthy lawyer decides that his digient can make decisions in the real world and incorporates him. The digient, Voyl, can run on his own Data Earth account. Voyl pays taxes and can own property just like any human. The lawyer challenges the world to sue him. When Marco and Polo find out, they too want to be incorporated so they can make their own decisions. Derek doesn’t think they’re old enough and hopes someone does take Voyl to court so that precedents can be set for digients as corporations. He does one day want to be able to incorporate his digients.

Software Compatibility

Real Space is released as a competitor to Data Earth and software companies start porting their digients to Real Space so users can take them to the new world. At this point Blue Gamma is no longer around and there’s nobody to do the work to port the Neuroblast engine to Real Space. As people start to move over to Real Space, Data Earth becomes empty of people for the Neuroblast digients to interact and play games with. Ana and Derek deploy a private Data Earth server for their digients to live on while they figure out how to get the Neuroblast engine ported to Real Space. The digients are bored and sad and even ask to be suspended while they wait. Most of their friends are in Real Space while they’re stuck in a private Data Earth server.

Sexuality

A company called Binary Desire approaches the Neuroblast working group and offers to pay for the port to Real Space in exchange for an odd request. Binary Desire wants to use copies of the Neuroblast digients, now at least 10 years old, as sexual partners for players in Real Space. They believe that they won’t be selling an artificial sex toy, but developing a life long sexual partner of a new kind. Binary Desire believes that people will come to view digient sex as one of many other acceptable forms of expressing sexuality. They’ll do this by editing the reward maps of the digients, giving them anatomically correct reproductive organs and allowing them to develop sexual relationships with customers. Derek talks to Marco and Polo about the offer. Marco says it might be fun to have his reward map edited. He argues that Binary Desire editing his reward map is no different than Blue Gamma initially programming it the way they wanted. Marco understands that one copy of him will keep the same reward map and experience life with Derek, while another copy of him will have an edited reward map and be with a new person. The Neuroblast working group doesn’t seem to understand the nature of the digients as well as they understand themselves.

Personal vs. Digital Sacrifice

The alternative to selling copies of the digients to Binary Desire is Ana going to work for a company called Polytope, where she’ll be tasked with training Sophonce digients. Because of the asocial tendencies of Sophonce digients, Ana will be required to use an InstantRapport, a patch that delivers a cocktail of opioids to the user when they are in the presence of a specific person, so that she will artificially feel attached to the digients she’ll be training. Ana has to weigh the benefits of having her own reward map edited vs. having the Neuroblast digients copied and edited themselves.

Questions for Consideration

Could general AI be developed and trained without the time bounds of the real world and interaction and experiences without humans?

If an AI can have experiences, is it okay to subject them to negative reinforcement for learning?

Will an AI understand its own nature better than we can, and will our emotional attachments hinder their development and progress?

What level of intelligence does an AI need to have in order to be incorporated and make its own legal decisions?

Does an AI need to have experiences in the physical world in order to develop or can they live in entirely digital landscapes and learn by interacting with humans in text, video, and avatar form?

Is it wrong to make an experiencing software program do something that it doesn’t want to do?

Topics in the Book I didn’t Cover