AI Scholar: Adversarial Attacks Implications

Source: Deep Learning on Medium


This research summary is just one of many that are distributed weekly on the AI scholar newsletter. To start receiving the weekly newsletter, sign up here.

Neural networks are vulnerable to adversarial attacks that can lead them to make false predictions such as confusing a dog with a frog. But as you probably know, they are not limited to image classifiers. Some are designed to degrade model performance by generating specific outputs selected by the attacker. For instance, an attacker can use them to cause massive damage in other different scenarios such as self-driving cars.

With this in mind, it is crucial to proactively anticipate other unexplored adversarial goals to make machine learning systems more secure.

Adversarial Reprogramming of Neural Networks

In a recent study, Google researchers considered a new and more challenging adversarial goal: reprogramming a model to perform a task chosen by the attacker, without the need for the attacker to compute the specific desired output.

Example images (a) with adversarial programs of different sizes, and (b) with adversarial programs of different perturbation scales. In ©, the adversarial data + program (right) are hidden inside a normal image from ImageNet (left), yielding an adversarial image (center) that is able to reprogram the network to function as an MNIST classifier

The researchers found out that trained neural networks can be reprogrammed to classify shuffled images, which do not retain any of the original spatial structure which suggests that reprogramming across domains is possible.

They also found out that trained neural networks were more susceptible to adversarial reprogramming than random systems and that reprogramming still succeeds even when the data structure is very different from the structure of the data in the primary task.

Potential Uses and Effects

There it is! A demonstration of adversarial reprogramming on classification tasks in the image domain.

Can similar attacks succeed for audio, video, text, or other areas? Take the adversarial reprogramming of RNNs for instance — an attacker only needs to find inputs for performing simple operations in the RNN and they can reprogram model to perform any computational task.

It’s scary what attackers can achieve if a specially crafted input can reprogram machine learning systems. For instance, it would be easy for attackers to steal computational resources or perform tasks that violate the cloud-based service’s programming and more. All this shows the potential threat adversarial reprogramming can bring to AI systems.

As the AI community looks towards the future, it is crucial to be aware of the potential challenges AI advances can bring, and work towards possible ways to alleviate or defend against them.

Read more: https://openreview.net/pdf?id=Syx_Ss05tm

Thanks for reading. Please comment, share and remember to subscribe to our weekly AI Scholar Newsletter for the most recent and interesting research papers! You can also follow me on Twitter and LinkedIn. Remember to 👏 if you enjoyed this article. Cheers!