Brain in your browser

Original article was published by Rafał Rybnik on Artificial Intelligence on Medium


What if a website would be able to predict your preferences and adjust its appearance to maximize your experience?

Or if a web-based video conferencing application could replace a messy room with another background and cancel noises of your dog?

We already have the technology.

Thanks to JavaScript machine learning frameworks, such as Tensorflow.js or ONNX.js, we can easily implement these functions in the web environment.

However, they have one disadvantage — poor performance in comparison with native apps.

Neural hardware

Hardware architectures specialized for machine learning purposes emerged because of increasing demand for computing capabilities.

Progress in computing capabilities and web application do not always go hand-in-hand (fig. by Author)

Since 2017 (release of iPhone 8), Apple implements hardware support for neural networks in iPhone and iPad processors. Other platforms develop their own solutions as well.

These dedicated platforms help optimize the performance of native applications. And make them more energy efficient.

The problem is the Web is disconnected from that advanced machine learning hardware.

“The architecture of these new machine learning hardware is quite diverse. It is challenging to expose them by the general CPU and GPU compute web APIs, e.g. WebAssembly and WebGPU” says Ningxin Hu, Intel’s software engineer and member of W3C Device APIs working group.

In tests, web applications, despite using modern optimization techniques, have up to 10 times worse performance than native software.

The Web is disconnected from that advanced machine learning hardware (fig. by Ningxin Hu)

Brain in a vat

Neural networks are an attempt to simulate the network of neurons that make up a human brain.

This allows computers to learn things and make decisions in a human-like manner.

A core abstraction of neural networks behind popular deep learning frameworks is a directed graph with its nodes corresponding to operations (ops) and input variables.

Representation of a neural network. (fig. by @artux51 in the article explaining neural networks)

One node’s output value is the input to another node.

As the authors of the proposal claim, Web Neural Network API brings this abstraction to the Web.

Web Neural Network API

The architecture view of Web Neural Networks API (fig. by Ningxin Hu)

WebNN API provides dedicated low-level API for neural network inference hardware acceleration.

Ningxin adds, “WebNN introduces the primitives for deep learning, including tensors and operations. Developers can build a computational graph by wiring these elements.

Then developers can use WebNN to compile and execute the graph.

WebNN browser implementation would leverage OS ML API to access hardware.”

It is expected that Machine Learning frameworks will be key consumers of the Web Neural Network API (WebNN API) and the low-level details exposed through the WebNN API are abstracted out from typical web developers.

— Authors of proposal

However, authors also expect web developers with specific interest and competence in Machine Learning will want to interface with the WebNN API directly instead of a higher-level ML framework.

To prove the concept, we experimentally implemented the WebNN API in a customized Chromium browser, we followed the multi-process architecture of Chromium, we implemented the JavaScript interface of WebNN in Blink and the native API backend in GPU process.

— Authors of proposal

(fig. source, simplified by Author)

In tests, it has the close-to-native performance

Possible applications

  • Handwriting recognition — sample by authors of the proposal
  • Person Detection
  • Semantic Segmentation
  • Skeleton Detection
  • Face Recognition
  • Facial Landmark Detection
  • Style Transfer
  • Super Resolution
  • Image Captioning
  • Machine Translation
  • Emotion Analysis
  • Video Summarization
  • Noise Suppression

Takeaways

Neural networks are a widely used tool for building intelligent applications.

Web Neural Network API is a dedicated low-level API making neural network inference faster and energy-efficient by engaging hardware acceleration.

Although this is only an initial proposal, it is worth reading the full specification.

References