TensorFlow Dev Summit 2020: Livestream Highlights

Original article can be found here (source): Deep Learning on Medium


In the research arena, the team announced some new models — T5 Talk-to-Text Transfer Transformer and Meena. The T5 Talk-to-Text Transfer Transformer has over 11 billion parameters, and Meena is a conversational model with over 2.6 billion parameters.

Some other key announcements that will make the work of researchers easier are TensorBoard.dev, the Performance Profiler, and Research Add-ons & extensions.

TensorBoard.dev is tool that will allow developers and researchers to host and share their machine learning experiments for free.

The performance profiler is available within TensorBoard and will provide information on the model’s performance and guidance on debugging.

The performance profiler toolset includes:

  • Overview Page that will provide a summary of the workload running on the device.
  • Input Pipeline Analyzer for analyzing a TensorFlow input pipeline.
  • TensorFlow Stats that displays the performance of every TensorFlow operation that’s executed on the host or device during a profiling session.
  • Trace Viewer that displays a timeline that shows the duration for the operations that were executed by the TensorFlow model, as well as which part of the host device executed the operation.
  • GPU kernel stats that show the performance statistics and the originating operation for every GPU-accelerated kernel.

Some of the research add-ons and extensions highlighted include:

  • TF Probability, a Python library built on TensorFlow that makes it easy to combine probabilistic models and deep learning on modern hardware.
  • TF Graphics for making graphics functions that are accessible to the community by providing a set of differentiable graphics layers, such as cameras, mesh convolutions, and a 3D viewer in TensorBoard.
  • Mesh TensorFlow, a language for distributed deep learning. The language is able to specify a broad class of distributed tensor computations.
  • TF Agents for designing, implementing, and testing new reinforcement learning algorithms.
  • TF Text, which provides a collection of text-related classes and operations. The library is useful in performing pre-processing tasks for text-based models.
  • Neural Structured Learning for training neural networks by leveraging structured signals in addition to feature inputs.
  • TensorFlow Quantum, a library for hybrid quantum-classical machine learning.

Another package that was released before this summit but was highlighted was the Keras Tuner. This package makes the work of hyperparameter tuning hassle-free. The package works with Keras, TensorFlow, and scikit-learn.