MCT — Google AI Toolkit for Presenting ML Models

Original article was published by Mikhail Raevskiy on Deep Learning on Medium


MCT — Google AI Toolkit for Presenting ML Models

The Model Card Toolkit (MCT) is a set of tools to support developers in collecting information for model cards. The model sheet describes the ML model at different levels to increase the transparency of the approaches used. The model card includes information such as goals, architecture description, metrics, model constraints, and performance evaluation. MCT includes a basic interface for the model card. The researchers have published a Colab tutorial with an example of MCT use.

More about the library

The creation of a model card involves filling in a JSON schema. The diagram describes the fields that are included in the card. The MCT automatically populates the schema with model data from ML Metadata (MLMD). Model data is, for example, the distribution of classes in the data and statistics of the model’s performance.

The pipeline of MCT work. Source: Google AI

Researchers publish a ModelCard API to represent a JSON schema instance and render it as a model card. The user can choose which metrics and graphs to show in the final model card, including metrics that show cases of input data on which the model performs worse.

After the model card with key metrics and graphs has been created, the user can add information about the purpose of using the model, restrictions, and ethical remarks for using the model. This information allows third-party developers to decide if the model is suitable for their specific task.

An example of a model card created with MCT

Currently, MCT is available for all TensorFlow Extended (TFX) users or on Google Cloud Platform.