Microsoft’s HummingBird-ML: A library for accelerating traditional machine learning models

Original article was published by Harsh Patel on Becoming Human: Artificial Intelligence Magazine

Microsoft’s HummingBird-ML: A library for accelerating inference with traditional machine learning models

Hello Guys. Today’s topic of our discussion is HUMMINGBIRD-ML.

We all have used traditional ML Models like Linear Regression, Logistic Regression, Decision Tree, etc, in the initial stages of our way in the field of Machine Learning.

Unfortunately, traditional ML libraries and toolkits (such as Scikit-Learn, ML.NET, and H2O) are usually developed to run on CPU environments. Which gives lack in performance of a model.

they do not use abstraction such as tensors to represent their computation. The lack of this extraction means that for these frameworks to make use of hardware acceleration, one would need to have many implementations ((for each operator) x (for each hardware backend)) which does not scale well. This means that traditional ML is often missing out on the potential accelerations that deep learning and neural networks have.

Big Data Jobs

So, How HummingBird help in this situation?

Frameworks like TensorFlow, PyTorch, and ONNX Runtime are built on the idea of individual units and have tensors as their basic computational unit. These frameworks can run efficiently on hardware accelerators like GPUs and their prediction performance can be further optimized with compiler frameworks such as TVM(TVM is an open source deep learning compiler stack for CPUs, GPUs, and specialized accelerators).

Hummingbird compiles traditional ML pipelines into tensor computations to take advantage of the optimizations that are being implemented for neural network systems. This allows users to seamlessly leverage hardware acceleration.

HummingBird converts scikit_learn, ONNX.ML, etc models to PyTorch framework so that it can run with GPUs and attain a great performance.

This first open-source release of Hummingbird currently supports converting the following trees to PyTorch:

Scikit-learn:

Tree-based operators

  • DecisionTreeClassifier
  • DecisionTreeRegressor
  • ExtraTreesClassifier
  • ExtraTreesRegressor
  • GradientBoostingClassifier
  • GradientBoostingRegressor
  • HistGradientBoostingClassifier
  • HistGradientBoostingRegressor
  • IsolationForest
  • RandomForestClassifier
  • RandomForestRegressor

Trending AI Articles:

1. Fundamentals of AI, ML and Deep Learning for Product Managers

2. The Unfortunate Power of Deep Learning

3. Graph Neural Network for 3D Object Detection in a Point Cloud

4. Know the biggest Notable difference between AI vs. Machine Learning

Linear methods

  • LinearRegression
  • LinearSVC
  • LogisticRegression
  • LogisticRegressionCV
  • SGDClassifier

SVM

  • NuSVC
  • SVC

Classifiers: Other

  • BernoulliNB
  • GaussianNB
  • MLPClassifier
  • MLPRegressor
  • MultinomialNB

Preprocessing

  • Binarizer
  • Normalizer
  • OneHotEncoder
  • RobustScaler
  • MaxAbsScaler
  • MinMaxScaler
  • StandardScaler

Matrix Decomposition

  • PCA
  • KernelPCA
  • TruncatedSVD
  • FastICA

Feature Selectors

  • SelectPercentile
  • SelectKBest
  • VarianceThreshold

Feature Pre-processing: One-to-One

  • SimpleImputer
  • MissingIndicator

Feature Pre-processing: Other

  • PolynomialFeatures

LightGBM:

  • LGBMClassifier
  • LGBMRanker
  • LGBMRegressor

XGBoost:

  • XGBClassifier
  • XBGRanker
  • XGBRegressor

ONNX.ML:

  • ArrayFeatureExtractor
  • LinearClassifier
  • LinearRegressor
  • Normalizer
  • Scaler
  • TreeEnsembleClassifier
  • TreeEnsembleRegressor

Have a look at the performance and efficiency difference between scikit-learn model, Pytorch with CPU as resource and Pytorch with GPU as hardware accelerator.

That’s all about HummingBird, in upcoming months Microsoft is planning to introduce some more supports to HummingBird.

So Stay Tuned, Thank You!

Don’t forget to give us your 👏 !


Microsoft’s HummingBird-ML: A library for accelerating traditional machine learning models was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.