Deep Learning/Machine Learning Libraries — An overview

Original article was published by RAVI SHEKHAR TIWARI on Becoming Human: Artificial Intelligence Magazine

Deep Learning/Machine Learning Libraries — An overview

Deep Learning Library and Environment Set-up

If Artificial Intelligence is an art then researchers are artists. Artificial Intelligence is an art of transforming the real world’s numerical data, processing it and sending these transformed data through algorithms or set of formula. These algorithm and set of formula enables machine to learn and predict on seen and as well as unseen data.

In recent time we have seen substantial and as well as reliant implementations of subsets of the artificial intelligence with a promising result. In recent scenario of pandemic i.e. COVID-19 where human have become vulnerable to the virus, the implementation of subset of the artificial Intelligence is paving a way for us in health care, finance, measuring social distancing and numerous realtime implementation.

Fig 1. Subsets of AI

We all as a beginners faced challenges when we were taking first step into this innovative field — How to set up the environments ? Which library I should start with ? And numerous other question. So we will take our step into this by setting up our system.

1. Installation and Environment Setup

This phase can be described as a skeleton on top of it we will do all our implementation. It is like a notepad where we write, edit and execute our python scripts.

1.1. Installation of IDE’s

IDE which is known as Integrated Development Environment plays a very crucial role it gives us the power to manipulate, debug and do variety of action. There are numerous IDE is available which we can download free of cost in Mac, Linux or in Windows.

1.1.1 Anaconda

Anaconda basically is a collection of IDE which includes Spyder, Jupyter Notebook and more but we will going to use Spider and Jupyter Notebook. Visit the link and select the individual edition www.anaconda.com/product/individual and download the file. After downloading is complete you can follow the instruction according to the os which you are using www.docs.anaconda.com/anaconda/install/ and follow the instruction.

Artificial Intelligence Jobs

1.1.2. Google Colab

As we all know that processing, training and testing require huge processing power and acquiring this a big issue for beginners . As a result one of the big IT gains has given us the processing power with our Gmail account, it includes : 13 gb of ram, 12 hrs of session and 64 gb of disk space also we have ability to select GPU or TPU.

1.2. Installations of Packages

We are blessed with the innovation of the technology and as a result now we have pre-defined packages which will take care of the calculation by itself. There is numerous way by which we can install the packages but the most convenient way is to make a .bat and .txt file and run by the help of bash command in cmd or in anaconda cmd.

Step: 1. Create a file name as packages_installed.txt.

Fig 2. Sample Packages

Step: 2. Inside the file name as packages_installed.txt write the packages.(For example, I have listed few packages you can try and add more as per your requirements)

Step: 3. Change the extension of the file to .bat. or you can use pip install -r requirements.txt

Step: 4. Open cmd (anaconda cmd or windows) navigate to the folder by using cd command, then give the file path and press enter.

Fig. 3 Sample Packages Installation in cmd

If you are using Colab just u need to add ‘!’ In front of pip and run the cell, packages will be installed.

Fig. 4 Sample of Packages installation in Colab

Note : If you want to install the gnu version of Tensorflow and Keras, you can add suffix “ — gpu”

Trending AI Articles:

1. Fundamentals of AI, ML and Deep Learning for Product Managers

2. The Unfortunate Power of Deep Learning

3. Graph Neural Network for 3D Object Detection in a Point Cloud

4. Know the biggest Notable difference between AI vs. Machine Learning

1.2. Basic Deep learning Packages/libraries

1.2.1. Pandas

Pandas is a library which is used for data analysis and manipulation written especially for python. It is based on numpy, came into existence in 2008. It read data from files csv, txt, JSON and other as data frame and it can be used to preprocess these data as per our requirement.

Installation : pip install pandas

1.2.2. Numpy

Numpy is a python library which is used to manipulate the array, it can be used to handle huge data and has a working domain which includes Fourier transformation, matrices and linear algebra as well.

In python we have concept called list which is slow when we deal with humongous data, so there was requirement of the package which deals with humongous data at fast speed and hence numpy was developed. Numpy has speed 50x faster than a normal list because it store data in a continuous space. It is written in partially in python and in C/C++.

Installation : pip install numpy

1.2.3. Scikit learn

It is a library for python which consists of wide range of supervised and unsupervised algorithm like regression, clustering, ensemble, feature selection, SVM, Neural network, Parameter tuning ,etc.

Scikit Learn has been build upon SciPy which means we have to install SciPy before installing or using the Scikit learn. The prerequisites are numpy, Matplot, IPython, Sympy and Pandas as well.

Installation : pip install scikit-learn.

1.2.4. OpenCv

OpenCv is one of the most important library when it comes to Computer Vision with bindings C, C++, Java and Pythons. It enables us to read the images or video as an array which can be converted into to numpy array and further we can preprocess the dataset according to our need.

The OpenCv has state of the art algorithms which includes Harcascades, Convolutional Filters such as Laplacian, Sobel, Edge detection algorithms, feature matching as well as detection and tracking algorithm. It work at with with any visual devices with minimum human interaction.

Installation : pip install python-OpenCV

1.2.5. Visualisation Libraries

Python is a library which has wide variety of library such as Matplot, Seaborn, poorly and many more but two of the most widely used plotting library is Matplot and Seaborn.

Matplot gives us very flexible way of plotting 2D and 3D data as well. It supports all the popular charts ( scatter plot, pie graph, bar graph, Line graph) as well as spectrogram, signals, steam graphs and variety of other graph. It is very flexible library and we can customise it up to a very high extend.

Installation : pip install matplotlib.

1.2.6. Tensor

Tensor is most important data structure in data science, it is used to store gamut of data ranging from scalar (Rank 0), 1D-vector, (Rank 1), 2D-Vector (Rank 2) and as well as ND- metrics (Rank 3). These can be assigned under variable and can be used collectively ,individually or as a subset by slicing where index representation. Numpy and tensor are two way communicable data structure which means they can be converted in any one of them for faster manipulation of data.

Fig 5. Tensor Representation

Every tensor has its rank which is defined by its magnitude and directions (below table describe the rank and magnitude of tensor):

Almost all deep learning frame work uses tensor to manipulate their parameter. Framework like Pytorch, Tensorflow and Keras and dependent on the tensor for any update in the parameters. Tensor has give us tremendous power in terms of dealing with the humongous amount of data especially floating point numbers.

1.2.7. Tensorflow and Keras

Tensorflow is open source library which is extensively used with python for research and deployment. Tensorflow is written upon python, C++ and cuda, released in 2015. Tensorflow library is widely used across the globe but fraction of people uses it with C, Java, Javascript, Swift. Third party library are also available with C#, R, Matlab and many other languages.

Keras is open source software capable of running on top of tensorflow, Theano, Microsoft Cognitive toolkit or paidML. It is higher lever abstraction of tensorflow which can be use to develop and deploy deep learning models.

1.2.8. Pytorch

Pytorch is an open source library developed by Facebook. Pytorch is extensively used with python but it also support C++ interference. Pytorch is written upon python, C++ and cuda released in 2016. Pytorch is being used by top notch company such as Tesla, Uber and many more.

Special Thanks:

As we say “Car is useless if it doesn’t have a good engine” similarly student is useless without proper guidance and motivation. I will like to thank my Guru as well as my Idol “Dr P. Supraja”- guided me throughout the journey, from bottom of my heart. As a Guru, she has lighted the best available path for me, motivated me whenever I encountered failure or roadblock- without her support and motivation this was an impossible task for me.

Contact me:

If you have any query feel free to contact me on any of the below-mentioned options:

Website: www.rstiwari.com

Google Form: https://forms.gle/mhDYQKQJKtAKP78V7

Notebook for Reference:

Jupyter Notebook: https://jovian.ml/tiwari12-rst/installationguide

References:

Jovian: https://jovian.ml/docs/user-guide/install.html

Youtube: https://www.youtube.com/channel/UCFG5x-VHtutn3zQzWBkXyFQ

Tensorflow: https://www.tensorflow.org

Keras: https://keras.io

Pytorch: https://pytorch.org

Scikit-Learn: https://scikit-learn.org/stable/

Numpy: https://numpy.org

Pandas: https://pandas.pydata.org

Matplot: https://matplotlib.org

Seaborn: https://seaborn.pydata.org

Don’t forget to give us your 👏 !


Deep Learning/Machine Learning Libraries — An overview was originally published in Becoming Human: Artificial Intelligence Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.