Model Training on Supervisely using AWS Cluster

Original article was published by Deepti Tiwari on Deep Learning on Medium


Now that I am done with my dataset, I need to have a neural network on which I will do the training.

Supervisely provides us with various pre-built neural networks. From the “+ADD” option at the top, we can select the required neural network.

Selected Neural Network is Highlighted.

List of added Neural Networks.

To start the training, press the “TRAIN” button in front.

And then you will end up with the following screen pop-up.

Where are the AGENTSSS!!

WHY THIS HAPPENED!!!???

Supervisely only gives models, DTL codes for augmentation and annotation, and many other things but resources to run it.

Supervisely acts as a manager in the cluster and we need to provide it with agents who have resources(RAM, GPU, etc).

Current List of Running Clusters.

In this project, we will train our model on a slave launched using AWS Cloud Services.

STEP1:

Create a new account, if you don’t have already and then go to the AWS Management Console.

Home Page
MENU for AWS MANAGEMENT CONSOLE

STEP2:

Find EC2 Services.

Scroll down to find the LAUNCH INSTANCE button, and select “Launch Instances option”.

STEP3:

A list of multiple OS (Amazon Machine Image: AMI) images will appear.

For supervisory to train our model, a slave must be Linux OS, with NVIDIA CUDA GPU, DOCKER, and NVIDIA-DOCKER. Therefore, I made the final call on the highlighted option.

This image will be used as a slave for our master node.

STEP4:

The next step after selecting an image is to select the instance type.

For the neural networks, we need an instance with GPU. So I filtered the list accordingly.

List of available instances option.

I selected g4dn.xlarge instance.

You may not be having a key-pair, therefore create a new one, otherwise, select the existing one.