Behavioral Cloning

Original article was published by Kushal B Kusram on Artificial Intelligence on Medium

Step 2b. Ending the data collection phase

Step 2b
Click on Pause button to end the data collection phase. The simulator will replay the entire driving session again while displaying a percentage of total data that was driven. Once the simulator replays the entire drive and done collecting the data, the simulator returns to where drive had ended. You may now close the simulator.


The heart of behavioral cloning here is the architecture of the network. A good architecture coupled with good data shall determine whether the car will remain on the track or not during autonomous mode.

The theory behind network has already been discussed. The network is developed in and consists a generator function to feed images to the network in batches in case the entire data does not fit in the memory. The generator also includes code to function as an augmentation engine which helps to include images from left and right camera along with a correction angle, thus allowing them to be used for training as though viewed from the center camera.

Once training is complete, the code shall export a trained model. The saved model file can be used for inference.

The parameters laid out during training are a batch size of 32, number of total epochs being 1 (can be increased with more data) and uses Adam optimizer.

Note: The amount of data that was used to train the demo model in the repository was very brief and the parameters were laid accordingly. As a result you may observe in the video (at the end of this article) that the car swerves outwards while maneuvering difficult curves.


The objective of is to load the trained model, save images from an inference run and handle the Flask app which is used as a communication mechanism between the model and simulator.

In order to begin an inference session:

Step 1
Execute the with path where model is saved as an argument, wait for the model to load and Flask app has been initiated.

Step 2
Start the simulator and initiate autonomous mode. If step 1 has successfully completed and the socket to communicate model output with the simulator has been setup then you shall see the car drive by itself around the simple track.