How to install TensorFlow GPU with CUDA Toolkit 9.0 and cuDNN 7.2.1 on AWS EC2 Ubuntu 16.04

AWS EC2 Specs

  • Instance Type: p3.2xlarge
  • AMI: Ubuntu Server 16.04 LTS (HVM), SSD Volume Type — ami-2a7d75c0
  • Volume Size: more than 20GB


Update system & Install Dependencies

# Update apt-get
sudo apt-get update
sudo apt-get upgrade
# Install Dependencies
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get install build-essential
sudo apt-get install cmake git unzip zip
sudo apt-get install python2.7-dev python3.5-dev python3.6-dev pylint
  • Install linux kernel header
uname -r
# install linux header
sudo apt-get install linux-headers-$(uname -r)

Download the NVIDIA CUDA Toolkit

Go to and download Installer for Linux Ubuntu 16.04 x86_64 deb[local]

This is step requires to go to NVIDA website and download it, I worked around this step by installing it locally then upload it the EC2 using FileZilla

Transfer CUDA-9.0 to EC2 using FileZilla

After transferring the CUDA to the EC2, execute the below commands

sudo dpkg -i cuda-repo-ubuntu1604-9-0-local_9.0.176-1_amd64.deb
sudo apt-key add /var/cuda-repo-9-0-local/
sudo apt-get update
sudo apt-get install cuda-9.0
sudo reboot

nano ~/.bashrc

# add those 2 lines at the end of the file then save it
export PATH=/usr/local/cuda-9.0/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64:$LD_LIBRARY_PATH

After that execute the bellow commands

source ~/.bashrc
sudo ldconfig

If you got nvidia-smi is not found then you have unsupported linux kernel installed.

Install cuDNN 7.2.1

Go to and download Membership required

After login Download the following:

cuDNN v7.2.1 Runtime Library for Ubuntu16.04 (Deb)
cuDNN v7.2.1 Developer Library for Ubuntu16.04 (Deb)
cuDNN v7.2.1 Code Samples and User Guide for Ubuntu16.04 (Deb)

Download Cudnn library

Transfer these files using FileZilla to the EC2 then execute the below commands:

sudo dpkg -i libcudnn7_7.2.1.38–1+cuda9.0_amd64.deb
sudo dpkg -i libcudnn7-dev_7.2.1.38–1+cuda9.0_amd64.deb
sudo dpkg -i libcudnn7-doc_7.2.1.38–1+cuda9.0_amd64.deb

Verifying cuDNN installation:

cp -r /usr/src/cudnn_samples_v7/ $HOME
cd $HOME/cudnn_samples_v7/mnistCUDNN
make clean && make

If cuDNN is properly installed and running on your Linux system, you will see a message similar to the following:

Test passed!

CUPTI ships with the CUDA Toolkit, but you also need to append its path to the LD_LIBRARY_PATH environment variable

nano ~/.bashrc

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/extras/CUPTI/lib64


source ~/.bashrc

Make sure you satisfied the below requirements:

TensorFlow GPU support

Install Tensorflow-GPU


pip3 install tensorflow-gpu


pip install tensorflow-gpu

Verifying TensorFlow scripts are running in GPU

import tensorflow as tf
with tf.device('/gpu:0'):
a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)
with tf.Session() as sess:
print (

If this script runs without giving any error you are using gpu version

Troubleshooting & References


Source: Deep Learning on Medium