Using Jupyter Notebook running on a remote docker container via SSH

In my research I work with Machine Learning/Deep Learning algorithms. As such, I find it useful to test different frameworks (such as Keras, PyTorch, Tensorflow…). This lead me to using Docker, and in particular ufoym/deepo image as it provides a very complete environment.

Furthermore, I use a remote machine with GPUs, where I installed Docker. When programming and testing some code snippets, I find it really useful to use Jupyter. I had previously used jupyter on my local machine running also on my local machine, but question here is how can I use it on my local machine but running on a remote machine. And on top of that, what if I am using a Docker container on my remote machine?

In this tutorial I explain how I answered this question.

My working environment

My working setting is usually as follows:

  1. As the host machine I simply use my regular laptop.
  2. I connect to the server of my department (remote) via ssh.
  3. I then connect to a specific machine within the server which has GPUs.
  4. Finally I start the docker container.

In my particular case, the machine I work on is the one labeled as GPU in the above picture, which happens to be only accessible from another machine within the server. Hence the double ssh… But in you case you might be able to connect directly to your GPU machine from the outer world.

It sounds like messy, but you just need to organise it once and for ever.

Pre-requisites

Docker

Make sure to have docker installed in your remote machine. If you are unfamiliar with Docker, please check some of the resources available (this and this).

GPU-support

If your remote machine has GPUs you might want to exploit this fact. Make sure you have the NVIDIA driver installed. You also need to install nvidia-docker. Note that installing nvidia-docker we automatically install the last stable release of docker-ce.

Setting up the connection

host — remote

Jupyter Notebook runs on a certain port on the machine. Hence, the basic idea would be to make that port reachable from your host machine. Luckily, ssh provides the -L option to specify port forwarding.

$ ssh -L <host port>:localhost:<remote port> user@remote

In my case I use port 9999 in both ends, namely <host port> = <remote port> = 9999.

remote — GPU

Here the approach is exactly the same as before.

$ ssh -L <remote port>:localhost:<GPU port> user@GPU

Again, I use <host port> = <remote port> = 9999.

GPU — docker

This step is slightly different. First, you need to create a docker container using some docker image of your preference. In my case, as aforementioned, I am using ufoym/deepo. Since I want GPU-support, I will use nvidia-docker to create the container.

$ nvidia-docker run -it \
-p <GPU port>:<container port> \
--name <container name> \
ufo/deepo bash

Note the option -p , which tells the docker container to do port forwarding. This way, we can access apps running in the docker on a certain port from the outside world. Here, I also use <host port> = <remote port> = 9999.

Running Jupyter

Well, once all the tunneling has been set up, we can start our jupyter app. We will use 0.0.0.0 ip and the same port we used when creating the docker container instance. Also, in my case, I use the option –allow-root , as I am root in my container and Jupyter won’t run unless I use this option.

$ jupyter notebook --ip 0.0.0.0 --port <container port> --allow-root

An extremely useful option when creating a container is -v, which allows you to access machine files from within the docker container.

Now, on my host, i simply go to localhost:9999 and voila, there you go.

Notes

Enable more ports

If you have a similar working environment, I would recommend to enable more ports for remote access. You can simply add those to your ssh command using -L <local port>:localhost:<remote port> . This way, if you ever happen to run other services on some ports inyour docker machine you can easily access them remotely.

Automate ssh login

As you add more options to your ssh command, it increases it size. An option is to define a host under ~/.ssh/config file. In my case I added a user <name>:

Host <name>
Hostname <remote IP>
Port <remote port for SSH (typically 22)>
IdentityFile <path to rsa key>
LocalForward 9999 127.0.0.1:9999

Any error?

A good way to test out if your tunneling is working, is to use http.server from python. This way you can check for every stage if the ports were correctly forwarded. Use the following command to run a simple http server on a particular port.

$ python -m http.server <remote port>

Note that this works for python3, for python2 versions use python -m SimpleHTTPServer <remote port> instead.

Source: Deep Learning on Medium