A Step-by-Step Introduction to Starting nbdev — Exploratory Programming

Source: Deep Learning on Medium

A Step-by-Step Introduction to Starting nbdev — Exploratory Programming

A simplified Hello Word Example using nbdev

“I really do think [nbdev] is a huge step forward for programming environments”: Chris Lattner, inventer of Swift, LLVM, and Xcode and Swift Playgrounds.

Jeremy Howard does not stop impressing us with his great libraries, e.g., Fastai (a high-level APIs for using PyTorch).

Today (2/12/2019) Jeremy Howard and Sylvain Gugger released another awesome library “nbdev” which allows to fully develop a library in Jupyter Notebooks, putting all your code, tests, and documentation in one place.

That is, you now have a true literate programming environment, as envisioned by Donald Knuth back in 1983!

Exploring the notebook file format in the nbdev source code

nbdev: exploratory programming (from nbdev)

Exploratory programming is a programming technique based on observations. While coding, coders tend to spend a lot of time explore and experiment different APIs, data, algorithms, etc. Especially ML engineers who spend a lot of efforts changing the hyper parameters and record them somewhere — experiment management. Such process is easier when you code using REPL or notebook systems such as Jupyter Notebooks. But these systems are not as strong for the “programming” part. This is why people use such systems mainly for early exploring, and then switch to an IDE or text editor later in a project.

Features of nbdev (from nbdev)

The main purpose of nbdev is to bring the key features of IDE/editors into the world to notebooks where you do not have to worry about the entire lifecycle. Moreover, nbdev provides the following features:

  • Python modules are automatically created for you, following best practices such as automatically defining __all__ (more details) with your exported functions, classes, and variables
  • Navigate and edit your code in a standard text editor or IDE, and export any changes automatically back into your notebooks
  • Automatically create searchable, hyperlinked documentation from your code; any word you surround in backticks will by hyperlinked to the appropriate documentation, a sidebar will be created for you in your documentation site with links to each of your modules, and more
  • Pip installers (uploaded to pypi for you)
    Testing (defined directly in your notebooks, and run in parallel)
    Continuous integration
  • Version control conflict handling

Setting up your nbdev (Hello World Example)

Setting up nbdev is easy:

  • clicking here and this link will direct you to Github where all you have to do is just to give a repo name
  • After that, clone the repo and edit settings.ini file to match your requirements. Do not forget to uncomment the required lines (lib_name, user, etc.)
  • After each edit, run the command nbdev_build_lib, it is pre-installed with nbdev.

Then start Jupyter Notebook, if you are in the base directory, you should see the following:

Click on 00_core.ipynb and BOOM here where you will create your first module!

As you start creating cells and write code, do not forget to start each cell with #export, or #exports to show the source code in the docs.

And finally, you have two options, whether to place the following code in the last cell:

from nbdev.export import *
notebook2script()

Or to run the command: nbdev_build_lib

Do not forget to edit index.ipynb. This will be converted into your projects README file, and will also be the index for your documentation.

After running the command nbdev_build_lib a new directory will be created with the same name you set your lib_name in settings.ini. Notice exploratoryprogramming:

The contents of exploratoryprogramming:

Now you are set to create amazing deep learning projects! Please let me know if you have any questions or comments.

Resources