Differentiable and Non-Differentiable OPs in Tensorflow Using only Python OS Module

Source: Deep Learning on Medium


Go to the profile of Mainak Dutta

I have come to know about this requirement while answering a question at StackOverflow earlier this year. I have decided to develop the code to fetch all the ops programmatically which will help the tensorflow users to write their code more efficiently. This comprehensive list is not present in the documentation of Tensorflow. This illustration used Tensorflow 1.13.

The complete project is present here.

The complete code is present here.

For illustration purposes, Differentiable Ops are those ops that have defined gradient and can be used for gradient-based learning. And Non-Differentiable Ops does not have a gradient defined.

The python code for fetching the list is pretty simple. We just had to know the installation folder of “Tensorflow”.

In my case the path was.

path1 = ‘/home/titanxpascal/Documents/cpuenv/lib/python2.7/site-packages/tensorflow/’

You will find the “tensorflow” folder in the directory of your virtual environment. In this case, the virtual environment name is “cpuenv” as can be seen.

Then we will try to find out all the directorty paths using the Os function “Walk”.

listi = os.walk(path1)

Walk gives both the name of the directory as well as the complete path of the directory and file names. We are only interested to find out the directory paths.

dirpaths = []
for dirpath, dirname,file in listi:
dirpaths.append(dirpath)

There is an amazing feature around Differentiable and Non-Differentiable variables. Differentiable variables are marked with ‘ ops.RegisterGradient’. And Non- Differentiable variables are marked with ‘ops.NotDifferentiable’.

Example –

@ops.RegisterGradient("FakeQuantWithMinMaxArgs")def def_FakeQuantWithMinMaxArgsGradient(op, grad):
&
ops.NotDifferentiable('ExtractGlimpse')
ops.NotDifferentiable('NonMaxSuppression')

Now, Now basically we try to find the phrases and related Ops name in the complete Tensorflow folder and save it into two different text files.

check_values = ['ops.NotDifferentiable', 'ops.RegisterGradient'] nondiff = [] 
diff = []

Lists nondiff and diff stores the name of Differentiable and Non-Differentiable Variables.

The below code fills the lists nondiff and diff by reading the texts of all the python or “.py” files present in the tensorflow root directory.

Now we save the lists into Text files using python file write.

These two files can be found at my Github Page — Github.com/Mainak431

Project LInk — Github.com/Mainak431/List-of-Differentiable — OPs-and-Non-differentiable-OPs — in-Tensorflow

You can follow me at my Linkedin- https://www.linkedin.com/in/mainak-dutta-182a03147/