Object Detection/Image Classification in QNX Platform

Source: Deep Learning on Medium

Overview on Darknet Framework

Darknet is a configuration based deep learning framework.You do not need to write the neural network architecture by using any programming language.Programmers only need to modify the configuration file to train , model & generate model.

Darknet is hosted at https://pjreddie.com/darknet/. Other famous open source is https://github.com/pprp/darknet-AlexeyAB.

Darknet support following features:

  • Object Classification.
  • Object Detection through YOLO v1/v2/v3.(Other detection algorithms such as SSD, RetinaNet etc. are not supported)
  • RNN (LSTM, GRU & Naive RNN only supported)
  • Style transfer
  • Image Segmentation

[Building Darknet Framework on QNX Platform]

1.set the QNX tool chain by executing qnxsdp-env.sh
2. download git clone https://github.com/achalshah20/darknet-cpp.git
3. Modified CmakeList.txt to disable CUDA & OpenCV(Please check the CMakeList.txt files)
4. place the qnx7_aarch64.cmake file
3. mkdir build_qnx
4. cd build_qnx
5. ../darknet-cpp/build_qnx$ cmake — verbose=1 -G “Unix Makefiles” -DCMAKE_TOOLCHAIN_FILE=../qnx7_aarch64.cmake ..
6. make

  1. Modified CmakeList.txt to disable CUDA & OpenCV(Please check the CMakeList.txt files)
    4. place the qnx7_aarch64.cmake file
    3. mkdir build_qnx
    4. cd build_qnx
    5. ../darknet-cpp/build_qnx$ cmake — verbose=1 -G “Unix Makefiles” -DCMAKE_TOOLCHAIN_FILE=../qnx7_aarch64.cmake ..
    6. make

[Architecture]

Deep learning pipeline can be broadly divided in to

  • Input data pipeline
  • Defining Model Architecture
  • Training
  • Debugging, Profiling & visualization
  • Performance Optimization
  • Deployment

Input data pipeline

[Input data pipeline in Darknet]

[Defining Model Architecture]:

In any neural network,we should define

  • Layers : convolution, max pooling, batch normalization,dropout,skip-connection, softmax etc.

[convolutional]
filters=16
size=3
stride=1
pad=1
activation=leaky

[maxpool]
size=2
stride=2

  • Activation Function : logistic, relu, elu, tanh, leaky,linear etc.
  • Optimization/Learning Algorithm: By default it uses Adam
  • Cost Function: SSE(Sum Squared Error),Smooth,L1 etc.

[cost]
type=sse

  • Anchor boxes (used for object detection)

In Darknet, model architecture is defined in the configuration file.Darknet supports all most all the layers such as Convolution,dropout,fully connected layer etc.You can check all the layers at https://github.com/pjreddie/darknet/blob/master/src/parser.c . If you are using some layers in your neural network,but they are not part of darknet,then you can not use your neural network architecture in darknet framework.Please refer the below link to know more about how to write configuration file of your neural network in darknet framework.

https://blog.paperspace.com/how-to-implement-a-yolo-v3-object-detector-from-scratch-in-pytorch-part-2/.

[Training]:

Hyper parameters(such as learning rate, momentum, decay rate, policy,batch size etc.) are defined in configuration file under net block.

Training in Darknet Framework

[Debugging, Profiling & Visualization]

Base version of darknet only have “ — visualize” option to see the CNN layers.There are few contributors who added layer visualization on top of Darknet base version.Please check https://github.com/schwittlick/ofxDarknet/tree/master/.

[Performance Optimization]

  1. Optimization in only CPU Configuration:

NNPACK is an acceleration package for neural network computations.
NNPACK aims to provide high-performance implementations of convnet
layers for multi-core CPUs.NNPACK is not intended to be directly used by machine learning researchers; instead it provides low-level performance primitives to be leveraged by higher-level frameworks, such as Caffe, Torch, MXNet, Theano, Tensorflow, and Mocha.jl.It’s hosted at :

https://github.com/shizukachan/darknet-nnpack.

https://github.com/digitalbrain79/darknet-nnpack.

Reference https://github.com/shizukachan/darknet-nnpack.

2. Optimization in Neural Network using Quantization Technique:

Quantization refers to the process of reducing the number of bits that represent a number. In the context of deep learning, the predominant numerical format used for research and for deployment has so far been 32-bit floating point, or FP32. However, the desire for reduced bandwidth and compute requirements of deep learning models has driven research into using lower-precision numerical formats. It has been extensively demonstrated that weights and activations can be represented using 8-bit integers (or INT8) without incurring significant loss in accuracy. The use of even lower bit-widths, such as 4/2/1-bits, is an active field of research that has also shown great progress.

The more obvious benefit from quantization is significantly reduced bandwidth and storage. For instance, using INT8 for weights and activations consumes 4x less overall bandwidth compared to FP32.
Additionally integer compute is faster than floating point compute. It is also much more area and energy efficient.

Quantization support in Darknet is hosted at https://github.com/AlexeyAB/yolo2_light#yolo2_light.Please refer this link for for more information.

[Deployment]

Deployment of darknet model on different platform

Darknet model can be converted in to tensorflow model by using darkflow tool.After conversion we can deploy the tf model directly on the target or we can use tensorflow serving feature.If on the target platform OpenCV is supported,then we can use it’s feature to load darknet model & do the prediction.