Detect COVID-19 with Deep Learning- A survey on Deep Learning for Pulmonary Medical Imaging

Original article was published on Deep Learning on Medium

Overview of Deep Learning

Deep Learning Algorithms tend to learn automatically from large amounts of data. The field of Radiology is the natural application of this field as it provides results based on information extracted from images. In medical imaging accurate diagnosis depends on image collection and its computer-aided diagnosis (CAD) mainly focused on the doctor’s past experience and knowledge. Various techniques have now been developed which assist doctors and professionals in understanding the features related to these diseases.

In the year 1993, deep learning was algorithm was first applied to medical imaging in which neural networks were used. At that time DL was not accepted in this field as it required more computational power and large data.

The detection criteria of medical image mainly consist of finding possible lesions (region/organ suffering damage) and tumors. Detecting them in pulmonary nodules is challenging mainly due to their varying shape, size, and density. Secondly due to similar texture features between two diseases.

Variety of lesions (in green boxes).

Deep learning NN was called cybernetics back in the 1940s. Hubel et al in 1962 studied the brain’s understanding and detecting of images. This model helped understand the functions of the brain. Historical perspective on networks is briefly described in the table below-

History on the Networks

Deep Learning in Medical Image Analysis, consists of 3 main parts-

Classification– Classify and stage disease severity (normal or abnormal). Binary or multiclass.

Detection– it is a pre-processing step of segmentation. Detecting the region of interest (ROI).

Segmentation– CNN is used for segmenting meaningful parts, organs, substructure, and lesion, and extraction of features. The below figure is a U-net published by Ronneberger et al became the most famous Convolution NN architecture for medical image segmentation. The left part uses convolution for image extraction and the right part does deconvolution for recovery of original image size.

The U-net

Deep Learning in Medical Pulmonary Image

Detecting Lung Cancer using Deep Learning:

Lung Cancer is the most severe and life-threatening cancer types and can be easily prevented if the Pulmonary Nodules in the lungs are detected early and diagnosed correctly. In this section, we will see how the Classification, Detection, and Segmentation of Pulmonary nodules are done using Deep Learning.

Pulmonary nodule Classification:

CNN’s has a great ability to self -learning and generalization and have therefore been widely used in nodule classification task (benign or malignant). Below is a nodule image network structure proposed by Netto et al for recognizing three types of nodules- Solid, semi-solid, and ground-glass opacity. In the final step, the nodule is divided into nodule and non-nodule using SVM. The structure of the classification network with CNN is in the below image. There have been various classification techniques proposed and implemented by a lot of great researchers using transfer learning and 3D CNN’s.

A Classification Network

Pulmonary nodule Detection:

Researchers have mainly focused on two-stage networks for nodule Detection, as high-performance ones must have high sensitivity and precision both. Below are a general detection process and a proposed deconvolutional structure using R-CNN’s for candidate detection and 3D DCNN for false-positive reduction.

Pipeline for Lesion Detection
Structure of Detection Network

Pulmonary nodule Segmentation:

The above-mentioned U-net architecture and unsupervised learning are widely adopted for segmentation tasks. Because the segmentation label is difficult to obtain, a weakly-supervised method to generate accurate voxel-level nodule segmentation has been proposed by researchers.

Fatal Lung Diseases and their Diagnosis with Deep Learning

Pulmonary Embolism

When an artery in the lung becomes partially or completely blocked, it results in a condition called Pulmonary Embolism (PE). About 650000 cases are occurring annually and it is the third most threatening disease. The traditional approach for its diagnosis by doctors is by carefully tracing each pulmonary artery for any suspects-which is known as CT Pulmonary Angiography (CTPA). Each image represents one slice of lung and it is difficult to differentiate and is a complicated task; as it may result in wrong diagnosis and high false-positive results. Thus, CAD comes to rescue, and the approach called Neural Hypernetwork is used. A lot of knowledge-based hybrid learning algorithms are used for better results. Following are the established approaches to this disease with Deep Learning-


This disease is common among children and is the main cause of death amongst them. Countries and regions with less medical advancement lack timely diagnosis of this disease. Early detection of this disease can save a lot of lives. The chest X-Ray examination is the most common method used; however, this method is laborious and entirely depends on the professional’s experience. A rough estimation is made on the observed tissues as the images of pneumonia are very similar. However, many templates matching algorithms in Deep Learning are used to carefully diagnose and distinguish between such tissues. CUDA and other CNN architectures have shown promising results. Below are the other famous methods proposed-


It is a chronic infectious disease transmitted by the respiratory tract. The pathogen ‘Mycobacterium tuberculosis’ invades the person’s body and causes this disease. Current diagnosis methods are mainly dependent on signs and symptoms, X-rays, and the sputum (saliva or mucus) Mycobacterium tuberculosis pathogen’s examination. CNN’s are popular for accurately classifying heterogeneous images. A lot of approaches to machine learning like multi-instance learning combined with DL architectures of RNN’s have achieved good results.

Interstitial Lung Disease (ILD)

This group of heterogenous non-neoplastic and non-infectious diseases is a result of pathological changes. Accurate classification of such diseases is important as they involve abnormal imaging patterns. A lot of methods in Deep Learning are proposed to accurately segment and detect this disease. As more and more common diseases are developing, a lot of deep learning models are emerging for biomedical image segmentation.

Existing Datasets for these diseases

The above-discussed diseases have open-source datasets available and provide a unique opportunity for trying existing models and developing new deep learning models. I am providing links to these datasets below.

Performances of the two pulmonary nodule datasets LIDC- IDRI, and LUNA16 is below-

Performances of the datasets


This newly emerging field of medical image processing has a lot of prospects for Deep Learning researchers and aspiring enthusiasts like us. This intersection of computer and medical fields can enhance and completely transform the traditional healthcare system. A sincere thanks to Jiechao Ma, Yang Song, Xi Tian, Yiting Hua, Rongguo Zhang, and Jianlin Wu for this extensive survey on Deep Learning for Pulmonary Medical Imaging. This is a consolidated guide defining the breakthroughs in this field and a detailed analysis of various datasets used. I hope this article inspires a lot of people to add more feathers in the Deep Learning hat. Thanks for reading!