Original article was published on Deep Learning on Medium
Experiment: Improving Deep Neural Network’s Segmentation Efficiency Of Medical Images By Using Image Features Explicitly
I am always curious about data, particularly images. Neural Networks fascinates me, so does an image. While I was working, I got many chances to explore Neural Networks, play with its architecture, use various types of models, understand the roles of hyperparameters, etc. etc. However, as I dived more, I got curious about the potential of data and how it can influence model learning and enhance its capabilities. Hence this time I decided to experiment purely with data instead of experimenting with the model. This blog talks in detail about my latest experiment.
Experiment: Augmenting image data in order to boost model segmentation capabilities. Augmentation is done by selective feature engineering suited to particular data. These are very basic ones like image negatives, gradients, and pseudo coloring. The features are prepared from the original image and added as an additional channel to it. I just wanted to curb my curiosity of knowing what happens if we pass them explicitly even when we know that lower layers of the model learns it. However, the fact needs to be admitted here is that a technique that might work well for one dataset, may or may not work well for others. Hence images are first analyzed to determine what feature engineering might enhance it and might help in improving NN’s segmentation capabilities. In order to conduct this experiment, I chose U-Net and trained two models. A baseline model trained using the grayscaled image and other, an experimental model where feature engineering is done on each image and added as an additional channel to it.
Dataset used: Chest X Rays containing 704 train and 96 test images. Cell membrane containing 30 train and 30 test images. DRIVE Digital Retinal Images for Vessel Extraction containing 20 train and 20 test images
Training dataset preparation: The newly generated features serves as additional channels to the input image.
- Image Negatives: Information that seems to be hidden in the original image, seems to contrast in the negatives and helps us to see the features more clearly to some extent. A simple intuitive decision was made, if it can help us better visualize a grayscale image, it will definitely help the model. Hence image negatives were taken as the first additional feature channel.
- Gradients: Next, image boundaries are identified so that model is able to learn more about the shape of the segments in correlation with an image as a whole. Although lower layers of neural networks are said to learn these features, I wanted to try whether giving them in as an additional input will help the model in some way or not. Hence I computed the gradient of an image to determine its skeleton.
- Pseudo coloring: For my final feature image I focused on pseudo coloring techniques that help us to assign colors to gray value based on specific criteria. By varying the number of colors and the span of intensity colors, one can quickly determine the characteristics of intensity variation in grayscale images. This is used by many people in the medical industry to visualize Xrays and medical images in a better way. Hence I decided to use this for my experiment as well.
- Superpixel: In one draft run, I also played with superpixels. However, for this experiment, I keep it out.
Once the individual features were prepared, they were stacked together channel-wise. This resulted in a six-channel input image.
Training and Result: The baseline and an experimental model were trained with a fixed number of epochs. We can see that the experimental model does slightly better than the baseline model. However, the result appears to be more significant when visualized.
If we analyze the baseline model with the new experimental model for Chest X-Ray/ Cell Membrane and Retinal vessel dataset, we can see that new model is able to generate finer masks, has better localization and generates no false-positive areas as compared to baseline model which generates broken/irregular masks, no smooth boundaries, and false-positive areas. This shows that my experimental model trained with explicit features performs slightly better as compared to the baseline.
Conclusions: It was a simple experiment to see whether giving manually extracted features explicitly will help model’s learning capabilities or not. From the results, it does appear that if we are able to manipulate data in a way such that it enhances the information contained in it, then it will surely improve NN’s learning capabilities.
I hope this little experiment was insightful. You can play and try any of your own ideas, even if it’s small. It will be a building block towards your ML learning. For any feedback and suggestions please feel free to comment or reach out to me at my email or find me on LinkedIn. I would love to hear from you!