A Practice of Fabric Defect Detection by Using CNN

Original article was published on Deep Learning on Medium

In finally, our model consists of 7 layers which include on:

  • 1st: Convolution-layer with 16 filters 7×7 stride 2×2 max-pooling
  • 2nd: Convolution-layer with 32 filters 5×5 stride 2×2 max-pooling
  • 3rd: Convolution-layer with 64 filters 3×3 stride 2×2 max-pooling
  • 4th: Convolution-layer with 64 filters 3×3 stride 2×2 max-pooling
  • 5th: Fully-connected-layer with 64 nodes
  • 6th: Fully-connected-layer with 64 nodes
  • 7th: Fully-connected-layer with 1 node

Basically, the last layer is the model output. For each layer, it mostly uses ReLu as the activation function since it does not cause the vanishing gradient problem except the output layer prefers using the sigmoid function. Furthermore, we avoid overfitting by adding dropout and batch normalization. For model Compilation, based on computational time, stochastic gradient descent (SGD) was set as an optimizer and binary cross-entropy as the loss function.

from tensorflow.keras import datasets, layers, models, optimizers, regularizers, callbacks
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
def create_model(image_shape=(256, 4096, 1), print_summary=False):
# Initial model
model = models.Sequential()
# CONV layer: filter 16, stride 7x7
model.add(layers.Conv2D(16, (7, 7),input_shape=image_shape))
model.add(layers.BatchNormalization())
model.add(layers.Activation("relu"))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Dropout(0.25))
# CONV layer: filter 32, stride 5x5
model.add(layers.Conv2D(32, (5, 5), padding=”same”))
model.add(layers.BatchNormalization())
model.add(layers.Activation("relu"))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Dropout(0.25))

# CONV layer: filter 64, stride 3x3
model.add(layers.Conv2D(64, (3, 3), padding=”same”))
model.add(layers.BatchNormalization())
model.add(layers.Activation("relu"))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Dropout(0.25))

# CONV layer: filter 64, stride 3x3
model.add(layers.Conv2D(64, (3, 3), padding=”same”))
model.add(layers.BatchNormalization())
model.add(layers.Activation("relu"))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Dropout(0.25))

model.add(layers.Flatten())

# Fully connected layer -> nn layer with 64 nodes
model.add(layers.Dense(64))
model.add(layers.BatchNormalization())
model.add(layers.Activation("relu"))
model.add(layers.Dropout(0.25))
# Fully connected layer -> nn layer with 64 nodes
model.add(layers.Dense(64))
model.add(layers.BatchNormalization())
model.add(layers.Activation("relu"))
model.add(layers.Dropout(0.25))

# Output layer
model.add(layers.Dense(1, activation="sigmoid"))

# Set model compiler
model.compile(optimizer="SGD", loss="binary_crossentropy", metrics=["accuracy"])

# Show the CNN model detail
if print_summary:
model.summary()

return model

The summary of the model is:

Layer 1st to Layer 3rd
Layer 4th to Layer 6th

For a description of each layer,

  • Conv2d layer. Convolution layer
  • Max pooling layer. To reduce the size of the matrix.
  • Activation layer. The output function for the previous layer
  • BatchNormalization layer. Normalize by re-centering and re-scaling
  • Dropout layer. To avoid overfitting.
  • Dense layer. The fully connected layer
  • Flatten layer. Flatten the convolution layer into a vector.

4. Train and Evaluate the CNN model

To efficiently measure the model performance. We use the stratified k-fold method, where the algorithm consists of test and train cycles as followed:

Fig 4: Stratified k-fold Partition. Source

For stratified k-fold, each fold separates the dataset to be two groups:

  1. Training dataset. Use in the CNN mode training.
  2. Validate dataset. Evaluate the model accuracy that had been trained by the training dataset.

In each iteration, we have to train the model several times. For simplifying the process, let’s write a train function:

def train_model(model, xtrain, ytrain, xval, yval, n_epoch, batch_size):
# Train CNN model
# Batch size to reduce memory usage
# Set early stopping to avoid overfitting

earlystopping = EarlyStopping(monitor="val_accuracy", patience=2)
history = model.fit(xtrain, ytrain, epochs=n_epoch, batch_size=batch_size, validation_data=(xval, yval), callbacks=[earlystopping])
return history

During training, we use the callback function named earlystopping to halt the training process when the model does not improve. We set a parameter with 2 epochs patience threshold, meaning if the validation accuracy is not able to increase in 2 iterations, the training process will automatically stop. The function helps us save time.

Now, we ready to evaluate the model:

from sklearn.model_selection import StratifiedKFold# Set number of split
kfold_splits = 4
# Set number of epoch
n_epoch = 10
# Set batch size
batch_size = 10
# Create StratifiedKFold
skf = StratifiedKFold(n_splits=kfold_splits, shuffle=True)
for index, (train_indices, val_indices) in enumerate(skf.split(X, y)):

print(“Training on fold {}/{}…”.format(index+1, kfold_splits))
# Declare x train and x validate
xtrain, xval = X[train_indices], X[val_indices]

# Declare y train and y validate
ytrain, yval = y[train_indices], y[val_indices]
# Print number of class portion
print(“ytrain: number of samples each class: {}”.format(np.unique(ytrain, return_counts=True)))
print(“yval: number of samples each class: {}”.format(np.unique(yval, return_counts=True)))

# Clear the model
model = None
# Create cnn model
model = create_model()
print(“Training new iteration on {} training samples, {} validation samples, this may be a while…”.format(xtrain.shape[0], xval.shape[0]))

# Train CNN model
history = train_model(model, xtrain, ytrain, xval, yval, n_epoch, batch_size)
print(“— — — — — "*5)

Since the model is evaluated, we can calculate the average of the accuracy and loss which will be shown in the result section.

5. Export CNN model

To be able to export the model, we need to alter the code in that we have split the dataset into a testing set and a training set before the training process. The model will not be aware of the test set until the final evaluation process. In contrast, the validation set is used to tune the model in the training process.

from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.15, random_state=42)print(“y_train: number of samples each class: {}”.format(np.unique(y_train, return_counts=True)))print(“y_test: number of samples each class: {}”.format(np.unique(y_test, return_counts=True)))

In the process, we include a callback named ModelCheckpoint. It helps us save the best performer when val_accuracy has improved.

cnn_model = None
cnn_model = create_model(image_shape=(256, 4096, 1))
# Define the callback function
earlystopping = EarlyStopping(monitor="val_accuracy", patience=2)
filepath = project_path + “/model/weights_best.hdf5”
checkpoint = ModelCheckpoint(filepath, monitor="val_accuracy", verbose=1, save_best_only=True, mode="max")
callbacks_list = [checkpoint, earlystopping]
# Train model
cnn_model.fit(X_train, y_train, batch_size=10, epochs=10, validation_split=0.2, callbacks=callbacks_list)

We exported the model weights into a file weights_best.hdf5. The requirements for model reconstruction are:

  1. Model. The original model from create_model()
  2. Weights. Neural connection weights which produced after trained by the dataset

(Optional) Import model into the edge device

Fig 5: Raspberry PI with a camera. Source

The application can be used in the manufacturing process to detect the fabric defect by using our model based on the Edge AI concept. Although we don’t have the devices, this article can guideline the procedure which itself is not a complex implementation. Let us assume that a Raspberry PI connected with a camera or image feed is provided. Since we need to use the same CNN model, the prerequisite program of Raspberry PI should be the same as our article (e.g. Python, TensorFlow, etc.) as you can read in How-to-install-TensorFlow-on-Raspberry-Pi. Also, dependencies for using camera devices acquire to install such as PiCamera, OpenCV, etc. Read here: Accessing-the-Raspberry-Pi-Camera-with-OpenCV-and-Python. For the predictive program, it should consist of create_model(), model.fit(), and parts of reading the input image from the camera.

For the example code:

# DefectDetectionPrdictive.py# Import the necessary dependency
from tensorflow.keras import datasets, layers, models, optimizers, regularizers, callbacks
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
from picamera.array import PiRGBArray
from picamera import PiCamera
...# Initial create CNN, Same function as 3.
model = create_model()
# Apply weights to CNN
model.load_weights("weights_best.hdf5")
# Initialize PiCamera
camera = PiCamera()
# Capture image from camera
rawCapture = PiRGBArray(camera)
...# Grab an image from the camera
camera.capture(rawCapture, format="gray")
# Predict the input image
print(model.predict(image))
...

To compile the program:

python3 DefectDetectionPrdictive.py

Note — This part is not a real demonstration. Therefore, errors may occur. This is simply the concept of such implementation. Furthermore, the program can be adjusted depending on the proposed system specification. This idea of its application in edge device implementation for the smart factory has been discussed extensively in various research. For instance, D. Huang, et al., 2018 presented the architecture of the smart factory in the defect detection systems.

Result

The graphs below illustrate the performance of the model between each training epoch.

Our model accuracy: