Producing Synthetic Data for Better Defect Detection

Can the training workflow and performance of CNNs for defects detection be improved by buttressing the training database with synthetic images of defective parts generated using GANs?

In the manufacture of high-end cinematographic lenses, a careful inspection needs to be performed on anodized aluminum housings of lenses to look for scratches on the surface of cases before they are delivered to the customers. To automatize this inspection process, traditional image processing algorithms are very limited because the anodized aluminum surface has a granular surface, when it is observed carefully. In order to provide a more general solution yet a robust result, a convolutional neural network (CNN) is used to carry out this inspection. However, training CNNs is challenging due to the low amount of available defect images.

To overcome these challenges, the proposed master’s thesis will investigate the use of Generative Adversarial Networks (GANs) to generate synthetic data for the minority classes for CNN training as suggested in the original paper of GANs. Those synthetic images from GANs will then be used to train a CNN for defect detection and the performance of the newly trained CNN will be measured by its classification accuracy on samples from real world.

The proposed research will involve three stages:

1. Finalizing optimizations to the hardware of the laboratory setup

2. do some basic testing to decide which of the following GAN approaches will be taken

   1. Transparent defect image

In this approach, scratches will be generated through GANs. The synthetic scratches from the GANs will be overlayed to different real scratch-free surface images to enlarge the training dataset for the defect detection CNN.

1. 2. Patch based/Inpainting approach

In this approach, a GAN which can generate patches of surfaces with scratches will be trained. The patches generated need to be able to be seamlessly integrated into a real surface. The approach is similar to the idea of Inpainting, with the masked area being filled by synthetic scratches instead of visually realistic pixels.

1. 3. Image-to-image translation

In this approach, a GAN will either turn a defect map(2D array indicating the desired locations of scratches) into a synthetic surface with scratches, or a real surface with scratches into a defect map. The second approach shows some resemblances to the task of semantic segmentation in the paper .

3. Training a CNN with a dataset generated from GAN and evaluate performance of the trained CNN with data unknown to the CNN, acquired from a variety of aluminum housing geometries and the laboratory setup with different settings.