The challenges and limits of today’s technology, working circumstances, and other factors all significantly affect the quality of final products during the industrial production process. Surface defects are the most visible indicator that a product’s quality has been affected. As a result, detecting surface defects in products is crucial to ensuring a high qualification percentage and reliable performance. During processing, some unexpected defects emerge on the surface of parts in industries and manufacturing plants. Surface defects impact not just the aesthetic of the final product but also its quality to some extent.
This article aims to provide a comprehensive overview of surface defects and to go into more detail regarding the significance of surface defect detection, its methods, and its industrial applications. Additionally, it looks for a link between this technique and deep learning. Last but not least, this study exemplifies the SAIWA group’s distinctive approach to surface defect detection.
Read Also: What is anomaly detection, and why you need it?
What does surface defects mean?
Surface defects are boundaries or surfaces that divide a material into regions where each region has the same crystal structure but a different orientation. Surface defects are usually caused by surface finishing methods, including embossing, as a result of degradation caused by weathering or cracks caused by environmental stress; Defects may also occur during the processing and use of metals in service.
When defects occur early in the processing chain, they usually cause processing problems or initiate failures at later stages. Some of these defects are examples of complex metallurgical, chemical and physical reactions that metals undergo during this processing and sometimes it is impossible to avoid them.
Surface defects may cause corrosion and damage to the coating. The presence of surface defects can widely change the corrosion resistance and mechanical properties of materials. A combination of corrosion and cyclic loading can potentially destroy materials, leading to an unpredictably short useful life and the release of harmful corrosion products.
Perforation can be initiated by a small surface defect such as a scratch or local change in the composition or by damage to a protective coating. Due to its physical and chemical properties, various surface defects inevitably occur in production areas, including the steel strip production process. Defects not only destroy the appearance of steel strips, but also destroy their important functional characteristics such as corrosion resistance, wear resistance and fatigue strength. Unlike polished surfaces, they show more resistance to punctures.
How many types of surface defects are there?
There are several types of surface defects, some of which are described below:
Material Surface
The material’s external dimensions represent the surfaces where the lattice suddenly stops. The surface atoms no longer have the correct coordination number, and atomic bonding is disturbed. The material’s outer surface may also be highly rough, including microscopic notches, and be considerably more reactive than the bulk of the substance.
Grain Boundaries
Grain boundaries are another type of surface defect. In polycrystalline materials, an interface or grain boundary exists when a border separates two tiny grains or crystals with different crystallographic orientations.
Stacking Faults
A stacking fault is a type of planar defect resulting from a defect in the stacking sequence of atomic planes in a crystal.
Twin Boundaries
A twin boundary is a type of surface defect in which two different atomic arrangements are separated by a twin boundary, with one side of the atomic arrangement having the same mirror orientation as the other.
AI in Studying Surface Defects in Crystals
By gaining deeper insights into the defects, materials scientists can explore their effects on material properties, paving the way for innovative applications. AI assists in quantifying defect density, understanding defect size distribution, and evaluating their impact on structural, electrical, and optical properties. This comprehensive analysis is vital for enhancing the performance and reliability of materials used in various high-tech applications.
Quantifying and Analyzing Defects
Quantifying and analyzing surface defects in crystals represent a pivotal aspect of AI applications in materials science. AI is instrumental in assessing the defect density, defect size, and their distribution, providing valuable insights that were once challenging to obtain through traditional methods. One of AI’s primary contributions is automating defect quantification. This quantitative data is vital for assessing the health of a material and making informed decisions regarding its use.
AI aids in understanding distribution of surface defects in crystals. By analyzing data from different regions, researchers can identify patterns and trends. These insights enable them to assess the uniformity of defects and their potential impact on material properties.
Predicting and Controlling Defect Formation
Predicting and controlling defect formation in materials is a critical aspect of materials science, and AI brings unprecedented capabilities to this domain. AI is employed to forecast the occurrence of defects during crystal growth processes and to implement strategies for controlling defect generation. Strategies for controlling defect generation are multifaceted. AI assists in optimizing growth parameters to create conditions less conducive to defects. Moreover, AI-driven feedback control systems can continuously monitor the growth process and make real-time adjustments to mitigate defect formation.
In the world of materials science, the integration of AI for predicting and controlling defect formation represents a significant leap forward. It enhances the reliability and performance of materials, making them more suitable for high-tech applications in fields such as electronics, optics, and renewable energy.
Surface defects in crystalline materials
Surface defects are boundaries that have two dimensions and are usually separate areas of materials that have different crystal structures or crystallographic orientations. These defects include:
- External surfaces: The external dimensions of the materials represent the surfaces where the network ends abruptly. The outer surface may also be very rough, may have small cuts, and may be more reactive than the bulk of the material.
- Grain boundaries: the direction of atomic arrangement or crystal structure is different for each adjacent grain. At some points of the grain boundary, the atoms are so close that they create a compression region, and in other regions they are so far apart that they create a tension region.
- Stacking faults: These occur in cubic metals and indicate an error in the stacking sequence of packed plates. Accumulation faults interfere with the sliding process.
- Twin boundaries: The twin boundary is a special type of grain boundary that has a special mirror lattice symmetry throughout. Twin boundaries interfere with the slip process and increase the strength of the metal. The movement of twin boundaries can change the shape of the metal.
The effectiveness of the surface defects in interfering with the sliding process can be measured from the surface energies. Energetic grain boundaries are much more effective at blocking dislocations than stacking faults or twin boundaries. Surface defects are usually observed at the boundary between two grains or between small crystals in a larger crystal. This is due to the slightly different orientations that the rows of atoms in two different grains may be placed in, which leads to mismatches at grain boundaries.
The outer surface of a crystal is also technically a surface defect because atoms on the surface adjust their positions to accommodate the absence of neighboring atoms outside the surface.
What is surface defect detection?
Advanced industrial systems require improved product performance, and manufacturing quality control is becoming more critical. Defects, such as scratches, stains, or holes on the product’s surface, on the other hand, negatively impact the product’s performance as well as its appearance and user-friendliness. An excellent way to lessen the adverse effects of product flaws is through defect detection. The process of quality control and assurance includes surface defect detection. It is part of quality control and monitoring stages in which its main job is to detect the surface defects in the objects.
Surface defect detection methods
There are two kinds of methods for surface defect detection
Manual surface detection
Manual detection is mainly employed as the method for surface defect detection. However, the accuracy rate of manual defect detection could be higher and is highly influenced by the inspector’s subjective variables and work experience. The human eye cannot see a minor flaw; hence manual inspection is not appropriate when some inspection settings are damaging to human health. A vital component of industrial product detection of surface defect in recent years has been the progressive replacement of manual defect detection by surface defect based on visual perception.
Non-contact automated surface detection
A non-contact automated detection method is surface defect detection based on visual perception technology. It can operate for an extended period in a challenging manufacturing environment and has high precision and accuracy.
Surface Defect detection based on deep learning
Surface defect detection of industrial goods based on visual perception may be broadly split into deep learning-based surface defect detection and traditional image processing-based surface defect detection, with the latter being the central topic of this article.
Deep learning-based surface defect detection for industrial products requires automated feature extraction from vast data. Although this automated feature extraction technique is very environment-adaptive, it needs many training data. Due to the low likelihood of defective samples occurring in the natural industrial setting, it will be challenging to collect defective samples, and defect labeling demands a lot of personnel and material resources.
The versatility of deep learning-based defect detection allows the network to find specific problems based on the data set. Additionally, similar networks may be built using the network parameters learned for one network to achieve high success rates for surface defect identification. Additionally, a specific code is not required to train various defects. Labeled data provide a highly adaptable defect detection algorithm for various defects with the proper network.
Surface Defect Detection for Industrial Products
In industrial applications, the capacity to detect flaws is crucial. Defect detection is necessary to ensure that a manufacturing process is controlled and performed as it should. Appropriate remedial measures can be taken to guarantee that process performance stays adequate depending on the kind and severity of the defects. These tasks might be anything from changing a machine’s tool to maintaining other components. Defect detection may be seen as an introduction to the machine learning as a service maintenance process of diagnosis.
Defect detection is a vital step in the inspection process that determines whether to accept or reject a component provided by a supplier or manufactured by a process. It can also facilitate part repair and rework, minimizing material waste. If flaws are found early enough, specific manufacturing processes contain a feedback control mechanism that may be utilized to stop the creation of defects. Building process models that can be applied to process optimization also requires defect detection.
Defect Prediction and Root Cause Analysis
Looking forward, AI has prospects beyond just surface defect detection:
· Predictive models could forecast defects before they appear based on process conditions and equipment telemetry sensing. This enables prevention.
· Diagnostic techniques can help uncover root causes of defects by learning correlations between defects and manufacturing variables.
· This could reduce costs through proactive optimization of processes, yield improvements, and focused corrective maintenance where needed.
Essential challenges in surface defect detection
There are two critical challenges regarding surface defect detection:
Small Sample Problem
Deep learning methods are currently widely used in various computer vision tasks, and Surface defect detection is generally considered as its special application in the industrial field. In the traditional understanding, the reason why deep learning methods cannot be directly applied to Surface defect detection is that in the real industrial environment, there are very few examples of industrial defects that can be presented.
In many real industrial scenarios, there are only a few defective images, and for the small sample problem, which is one of the key problems in industrial-level fault diagnosis, there are 4 solutions:
Enhance and generate data
The most common method of expanding the defect image is to use several image processing operations such as mirroring, rotation, translation, filtering and contrast adjustment on the original defect samples to obtain more samples. Another method that is more common is data synthesis, in which individual defects are usually combined and placed on normal or non-defective samples to form defective samples.
Pre-network training and transfer training
In general, the use of small samples for training deep learning networks can easily cause overfitting, as a result, methods based on pre-training networks or transfer learning are one of the most common methods for samples.
Logical design of network structure
By designing a reasonable network structure, the need for samples can be greatly reduced. According to the dense sampling theorem to compress and expand small sample data, CNN can be used to directly classify the features of dense sampling data. Compared to the original image input, input compression can greatly reduce the network demand for samples. Also, the surface defect detection method based on the twin network can be considered as a special network design and can greatly reduce the sample requirement.
Unsupervised or semi-supervised method
In the unsupervised method, only normal samples are used for training, so there is no need for defective samples. The semi-supervised method can use unlabeled samples to solve the problem of network training in case of small samples.
Real-time Problem
Data annotation, model training, and model inference are the critical components of the deep learning-based defect detection methods used in industrial applications. Real-time in practical industrial applications gives model inference additional consideration. Currently, most defect detection methods focus on the precision of classification or identification while giving little thought to the effectiveness of model inference.
Importance of surface defect detection
Manual examination for product quality control was the most comprehensive approach in manufacturing until the last decades. Then, automated surface inspection (ASI) was introduced using hardware and software technologies and has developed quickly. More factories have started to employ embedded machines for product inspection in response to demands for lower labor costs and enhanced examination efficiency. ASI platforms use particular light sources, industrial cameras to capture the images of the product surface, and machine/computer vision technologies to filter out defective products, which can reduce labor significantly. Therefore, high-performance cross-product ASI algorithms are urgently needed in manufacturing.
What is Defect Detection Efficiency?
The percentage-based Defect Detection Efficiency (DDE) of a phase is the ratio of defects found in that phase to all defects found in that phase. DDE is used to evaluate the effectiveness of each phase.
Why use Defect Detection Efficiency?
Unlike system testing and acceptance testing, which mostly involve human testing, unit testing and integration testing are often fully automated. Finding defects in the first two stages would be useful because manual testing is expensive. In addition, if a bug is found at a later stage, it has to be retested at earlier stages. Obviously, the sooner a problem is discovered, the better; the test cycle is shorter, fewer resources are needed, and costs are reduced. DDE is a measure that helps us understand the effectiveness of our testing procedures.
What is unsupervised learning?
Supervised machine learning techniques are used in the vast majority of applications. By manually categorizing the collected data, supervised learning requires that we provide the model with ground truth data. Collecting and labelling data for a production line can be problematic, as there is no way to collect every variation of crack or dent on a product to ensure correct identification by the model.
Unsupervised machine learning methods allow you to identify patterns in a data set without using pre-labelled results, and to understand the underlying structure of the data in situations where it is difficult to train the algorithm normally. Unlike supervised learning, the training process requires less effort because we expect the model to find patterns in the data with a greater tolerance for variation.
How is anomaly detection affected by unsupervised learning?
Anomaly detection in machine learning is related to the problem of error detection. Even if we don’t rely on labelling, there are several unsupervised learning techniques that try to organize the data and provide pointers to the model.
- Clustering is the process of combining unlabeled instances into groups based on similarities. Recommendation engines, market or customer segmentation, social network analysis, or search result clustering are all common uses of clustering.
- Finding recurring patterns, connections or interactions in databases is the goal of association mining.
- Latent variable models are created to model the probability of distribution using latent variables. It is most commonly used to prepare data, reduce the number of features in a dataset, or split a dataset into different parts depending on the features.
By choosing this use case on the assumption that the image labels cannot be known in advance during training. Since training is done using an unsupervised technique, only a test dataset is labelled to assess the accuracy of the model’s predictions.
Advanced Model Architectures
Innovative neural network architectures suit the spatial nature of surface data:
· Graph neural networks directly model irregular 3D geometry like point clouds instead of rasterizing into grids. This avoids information loss.
· Transformers are emerging as an alternative to CNNs, using attention mechanisms to learn dependencies between surface locations.
· Autoencoders leverage unsupervised pretraining to map inputs into lower-dimensional latent representations useful for anomaly detection.
· Generative adversarial networks can hallucinate realistic defect-free surfaces, with the generator learning to remove defects in a self-supervised fashion.
· Combining these modern architectures expands the frontiers of surface defect detection.
Emerging Data Sources
Beyond conventional camera imagery, researchers are exploring alternative sensing modalities for surface defect detection:
· Hyperspectral imaging captures hundreds of narrow bands across the electromagnetic spectrum. The rich spectral signatures enable detecting subtle surface defects unnoticeable in RGB images. However, high-dimensional data poses modeling challenges.
- 3D scans from structured light systems or LiDAR provide direct shape measurements to identify depth anomalies on surfaces. But denser scans come at a cost of slower acquisition.
· Infrared thermography measures thermal patterns, exploiting traits like abnormal heat dissipation around subsurface defects. But background temperature variation can complicate analysis.
· Acoustic and ultrasonic waves also discern discontinuities below surfaces based on vibration changes. However, coupling the signal requires direct physical contact.
· Each modality provides a unique signature sensitive to certain defect types. Fusing multiple data sources can combine their strengths for enhanced defect screening.
What deep learning techniques does Saiwa use in its surface defect detection service?
The ultimate objective of saiwa surface detect detection service is to provide a simple user interface for data-driven algorithms calibrated for various surface defects. The deep learning algorithms will learn from the available datasets in the literature. These are:
- The Kolektor Surface-Defect (KSDD) dataset is constructed from images of defective production items provided and annotated by Kolektor Group d.o.o… This Group provided and annotated images of faulty production products, which were used to create the dataset. In a real-world scenario, the images were shot in a controlled industrial environment.
- The KSDD2 dataset from the same Group is made up of images of defective manufacturing products given and annotated by Kolektor Group d.o.o. On the item’s surface, many forms of defects were discovered. The images were taken in a controlled industrial setting.
- Severstal steel defect dataset consists of different types of defects (scratches, broken parts, welding sediments, etc.) in the production process of flat sheet steel. Severstal is a pioneer in sustainable steel mining and manufacturing. They believe that the future of metallurgy involves development in the industry’s economic, ecological, and social elements—and they take corporate responsibility seriously. Severstal is now looking at machine learning to enhance automation, improve efficiency, and keep high quality in their production. Flat sheet steel manufacture is very sensitive. Several pieces of equipment touch flat steel before it’s ready to transport, from heating and rolling to drying and cutting. Severstal now powers a flaw detection algorithm with pictures from high-frequency cameras.
- Northeastern University (NEU) open surface defect database consists of six kinds of typical surface defects of the hot-rolled steel strip that are collected, i.e., rolled-in scale, patches, crazing, pitted surface, inclusion and scratches.
- The customized dataset of polymer surface defects initially made by the saiwa team; and 6- the DAGM (German Association for Pattern Recognition) texture dataset, a synthetic dataset for industrial optical inspection and contains ten classes of artificially generated textures with anomalies. The data is artificially generated but similar to real-world problems.
What network models does saiwa train for surface defect detection datasets?
Several network models have been trained for these datasets, including:
UNet
U-Net is a convolutional neural network created at the University of Freiburg’s Computer Science Department for biomedical image segmentation. The network’s architecture was updated and expanded to work with fewer training images and provide more exact segmentations.
Seg Dec Net
Segmentation-Based Deep-Learning Approach for Surface Defect Detection.
ResNet34
Resnet34 is a cutting-edge image classification model defined in “Deep Residual Learning for Image Recognition” as a 34-layer convolutional neural network. Restnet34 has been pre-trained on the ImageNet dataset, which comprises over 100,000 photos from 200 classifications.
At saiwa, one may try the trained models freely by a simple UI on his/her images and in case of interest, saiwa team may retrain the networks on the user-specific dataset or other types of surfaces and defects.
FAQ
Most frequent questions and answers
What are some standard methods for surface defect detection?
Some standard methods for surface defect detection include visual inspection, machine vision systems, ultrasonic testing, X-ray inspection, eddy current testing, and magnetic particle inspection.
Why is surface defect detection important?
Surface defect detection is essential because defects on the surface of a material can compromise its integrity, reduce its strength, and impact its performance. Surface defects can also cause safety hazards in applications such as the aerospace or automotive industries.
What types of materials can be inspected for surface defects?
Surface defect detection can be used on various materials, including metals, plastics, ceramics, composites, and glass.
How accurate are surface defect detection methods?
The accuracy of surface defect detection methods depends on various factors, including the type of method used, the quality of the equipment and software, and the operator’s expertise. Generally, automated systems are more reliable and accurate than human inspection.
Can surface defect detection be used in real-time production environments?
Many surface defect detection methods can be used in real-time production environments. Automated systems can inspect surfaces quickly and accurately, providing real-time feedback and enabling corrective action to be taken immediately.
What are some benefits of using surface defect detection?
Surface defect detection can help to reduce scrap, increase productivity, improve product quality, and enhance safety. It can also help identify potential problems early, enabling corrective action before defects cause more severe issues.
