RESEARCH ARTICLE

Defect detection on button surfaces with the weighted least-squares model

  • Yu HAN ,
  • Yubin WU ,
  • Danhua CAO ,
  • Peng YUN
Expand
  • School of Optical and Electronic Information, Huazhong University of Science and Technology, Wuhan 430074, China

Received date: 22 Dec 2016

Accepted date: 09 Mar 2017

Published date: 05 Jul 2017

Copyright

2017 Higher Education Press and Springer-Verlag Berlin Heidelberg

Abstract

Defect detection is important in quality assurance on production lines. This paper presents a fast machine-vision-based surface defect detection method using the weighted least-squares model. We assume that an inspection image can be regarded as a combination of a defect-free template image and a residual image. The defect-free template image is generated from training samples adaptively, and the residual image is the result of the subtraction between each inspection image and corresponding defect-free template image. In the weighted least-squares model, the residual error near the edge is suppressed to reduce the false alarms caused by spatial misalignment. Experiment results on different types of buttons show that the proposed method is robust to illumination vibration and rotation deviation and produces results that are better than those of two other methods.

Cite this article

Yu HAN , Yubin WU , Danhua CAO , Peng YUN . Defect detection on button surfaces with the weighted least-squares model[J]. Frontiers of Optoelectronics, 2017 , 10(2) : 151 -159 . DOI: 10.1007/s12200-017-0687-7

Introduction

Machine vision is a promising solution to automatic visual inspection and performs competitively against human vision. Machine vision generally consists of image acquisition, image processing, and decision making. Image processing is important in the detection process. It extracts features that are useful for decision making. Image-based defect detection methods are used in steel bars [1], printed circuit boards [2], fabric [3], solar wafers [4,5], thin-film transistor-liquid crystal displays [6], bottle caps [7], and many other applications.
Buttons, as important clothing accessories, contain various defects, such as cracks, stains, printing deviation, fading, and uneven surfaces. Button defects are irregular and difficult to identify. The fast production of buttons and the various patterns on their surfaces make the automatic detection of defects even more difficult. Therefore, a fast and generally applicable defect detection method is urgently needed.
Several researchers have converted the defect detection problem to a template matching problem by calculating the similarity between an inspection image and a corresponding defect-free template image [8,9]; this approach is called the referential method. Normalized cross correlation is widely used to measure the similarity. In the frequency domain, many techniques, such as Fourier transform [10], wavelet transform [11,12], and Gabor filter [13,14], are also widely used in defect detection.
Although calculating the similarity between an inspection image and a corresponding defect-free template image is a practical and widely used method, the process requires high alignment accuracy. To reduce the false alarms caused by misalignment, Wang et al. [15] applied the gray-level threshold method to extract significant points in an inspection window and calculated the correlation coefficient of only the selected significant pixel points. Bai et al. [8] extracted salient regions through phase-only Fourier transform and detected defects in these regions. Meanwhile, Zontak and Cohen [16] proposed a defect detection method based on anisotropic kernel reconstruction. In their method, the source image is estimated with the weighted sum of neighboring features from the reference image. Their method does not require image alignment and is robust to pattern variations. However, its computation cost is relatively high. Tsai et al. [9] proposed a dissimilarity measure based on the optical-flow technique. The measure is robust to misalignment and random product variation. It is applicable to gray-level images and suitable for detecting defects that are sparse signals with a uniform background.
Static templates and gray-level images are widely used in all these methods. The methods are sensitive to illumination vibration because the templates are static. A referential method using the weighted least-squares model was developed in the current study. The method does not require precise alignment and is robust to illumination variations. Instead of using gray-level images, all three channels of red–green–blue (RGB) images are used to improve the accuracy of the method.
The remainder of this paper is organized as follows. The proposed model and its implementation are presented in Section 2. The experiment results and a comparison of the proposed method with two other methods are provided in Section 3. The tuning of key parameter values is also included in this section. Section 4 presents the conclusions.

Weighted least-squares model and implementation details

The referential method relies heavily on alignment accuracy. Even if two compared images are perfectly aligned, local deviations may still exist in the aligned image due to random product variations in manufacturing. As a result, the residual image exhibits high intensity near the edge regions. To address this problem, the proposed method uses the weighted least-squares model to reconstruct a template image for each test sample adaptively (some details are shown in Section 2.2) and obtain a corresponding residual image. To segment defects effectively, the Gaussian mixture model (GMM) is used to model the background distribution according to the defect-free residual images. The entire procedure is summarized as follows (Fig. 1).
Fig.1 Block diagram of the proposed method for defect detection

Full size|PPT slide

1) Preprocess original images to extract the region of interest (ROI) and align the images to achieve a similar orientation;
2) Combine defect-free images to obtain matrix B through the weighted least-squares model;
3) Input test samples and decompose them into defect-free template and residual images adaptively;
4) Learn the GMM parameters with the defect-free residual images;
5) Input a test sample’s residual image and detect the defect region with the GMM parameters.

Preprocessing

In this step, the objects (buttons) were extracted from original RGB images {I} (IR640×480×3). The buttons were then rotated at certain degrees to ensure that all buttons had the same orientation. Most types of buttons have a common characteristic: circular holes distributed around the center symmetrically. Therefore, we calculated the rotation degrees after locating the holes and button center by using auto-threshold algorithms. Subsequently, we rotated the buttons around their centers to achieve the same orientation.

Weighted least-squares model

Given the data set{yi,xi1,xi2,...,xin},i=1,2,...,m, we assumed that response variable yi can be represented by a linear regression of n-vector variablexiwith residual termεi. The common least-squares model is in the form
yi=β1xi1+β2xi2+...+βnxin+εi,i=1,2,...,m.
This equation can be presented in a vectorized form as
y=X β+ε,
where y=[y1ym],ε=[ε1εm],β=[β1βn], and X=[x11x1nxm1xmn]. The error function is defined as
J(β)=iεi2.
To minimize J(β), its derivatives must be determined with respect to β. Thereafter, these derivatives need to be set to zero.
βJ(β)=XTXβXTy=0,
β=argminβJ(β)=(XTX)1XTy.
To minimize the ith error term εi in ε, a large weight ωi should be provided to εi. The weighted least-squares model is described as follows:
J*(β)=iωiεi2=(W12ε)T(W12ε),
where W=[ω100ωm]. The weighted estimated parameters are
β=argminβJ*(β)=(XTWX)1XTWy=By,
whereB=(XTWX)1XTW. For each RGB image IRM×N×3, M and N represent the height and width of an image, respectively, and 3 represents the three channels of an RGB image. I(i,j,c) is the intensity of the pixel at (i, j) of an image in the c channel. Furthermore, I can be vectorized as a 1D column vectorxRm×1,m=3MN.
If defect-free samples are selected and vectorized into column vectors, the training data matrix can be described asX=[x1x2xn], where n is the number of training samples and xkrepresents the kth sample in the training set. In the training step, the acquisition of matrix B can be implemented offline. In the testing step, inspection image D is vectorized as column vector d. As a result, the best parameter,β=B d, and the appropriate item can be described as a=X β. Adaptive template imageAcan be reconstructed from vector a, and residual image isE=|D-A|, as shown in Fig. 2.
Fig.2 Image decomposition process. (a) Inspection image; (b) adaptive template image; and (c) residual image

Full size|PPT slide

In the experiment, the residual images sometimes showed high intensity near the edge, and this high intensity may cause a false alarm when the common least-squares method is used (Fig. 3). In the weighted least-squares model, a large weight is provided to pixels near the edge regions to solve the problem. Figure 3(c) shows that the residual error was suppressed near the edge regions.
Fig.3 Detection results of different methods. (a) is the input image. (b) and (d) are the results of the common least-squares model. (b) is the enhanced residual image, and (d) is the defect segmentation result. (c) and (e) are the results of the weighted least-squares model. (c) is the enhanced residual image, and (e) is the defect segmentation result

Full size|PPT slide

In all the experiments, the weight matrix was in the form
W=diag(I),
where I represents the mean gradient intensity of the training samples. W is a diagonal matrix with the mean gradient intensity lying on the main diagonal.

Defect segmentation

GMM has been widely used in automated visual surveillance. GMM segments moving regions from image sequences. Given that our defect detection problem is rather similar to moving object detection, GMM was used to segment defects. The basic idea was to model each background pixel by a mixture of K Gaussian distributions, where K is a small number from 3 to 5. The probability that a certain pixel with a value of x exists at time t can be written as
p(x)=i=1Kωtiη(x;μti,Σti),
where K is the number of Gaussian components and ωtiis the weight of theith Gaussian component. It satisfiesi=1Kωti=1.η(x;μti,Σti)is the normal distribution of the component represented by
η(x;μti,Σti)=1(2π)d/2|Σti|1/2exp[12(xμti)T(Σti)1(xμti)],
where μti and Σti are the mean value and covariance of the ith component, respectively. In particular, Σti is defined as Σti=(σti)2I. To model background distributions, parameters are updated by adding new samples and discarding old ones. The parameters of the first Gaussian component are updated when they match the value of the new sample. The implementation details can be found in Refs. [1719]. In the test stage, each new pixel value is checked against existing model components according to the order of the fitness value. The match criterion is
dist(x, μti)Cσti,
where C is the control constant and dist(x, y) represents the Euclidean distance between vectors x and y. If no match is found, this pixel will be identified as a foreground pixel (defect).

Experiment results and discussion

The performance of the proposed method was evaluated in terms of true positive recognition (TPR) rate, true negative recognition (TNR) rate, and recognition (R) rate, which are defined as
TPR=NTPNTP+NFN,
TNR=NTNNTN+NFP,
R=NTP+NTNNTP+NTN+NFP+NFN,
whereNTP is the number of correctly detected defect-free samples,NFN is the number of falsely detected defect-free samples,NTN is the number of correctly detected defective samples, andNFP is the number of falsely detected defective samples. Positive and negative means defect-free and defective, respectively.
We evaluated the performance of the proposed method on three typical types of buttons by comparing it with two other methods, namely, saliency-based method in Ref. [8] and independent component analysis (ICA)-based method in Ref. [5]. For simplicity, we denoted these three types of buttions as sample1, sample2, and sample3. The details of the test samples are presented in Table 1. The proposed method was implemented in Matlab 2014a on an Intel 2.1 GHz PC with 4 GB of memory. The speed of this method was about 15 fps, which meets the requirement of button quality assurance.
Tab.1 Details of the test samples used in the experiments
samplenumber of
defect-free buttons
number of defective buttonsROI size/pixel
sample151104233×233×3
sample26039199×199×3
sample3151189167×167×3
The detailed detection rates are shown in Table 2. The saliency-based method showed a relatively high true positive rate but a low true negative rate because several defective samples were missed. The ICA-based method showed a good true negative rate but a low true positive rate because several normal regions were incorrectly identified as defects. In this experiment, the performance of our method was better than that of the two other methods in terms of the three detection rates.
Tab.2 Experimental performance of different methods
samplesaliency-based methodICA-based methodproposed method
TPRTNRRTPRTNRRTPRTNRR
sample10.920.720.780.780.910.870.940.940.94
sample20.980.510.790.770.870.810.9810.99
sample30.950.710.820.750.860.810.970.990.98

Comparison of detection results

The defect detection results of samples 1–3 are shown in Fig. 4. In the experiment, the parameters were according to the recommendation of ICA-based [5] and saliency-based [8] methods. Figure 4(a) shows the detection results for sample1, in which printed words exist on the surfaces of buttons. The main defect types were printing deviation, stain, crack, and other defects. The defect points are shown in white in the binary images. The first column presents one defect-free and three defective samples. The second and third columns show the detection results of the saliency-based and ICA-based methods, respectively. The last column presents the detection results of the proposed method.
Figure 4(b) shows the detection results for sample2, which contains the fade-type defect. The first column presents one defect-free and two defective samples. The second and third columns show the detection results of the saliency-based and ICA-based methods, respectively. The results of the proposed method are shown in the last column. The proposed method performed better than the two other methods because the two methods ignored color information. This result suggests that color information contributes significantly to the detection of fade-type defects.
Figure 4(c) presents detection results for sample3, whose main defects consist of uneven surfaces and cracking. The first column shows one defect-free and three defective samples. The second column presents the detection results of saliency-based method. This method detected edge-type and dot-type defects effectively, given that these defects showed large salient values in the saliency detection step. Owing to the use of the search window strategy (SW), the method is robust to spatial misalignments. The third column shows detection results of the ICA-based method, which is sensitive to misalignment and has a high false alarm rate. The result of proposed method is shown in the last column. Our method is robust to spatial misalignments and has a high recognition rate.
Fig.4 Comparison of defect detection results on three samples by using different methods. (a)–(c) correspond to samples1–3. The first column contains the original images; the second and third columns show the results of saliency-based and ICA-based methods, respectively; and the last column presents the detection results of the proposed method

Full size|PPT slide

Tuning of training dataset size

An experiment was conducted to reveal the relationship between recognition rate and training dataset size. Figure 5 shows that the recognition rate increased as the size of the training dataset increased. However, the recognition rate decreased slightly when the number was over 40 with the increasing computation cost. Therefore, the appropriate size of the dataset was 40. Our training dataset was composed of eight buttons. Aside from the original images, another four deviation images were generated by each button by rotating at a small angle around the button center. From the view of linear regression, insufficient data may cause underfitting, and excessive data may cause overfitting. Both scenarios may cause poor performance in the test dataset.
Fig.5 Recognition rate versus training dataset sizes in samples 1–3

Full size|PPT slide

Effect of weight matrix on recognition rate

We adopted a weight matrix to reduce the false alarms caused by large residual errors near edges. An experiment was conducted to verify the effectiveness of the weight matrix. Three weight matrices were used.Wdiag(I) indicates that the mean gradient intensity was used as weight matrix W in the proposed method. Wdiag(1) indicates that W is an identity matrix, and all pixels have the same weight. Wdiag(exp(I)) indicates that pixels on edges have a smaller weight than non-edge pixels.
Table 3 shows that when Wdiag(I) was used, the recognition rate improved. This result means that using the mean gradient intensity as the weight matrix can improve the recognition result.
Tab.3 Relationship between recognition rate and weight matrix W
recognition rate
sample1sample2sample3
Wdiag(I)0.940.990.98
Wdiag(1)0.880.930.95
Wdiag(exp(I))0.750.780.89

Effect of parameter setting

The key parameters included in the segmentation with GMM consisted of control constant C and number of Gaussian components K. Sensitivity analyses of these parameters were conducted as follows.

Receiver operating characteristic (ROC) analysis of control constant C

Control constant C determines if a pixel belongs to a Gaussian component. A large control constant provides a loose control limit and may ignore several defects, whereas a small control constant provides a tight control limit and may generate false alarms. To verify the detection performance quantitatively, the ROC curve of control constant C was constructed by scanning the control constant from a small number (1.5) to a large number (10), as shown in Fig. 6. In this experiment, the parameters of the number of Gaussian components K were all set to 5.
Fig.6 Receiver operating characteristic curves of the proposed method

Full size|PPT slide

The ROC curve of the proposed method rises swiftly upward when C ranges from 1.5 to 4.5. In Fig. 6, when C is equal to 4, good performance is obtained with both low false positive and high true positive rates.

Analysis of Gaussian components K

Gaussian components K determines the ability to describe the distribution of the background. A large K can describe a highly complex distribution. If K is set to 1, the model degenerates into a single Gaussian model. According to Refs. [17,18], Kshould be set from 3 to 5. To further verify this point, we conducted experiments on samples 1–3 under different K values (from 1 to 5). Control constant C was set to 4. The experiment results are presented in Table 4. The table shows that setting K from 4 to 5 is appropriate, so K was set to 5 in the rest of our experiments.
Tab.4 Recognition rate versus number of Gaussian components K
Gaussian components Krecognition rate
sample1sample2sample3
10.610.620.66
20.820.880.81
30.900.980.98
40.940.990.98
50.940.990.98

Effect of rotation deviation

In the proposed method, we rotated the buttons to the same orientation as that in the preprocessing step. Therefore, all buttons were coarsely aligned. However, small deviations still existed among the aligned buttons (most of the deviations were within 5° according to the experiment result). To evaluate the robustness of proposed method to this subtle rotation deviation, we generated another seven test sets by rotating the aligned images at angles ranging from 1° to 7° and evaluated the performances in the eight test sets. Figure 7 shows the performance in sample3 using the three methods. The saliency-based method and the proposed method presented a more stable performance than the ICA-based method, and the proposed method achieved a higher detection rate.
Fig.7 Comparison of the recognition rate of the proposed method with those of the saliency-based and ICA-based methods in terms of rotation deviation angles

Full size|PPT slide

Effect of illumination variation

To visualize defect detection performance, Fig. 8 shows the detection results for the same defective button under different illumination conditions. The first column presents the original inspection images under different illumination intensities ranging from 2500 to 4000 lx. The second column shows the reconstructed adaptive template images corresponding to the first column. The last column presents the binary results, which show that the four defective images can be detected steadily. In the experiment, illumination variation was equal to the effect of adding a light intensity offset to every pixel of the inspection image. The reason for the robustness to illumination variation is that to minimize the error function (Eq. (6)), the proposed method adjusts combination coefficients adaptively. As a result, it generates the most appropriate template image adaptively for each inspection image.
Fig.8 Effect of illumination variations. (a) Original inspection images under different illumination intensities (top to bottom correspond to 2500, 3000, 3500, and 4000 lx); (b) adaptive template images corresponding to the first column; and (c) binary results corresponding to the first column

Full size|PPT slide

To further evaluate the robustness of our method to illumination variation, we performed experiments with sample3 under different illuminations (ranging from 2500 to 4500 lx). In this experiment, the samples under 3500 lx were used as training datasets. Table 5 shows that the proposed method presents a good recognition rate in both under-exposed (2500 lx) and over-exposed (4500 lx) situations.
Tab.5 Recognition rates under different illuminations
illumination/lux25003000350040004500
recognition rates0.930.960.980.970.93

Conclusions

Referential methods have been widely used in surface quality inspection, but these methods rely heavily on alignment accuracy. To address this problem, we proposed a defect detection method based on the weighted least-squares model to suppress the residual error near the image edge. The defects were segmented with GMM.
The proposed method can be applied successfully to different types of buttons. It does not require precise alignment and is robust to both illumination variation and rotation deviation. The method can be potentially applied to various patterned surfaces without high alignment accuracy. To further improve the detection rate, defect detection considering local texture features is worthy of further investigation.
1
Li W B, Lu C H, Zhang J C. A lower envelope Weber contrast detection algorithm for steel bar surface pit defects. Optics & Laser Technology, 2013, 45(1): 654–659

DOI

2
Crispin A J, Rankov V. Automated inspection of PCB components using a genetic algorithm template-matching approach. International Journal of Advanced Manufacturing Technology, 2007, 35(3): 293–300

DOI

3
Arivazhagan S, Ganesan L, Bama S. Fault segmentation in fabric images using Gabor wavelet transform. Machine Vision and Applications, 2006, 16(6): 356–363

DOI

4
Li W C, Tsai D M. Wavelet-based defect detection in solar wafer images with inhomogeneous texture. Pattern Recognition, 2012, 45(2): 742–756

5
Tsai D M, Wu S C, Chiu W Y. Defect detection in solar modules using ICA basis images. IEEE Transactions on Industrial Informatics, 2013, 9(1): 122–131

DOI

6
Cen Y G, Zhao R Z, Cen L H, Cui L H, Miao Z J, Wei Z. Defect inspection for TFT-LCD images based on the low-rank matrix reconstruction. Neurocomputing, 2015, 149: 1206–1215

DOI

7
Zhou W, Fei M, Zhou H, Li K. A sparse representation based fast detection method for surface defect detection of bottle caps. Neurocomputing, 2014, 123: 406–414

DOI

8
Bai X, Fang Y, Lin W, Wang L, Ju B F. Saliency-based defect detection in industrial images by using phase spectrum. IEEE Transactions on Industrial Informatics, 2014, 10(4): 2135–2145

DOI

9
Tsai D M, Chiang I Y, Tsai Y H. A shift-tolerant dissimilarity measure for surface defect detection. IEEE Transactions on Industrial Informatics, 2012, 8(1): 128–137

DOI

10
Chan C H, Pang G K. Fabric defect detection by Fourier analysis. IEEE Transactions on Industry Applications, 2000, 36(5): 1267–1276

DOI

11
Ngan H Y T, Pang G K H, Yung S P, Ng M K. Wavelet based methods on patterned fabric defect detection. Pattern Recognition, 2005, 38(4): 559–576

DOI

12
Yang X, Pang G, Yung N. Robust fabric defect detection and classification using multiple adaptive wavelets. IEE Proceedings−Vision Image and Signal Processing, 2005, 152(6): 715

DOI

13
Ralló M, Millán M S, Escofet J. Unsupervised novelty detection using Gabor filters for defect segmentation in textures. Journal of the Optical Society of America A, Optics, Image Science, and Vision, 2009, 26(9): 1967–1976

DOI PMID

14
Kumar A, Pang G K. Defect detection in textured materials using Gabor filters. IEEE Transactions on Industry Applications, 2002, 38(2): 425–440

DOI

15
Wang C C, Jiang B C, Lin J Y, Chu C C. Machine vision-based defect detection in IC images using the partial information correlation coefficient. IEEE Transactions on Semiconductor Manufacturing, 2013, 26(3): 378–384

DOI

16
Zontak M, Cohen I. Defect detection in patterned wafers using anisotropic kernels. Machine Vision and Applications, 2010, 21(2): 129–141

DOI

17
Stauffer C, Grimson W E L. Adaptive background mixture models for real-time tracking.In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1999, 2: 246–252

18
Kaewtrakulpong P, Bowden R. An improved adaptive background mixture model for real-time tracking with shadow detection. Springer US, 2002: 135–144

19
Zivkovic Z. Improved adaptive Gaussian mixture model for background subtraction. In: Proceedings of International Conference on Pattern Recognition, 2004

Outlines

/