A method for inspecting the cleanliness of spinneret holes

At present, computer vision system is widely used for the cleanliness inspection for spinneret holes, but it has a high misjudgment rate for many holes with small dirt. In this paper, a method is proposed to improve the accuracy of the cleanliness inspection. The method has four sequential phases. First, the closed contour curve of a standard hole and its curvature are extracted. Based on double-threshold segmentation of the contour curve, line and arc segments are segmented to generate a closed piecewise curve model. Second, the model is fitted to the closed contour curve L1 of the hole to be inspected based on a nonlinear least squares principle, and the distance curve that represents the shortest distance between the closed piecewise curve L2 constructed from the aligned model and L1 is calculated based on the nearest neighbor search algorithm. Third, the dirt detection curve L3 is generated from the distance curve weighted by a blended weighting curve. Final, based on a global threshold combining with unevenness elimination of L3, the cleanliness index is calculated based on the segmentation and location of dirt, and is used to judge whether the hole is qualified or not. The experimental results of four databases demonstrate that, the proposed method provides better performance compared with the traditional method.


Introduction
Chemical fibers are mostly made of crystalline polymers. The fused polymer is ejected from the spinneret hole under the conditions of high temperature and high pressure, and the chemical fiber is formed after cooling and crystallization. After a period of use, the spinneret must be cleaned ultrasonically and dried, and it is necessary to inspect whether there is any remaining dirt before the spinneret is reused. Otherwise, the dirt will lead to abnormities of filament performance, which means that the cleanliness inspection of spinneret holes is an important procedure during the quality monitoring process of synthetic fiber production.

and
Tan 10 proposed the self-correcting position control system. Tan 10 studied many image processing algorithms for hole focusing and inspection. Komatsu et al. 11 proposed a critical dimension measurement of a hoe pattern. Nagasu et al. 12 used low-frequency sound for inspecting microholes with diameters of less than 100 µm. Cai et al. 13 proposed a roundness measurement with the least square circle fitting method for the processing of geometric feature recognition of holes. Zhang et al. 14 extracted the geometrical characteristics of round holes to judge whether them are blocked or not. Yang et al. 15 designed a high-precision calibration model for spinneret inspection system. Yang et al. 16 analyzed the influential factors of the measurement accuracy. Yang et al. 17 acquired the clear image of large spinneret by increasing number of auto-focus. Gu et al. 18 proposed an inspection route planning method based on improved ant colony algorithm. Wang et al. 19 achieved the judgment of block by area threshold method and calculated the shaped degree for non-circular spinneret hole. Chen et al. 20 developed a low cost digital spinneret inspection system with opencv toolkits. Chen et al. 21 developed an inclination measurement by using the imaging based autofocus method. Chen et al. 22,23 established a path generation method for circle-type, rectangle-type and ring-type spinnerets. In the last 10 years, American ASPEX Co. Ltd, Japanese TORAY Co. Ltd, Chinese Academy of Sciences et al. have already developed spinneret automatic inspection systems for industrial applications.
This paper focus on the problem of the cleanliness inspection for the holes, but so far, almost all the proposed method are based on the characteristic indexes such as area and perimeter to determine whether the hole is clean or not. We know dirt inside the hole will cause the index value to be abnormal. However, due to the fluctuation of area and perimeter index values caused by machining errors beyond the sensitivity range of the detection of tiny dirt, this method is only effective for holes with large dirt whose contour is significantly different from that of a normal hole. Therefore, the cleanliness inspection for the holes with tiny dirt, especially for special-shaped holes, has always been a difficult question.
To solve this problem, we proposed the approach based on the template matching error analysis method. Firstly, we extracted the standard contour template of the hole, and defined the distance curve as the error distribution of the alignment between the contour of the standard template and that of the hole to be inspected. Secondly, the dirt detection curve from the distance curve weighted with a blended weighting curve was defined. Finally, the cleanliness index was qualified based on the segmentation and location of dirt on the dirt detection curve. However, if the contour curve is used as a rigid-template aligned to that of the hole to be inspected, especially for complex-shaped holes, the local fitting error between the template and the deformed contour curve caused by machining errors can easily lead to a false-positive dirt identification. Therefore, according to the mechanism of the contour curve of hole mainly based on the combination of line and arc segments, we proposed a closed piecewise curve model with parameterized control points for non-rigid registration to the contour curve of the hole to minimize the errors, which can not only ensure the overall rigidity of the segments in the model, but also avoid their excessive deformation. The proposed method can improve the accuracy of the cleanliness inspection and be suitable for the most shapes of holes with various degrees of dirt distribution.
As Figure 1 shows, our approach includes six steps: (a) Extracted the closed contour curve L 1 of a standardized hole and L 2 of the hole to be inspected; (b) Generated a closed piecewise curve model with parameterized control points; (c) Fitted the model to L 2 based on the non-liner least squares principle; (d) Calculated a distance curve based on the aligned model and L 2 ; (e) Generated the dirt detection curve from the distance curve weighted with a blended weighting curve; (f) Segmented the dirt from the dirt detection curve and generated the cleanliness index and compared it with predefined criteria to judge whether the hole is qualified or not. Our main contributions are summarized as: (a) a closed piecewise curve model suitable for all the shapes of holes, (b) the generations of the distance curve and dirt detection curve, and (c) the definitions of the model of dirt segmentation and cleanliness index.

Image acquisition
Our Image acquisition system includes motor control unit, image acquisition unit, light unit, etc. The motor control unit was adopted with three-axis ADLINK servo motor control system, the acquisition unit was adopted with Basler 500M industrial camera (2592 × 1944 × 8 bit and 15 frames/s) and a 14× zoom lens, and the light unit was adopted with upper LED lighting for focusing and back LED lighting for acquisition. The software was compiled with C++ on a Windows system.
In this paper, four types of small image database were collected. Each database belongs to a same spinneret plate. Among them, DB1 has 36 images with Y-shaped holes (DPI = 1.4594 µm / pixel, 0.5 mm (petal length) × 0.18 mm(petal width), including 26 clean holes), DB2 has 268 images with flat-shaped holes (DPI = 2.3681 µm / pixel, 0.4 mm(petal length) × 0.06 mm (petal width), including 255 clean holes), DB3 has 22 images with cross-shaped holes (DPI = 1.9692 µm / pixel, 0.45 mm(petal length) × 0.1 mm (petal width), including seven clean holes), DB4 has 51 images with round holes (DPI = 2.3681 µm / pixel, 0.18 mm (diameter), including 37 clean holes). The holes in DB1 to DB4 have been identified with clean or dirt by experienced inspectors, and all images in each database have been cropped to the appropriate size. Figure 2      coordinate of the i -th point in curve, m 1 is the number of the curve point. We firstly calculated the curvature of L 1 . According to the characteristic distribution of the curvature, L 1 was divided into line and arc segments which were connected end-to-end based on double-threshold segmentation of the perimeter curve. The line or arc parameters of its corresponding segment were estimated based on the least squares fitting algorithm, 26 and the control point was located depending on the solver of the intersection of each adjacent segment pair. Finally, the model was generated, and the closed piecewise curve was constructed, which was used as an approximate representation of L 1 . , a circle was fitted using the least squares method 26 and its radius was used as the curvature operator c i . The curvature of L 1 was described by the equation Closed piecewise curve model with parameterized control points. Let c thr1 and c thr2 be the curvature threshold for the separation of line and arc segments, respectively, and usually c thr 2 ≪ c thr1 . Let g thr be the length threshold for segment fusion. If c i t hr > c 1 , l i 1 belongs to a line segment, shown as the black point in Figure 4(b). If c i t hr < _ c 1 and c i t hr > c 2 , l i 1 belongs to a large arc segment, shown as the green line in Figure 4(b), otherwise it belongs to a small arc segment, shown as the red line in Figure 4(b). The start and end positions of each segment were used as its control points, and two adjacent segments shared the same one. Compared with the line segment, the arc segment added a middle control point located at the center of the arc which indicated its direction. Let G g i f be the set of segment lengths, where g i denotes the length of each segment and f g denotes the total number of segments. By searching the minimum segment length g kmin in G , if g g kmin thr < , the kmin -th segment was discarded, and the control point of its two adjacent segments was replaced by the midpoint of the discarded segment. Repeat iteratively for calculation of G and fusion of the segments with minimum length and leave when no conditions are met. If there was only one arc segment after curvature segmentation and segment fusion, it means that the spinneret hole was round. In this case, it was divided into two connected arc segments. The parameterized model was established as follows: as the image for model generation, column 2 to 4 are holes with clean, tiny and large dirt, respectively, which are used to analyze the effects of cleanliness inspection lately.

Closed contour curve reconstruction
The edge of a clean hole is smooth and it will become irregular if there is dirt in the hole. The brightness of the dirty inside of the hole changes with the size of the dirt and its distance from the edge of the orifice. It is difficult to find a suitable fixed threshold for various dirt segmentation, because even small changes of threshold will lead to a shift of the segmented boundary. However, due to the properties of non-maximum and zero-crossings, the edge detector can locate the edge more precisely than the global thresholding method. We used the canny method 24 to extract the hole boundary, in which two thresholds are used to detect strong and weak edges, and the true weak edges are detected more likely and robustly, which is especially suitable for detecting the tiny dirt. Here, we used σ σ thr t hr 1 2 as the low and high gradient magnitude threshold pair to detect hole boundary. In general, the edge points in boundary may be broken, or there may be bifurcations or isolated noise, so we processed it with morphological operations and linked the edge points to form a closed-contour curve by edge tracing algorithm. 25 Figure 3 shows the results of reconstructed closed-contour curves, in which (a)-(d) correspond to the closed contour curves of Figure 1(d) to (g) respectively.

Model generation
, , be the closed contour curve reconstructed from Figure 3 Where F is the set of control points, P is the set of line segments, Q is the set of arc segments, C is the set of radii of arc segments, CPCM is the reconstructed curve of the model, s is the number of control points as starting points of segments, e is the number of control points as ending points of segments, c is the number of control points as middle points for arc segments, f n is total number of control points, f l is total number of line segments, and f c is the total number of arc segments. LINE  (  ,  ,  ) , , , is the arc segment function formed by three points (  ,  ,  ) , , , s c e , which can be expressed as multiple line segments connected end to end.   We used the ceres-solver optimal method (http://ceressolver.org) to solve equation (5), and the optimized control point set was obtained as F τ . Figure 5 shows the result of non-rigid registration of the model in Figures 3(a3) to

The distance curve
We redefined L 3 Note that L 4 may have many local fluctuations due to the irregular dirt edge. We therefore dilated it with a 1 9 × line structuring element to filter the sawtooth noise. Figure 6 shows the results of distance curve calculation from

The dirt detection curve
A smaller curvature often leads to a larger mechanical processing error, which results in a larger error between L 2 and L 3 , as shown in Figure 5(c). In order to suppress the error as much as possible, the distance curve L 4 was weighted based on the distribution of the curvature. The curvature weight of the line segment was set to 1, described by equation (8), where D l is the point set of the l -th line segment in L 3 . The gradient interpolation method in which a smaller curvature has a smaller weight was adopted for the weighting function of arc segments, described by equation (9), where D c is the point set of the c -th arc segment in L 3 . Equation (10) describes the mixed curvature weighting function W t1 , in which the coefficient jump of the curvature transition region among the weighted curve W l and W c was smoothed with a Gaussian kernel g i ( ) , where * represents the convolution operation, and µ 1 = 0, σ 1 = 3.0. Because of the discontinuity in the curvature distribution around the location of the control point between two adjacent segments, as shown in Figure 5(c), nonlinear fitting errors may appear between the location of the optimized control point and the curvature transition area of the contour curve. If the hole was dirty or its deformation was large, the error will cause the dirt curve to appear as a bloated artifact around the location of the control point, and it will be segmented as dirt. Therefore, we used the Gaussian weighting function W t 2 to suppress it as given by equations (11) and (12), where f i is the point index of the f -th control point, and σ 2 2 5 = . . Finally, the blended weighting function W t was given by equation (13), and the dirt detection curve L 5 denoting the distance curve weighted by the blended weighting function was defined by equation (14).  Figure 7 shows the calculation of the dirt detection curves in Figures 6(e). In Figures 7(b), the distance from each point on the blue curve along the normal direction to the red curve is its blended weighting coefficient. From the comparison between Figures 6(e) and 7(c) we can see that the false singularity blob caused by the nonlinear fitting errors is mainly suppressed by the blended weighting function.

Dirt segmentation and cleanliness inspection
Generally, the distance curve and the dirt detection curve of a standardized hole are approximately smooth and flat, and the dirt appears as a singularity blob in the detection curve, which can be segmented and located by a global distance threshold. However, due to the machining error of the spinneret, plate deformation and abrasion of the hole after many times of use, local distortion will occur at the contour curve of the hole, which leads to baseline unevenness in the dirt detection curve, as shown in Figures 8(a). This unevenness makes the global thresholding method unsuitable for dirt segmentation. Therefore, we adopted the top-hat transform 25 to eliminate the irregularity of the baseline in the dirt detection curve, as shown in Figures  8(b). Figure 8 shows the results of dirt segmentation and location in Figures 7(c). The model of dirt segmentation and its parameter definition are shown in Figure 8( , where n is the number of dirts. In the production of synthetics fiber, dirt in the spinneret hole can potentially break the filament. For this reason, the relative weighting of dirt with sharp shape (shown as Figure 1(c)) should be increased based on aspect ratio when calculating the cleanliness index. The cleanliness index of hole was calculated as follows: In Figure 8(e), the cleanliness index of the hole is shown in the center in parentheses, the same as below. Meanwhile, the location of dirt is labeled with the black wireframe, and the dirt parameters are shown as (W, H, S) next to it. Let P thr be the threshold of cleanliness. If P P thr > , the hole is unqualified as containing dirt. In this case, the hole will be blown automatically, and then be inspected again. If the hole was still unqualified, it will be cleaned manually. In this paper, D thr and P thr were optimized by maximizing the True Recognition Rate (TRR). False Accept Rate (FRR), False Rejection Rate (FRR) and TRR were defined as bellows. )? : Where m d and m c is the number of dirty and clean holes respectively, and P D i thr ( ) and P D j thr ( ) is the cleanliness index of i -th and j -th hole with its dirt segmented by D thr . The objective function for the optimization of P thr and D thr was defined as follows: In the factory, P thr usually be adjusted according to the actual quality monitoring requirements for cleanliness inspection.

Results and discussion
We firstly listed part of holes in DB1 to DB4, and aligned every contour curve of clean holes to that of its standard hole. Then we optimized the D thr and P thr for each database, and analyzed the results of the dirt inspection of examples in Figure 3. Finally, according to the traditional method, we fitted the multivariate distributions of the area and perimeter of clean and dirty holes with the Multivariate Gaussian Distribution (MGD) (https://en.wikipedia.org/ wiki/Multivariate_normal_distribution), and compared the accuracy of its identification with the proposed method.

Distributions of aligned contour curve of clean holes
Part of the clean holes in DB1 to DB4 collected in this paper are shown in Figure 9(a)-(d), and the dirty holes are shown in Figure 10(a)-(d). We aligned the contour curve of the clean hole in each database to that of its corresponding standard hole rigidly, and the deformation of each hole due to machining errors can be visually observed. Figure 11 shows the error distributions between the aligned contour curve of clean holes and that of its standard hole (shown as red line in the enlarged area in Figure 11(a)-(d)).   Optimization for D thr and P thr Figure 12 shows the optimization curves of TRR-D thr -P thr in DB1 to DB4. Figure 13 shows the histograms of the cleanliness index P calculated from the optimized D thr , in which red line represents the optimized P thr . Table 1 shows the optimized parameters and their TRR, FAR, and FRR for each database. From Table 1 we can see that under the highest TRR optimization goal, all clean holes in each database are correctly identified. However, there are still some holes with tiny dirt within the critical range (see a1, a6 and b12 in Figure 10) or their dirt locate at the top of flap and is suppressed by blend weighting curve (see b9, b13, and c9 in Figure 10), which cause them identified as clean holes. For round holes, the TRR is 100%. Figure 14 shows the cleanliness inspection results of holes in Figure 3 with optimized parameters, in which we can see that this method can satisfy the requirements for cleanliness inspection of holes with various degrees of dirt. In Figure 14(a) to (d), the holes are clean identified by manually, and there is no dirt detected except for a small dirt  with a very low cleanness index in Figure 14(c), which indicates that there may be errors in subjective evaluation or the rule of judgments for complex-shaped holes may be relaxed. From Figure 14(e)-(g) we can see that, for holes with tiny dirt, the non-rigid registration of the model can accurately relocate the control point to the target position of the contour curve, and the dirt can be recognized correctly both in the line and arc segments. Especially for the round hole in Figure 14(g), the curvature difference between the two arcs in L 3 is small enough to ensure the curve smoothly overcomes the connection between adjacent segments, which guarantees the generation of a dirt detection curve free from the interference caused by the nonlinear fitting errors around the location of the control points, and the dirt detection curve with baseline elimination can effectively illustrate the distribution of the dirt. For holes with large dirt in Figure 14(h)-(k), the optimization of relocating control points will fail during the nongrid registration process due to too much difference between L 1 and L 3 . However, the detection distance between L 2 and L 3 is still big enough to reflect the singularity of the dirt, which will only affect the quantitation of the cleanliness index, nor the judgment whether the hole is qualified or not.

Identification based on the traditional method
We defined the curve length of L 2 as the value of perimeter, denoted as x p , and the filling area of L 2 as the value of area, denoted as x s . The probability density function (pdf) of MGD was given as bellows: Where     Figure 3, respectively, (e)-(g) correspond to the results of (c2), (c3), and (c4) in Figure 3, respectively. Column 1: non-rigid registration; Column 2: dirt detection curve; Column 3: dirt segmentation and labeling.
? : Table 3 shows the values of FAR, FRR and TRR calculated from equation (21) to (23) in DB1 to DB4. We can see that in DB2 only Figure 10(b6) that blocked nearly in half was identified as dirty hole, and half of the dirty holes in DB1 were mistaken for clean holes, and in DB3 and DB4 the low FAR is due to the low overlapped area of two fitted MGD.    proposed method is advanced and much better than that of the traditional method. In Table 3, high FAR and FRR indicate that it is difficult to identify the hole is clean or dirty using only the parameters of area and perimeter. In actual inspection, high FRR will seriously reduce the work efficiency and high FAR will significantly decrease the product quality. Under the guarantee that FRR is zero, several holes with dirt within the critical range can be ignored. Hence, our proposed method can meet the requirements for cleanliness inspection in factory.

Application for complex-shaped spinneret holes
Here two types of complex-shaped spinneret holes were collected to verify the performance of the proposed method. One is 3C-shaped holes with three unconnected C-shaped holes, and anther is caterpillar-shaped holes, which are composed of continuous arcs and lines. The results of their cleanness inspection were shown in Figures  16 and 17. We can see that, the models of CPCM were correctly reconstructed and aligned to the contour curves of dirty holes, and the tiny dirt was located based on the dirt detection curve and their cleanliness index were given. Since most of the spinneret hole shapes are not more complex than those listed in this paper, it can be inferred that the proposed method can meet the requirements of the cleanliness detection for vast majority of irregular-shaped spinneret holes.

Conclusion
In this paper, we proposed a method to improve the accuracy of the cleanliness inspection. This method generated a closed piecewise curve model and aligned it to the contour curve of the hole. The cleanliness index was calculated from the distance curve representing the distribution of dirt by the model of dirt segmentation and parameters definition. The outcomes of this research work are as follows: (a) Four types of small image databases are collected by our acquisition system; (b) Our experiment produces good performance in cleanliness inspection of spinneret holes; (c) For round and special-shaped holes, we achieve the accuracy of 0% for FRR and 12.3% for FAR, which provide a much better performance than the traditional method; (d) Moreover, our method can be extended to the cleanliness inspection for the most shapes of holes and the geometry parameters of hole can be calculated in terms of its closed piecewise curve model in the future.

Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the Shanghai Municipal Science and Technology Commission under Grant No. 18030501400.