A method of lining seam elimination with angle adaptation and rectangular mark for road tunnel concrete lining images

Funding information National Natural Science Foundation of China, Grant/Award Number: 61701060 Abstract Road Tunnels are an important part of the current road transportation infrastructure. As the main form of tunnel lining diseases, cracks are easy to interact with other areas, which seriously affects the safe operation of the tunnel. Due to the similarity of brightness and linearity between surface cracks and lining cracks, the existing crack detection algorithms can not extract cracks accurately and quickly. An algorithm of lining seam crack elimination with rectangular mark is proposed here. First, the line segments in the image are detected by the Line Segment Detector algorithm based on the coarse percolation detection of the crack. Second, the distribution directions are calculated, and cracks from the lining seams are distinguished by the adaptive threshold judgment method. Third, by using the distribution characteristics of pixels, the line segments are extended to form rectangular marks perpendicular to the direction of lining seams. Finally, the marking information is used to remove the lining joints and obtain the real surface cracks of tunnel lining. Experimental results show that the algorithm can quickly and effectively remove any shape distribution of lining seam. The algorithm fills in the blank of concrete tunnel lining surface crack detection technology.


INTRODUCTION
With the rapid development of road tunnel construction, it is essential to build a safe and reliable high-quality tunnel. The permanent supporting structure constructed with reinforced concrete and other materials around the tunnel body is a permanent measure to prevent deformation or collapse of surrounding rocks. The role of tunnel lining is to improve the safety of the tunnel. In addition to meeting the use and enhancing the security, the tunnel lining structure should also have the characteristics of modification, beauty and durability. How to evaluate the tunnel structure dynamically and ensure its security in long-term operation has become a challenge [1]. The road tunnel images are shown in Figure 1. At present, there are many quality problems in road tunnels. Changes in temperature and humidity are likely to cause expansion and contraction deformation of tunnel lining structures. Uneven

FIGURE 2
The process of the proposed method thickness, blasting [4], frost heaving damage [5] and irregular construction. The degree of crack, regularity, direction and disease morphology are the key to determining the health of the tunnel lining structure [6][7][8][9][10][11][12][13][14]. In order to evaluate the safety index, stability and developmental trend of the tunnel lining structure, it is necessary to detect and extract surface cracks. To overcome these problems, the attention modules [15][16][17][18][19] are introduced and have been demonstrated to be effective by some methods. The intelligent and accurate crack detection technology based on digital image processing can obtain more real crack disease details, which provides a reliable reference for subsequent parameter calculation and evaluation analysis of crack disease. The contributions of this paper can be summarized as follows: 1. The algorithm can quickly and effectively extract complex real fractures with arbitrary direction distribution and multiple direction transformation. 2. The algorithm can mark the lining seam in any direction with rectangle marking method, and mark the cracks with reservation, which can remove the lining seam and retain the cracks more accurately. 3. The accuracy of the algorithm is much higher than that of the former algorithm, especially for the lining seam in the inclined direction, and the time of the algorithm is much shorter than that of the former algorithm. The process of performing our method is shown in Figure 2.
The rest of this paper is organized as follows. In Section 2, we briefly introduce the problems existing in the past crack detection methods and the methods used in this paper. Section 3 introduces the line detection algorithm and use the adaptive threshold setting to identify the lining seams, and introduces the multi-direction rectangle marking rules in detail. In Section 4, the selection basis of important parameters in this method is given, and the experimental results are compared with other methods. Section 5 draws the conclusion for this paper.

RELATED WORK
The current tunnel crack detection technology can be summarized into two categories. The first is a fixed detection method based on deformation data, and the second is a mobile detection method based on optical imaging and image processing. The fixed detection method has obvious disadvantages, such as the complicated installation of the sensors which still can not cover all the tunnel surfaces [20]. So, it can not be used as a universal detection method. The mobile detection method uses a detection car equipped with a camera to collect road tunnel images, and uses image processing technology to analyse and identify cracks in the images [21]. This method is characterized by good mobility. The collected data can cover the entire tunnel. The efficiency and accuracy of detection are greatly increased [22]. Therefore, the mobile detection method has become the main development direction of tunnel crack identification technology. The automatic crack identification method based on image processing can quickly acquire crack data [23], which is beneficial to the objective evaluation of structural safety. Landstrom et al. [24] proposed a crack detection method that utilized morphological processing and statistical classification by logistic regression. The accuracy of the method for detecting cracks is over 80%, but it requires a large amount of computational time and misses some tiny cracks. Muduli et al. [25] studied a crack detection technique based on non-destructive testing. The acquired images were processed by two crack detectors, which follow hyperbolic tangent (HBT) filtering and canny edge detection algorithms [26]. However, it can not apply to images with a lot of background noise and unclear cracks. Li et al. [27] used the BP neural network to classify cracks, calculated the geometric characteristics of the cracks, and studied the damage assessment. Valenca et al. [28] proposed a comprehensive method for combining photogrammetry and image processing for automatic crack detection. However, the tests were carried out under laboratory conditions without considering environmental noise. Qu et al. [29] proposed an adaptive non-destructive combination algorithm to eliminate the lining of the tunnel lining and extract cracks. Firstly, the preextraction was improved based on the grid cell analysis [30] to enhance the percolation rate. Secondly, the second accelerated percolation and the improved fracture connection were used to detect cracks accurately. Finally, the tunnel lining seam elimination algorithm based on the line feature unit was used to eliminate the lining seam and extract real cracks. Zhang et al. [31] proposed a method by using morphological image processing techniques and threshold processing techniques to segment local dark regions with potential crack defects from the original grayscale images. Matas et al. [32] proposed the progressive probabilistic hough transform (PPHT), which minimizes the amount of calculation by using the difference in the voting ratio, and selects the edge points randomly to speed up the calculation. Although this method works well for long segments, it is not suitable for small segment detection. Akinlar et al. [33] proposed a linear timeline segment detector that uses an edge drawing (ED) algorithm [34] without parameter adjustment to control the number of false detections. It does not apply to the images with a lot of environmental noise. Cho et al. [35] proposed a novel linear-based representation to simulate the intrinsic properties of line segments in rasterized image space and labelled a new benchmark data set to construct a line segment detection, verification and aggregation framework. But the process of annotating data sets is time-consuming. Gioi et al. [36] proposed a Line Segment Detector (LSD) that can support the estimation of regions in which pixels have similar gradient direction values. If the area passes the standard test of the Helmholtz principle [37], it is treated as a line segment. Qu et al. [38] proposed a marking method for removing lining cracks in vertical and horizontal directions is proposed. The algorithm can deal with the cracks with variable and continuous directions, and there are many cracks in different directions. This algorithm is one of the most effective algorithms for detecting surface cracks of tunnel concrete lining. The goal of LSD is to detect contours in an image, which are special areas in which the grayscale of the image changes from black to white. Therefore, it can be used for a wide range of detection applications [39].
From the previous research experience, it can be concluded that the biggest difficulty of fracture extraction lies in the inherent lining seams in the tunnel lining surface have similar characteristics to the cracks so that the existing crack detection algorithms can not extract a single crack accurately [40].
In this paper, the characteristics of tunnel lining cracks are analysed from the aspects of grey, direction, edge and shape distribution of lining seams. Because the direction angle of the linear segment of the lining seams edge is generally consistent with the direction of the lining seams, there are a large number of linear segments with the same direction angle as the main direction of the lining seams in the image. In this paper, an algorithm of lining seam elimination in any direction based on the rectangle mark is proposed, which can quickly and accurately remove the lining seams and extract the complete binary image of the crack [41]. First, LSD is used to detect the line segments in the lining crack image. Second, the angle and length of all detected line segments in the image are calculated, and the average value of all line segments is obtained. According to the mean value, the deviation threshold is adaptively set to get the set of line segments to be marked. In the third step, the length of the original line is quantitatively analysed through experiments, and 15 pixels are expanded backward along with the starting point and the end point. For each extended line segment, the pixels follow the algorithm rules and form rectangular markers with a certain width in the direction perpendicular to the line segment. Finally, according to the marking information, the cracks are retained, the lining seams are removed, the real surface cracks of the tunnel lining are obtained through seepage denoising [42].

Algorithm process
The steps of the lining seam elimination algorithm are as follows: 1. After preprocessing the image, enter the binary image of tunnel lining surface crack detection, and execute the LSD algorithm to obtain a line set. 2. Calculating the direction angle and length parameters of all the lines, the deviation threshold is set adaptively to get the line segment to be marked. 3. According to the slope of the line segments, the straight line to be removed is extended by 15 pixels at each end in the opposite direction of the line segments. 4. Traverse each pixel on the extended lines and mark them by rules.

Line detection algorithm
The experimental image set is provided by the actual project.
There are 100 tunnel lining surface images taken by professional engineers. We uniformly sample the pixels of all collected images to ensure that the influence of pixels on the image results is minimized. Traversing all the images, the total can be divided into three different lining backgrounds: vertical, horizontal and inclined lining seams. After downsampling, the size of these images is 400 × 300 [pixel] and the purpose of down-sampling the image is to reduce or eliminate the sawtooth effect that appears in the image.
As shown in Figure 4(a), select two representative images in each direction to demonstrate.
After the coarse analysis of the crack clustering feature of the percolation model [35], the crack binary images are obtained. The original tunnel lining surface images are shown in Figure 4(a), and the binary images after the percolation detec-tion are shown in Figure 4(b). For the inherent lining seams in the tunnel lining surface images, LSD is used to detect the line segments in the images of the tunnel lining surface. The linear detection results are shown in Figure 4(c). The contour information of lining seam can be detected in the form of line segments.

Angle adaptation
Each line segment consists of two endpoints, a start point and an end point. According to the coordinates of the start and end points, the length and angle of each line segment are calculated. Then, the main direction of the lining seam in the image is obtained by the weighted average of the length. The deviation threshold is adaptively set according to the main direction to minimize the deviation coefficient of the line segment within the threshold. According to the image selection criteria, the segments within the threshold are reserved as the segments to be marked. Because there are a large number of line segments on the lining seams, and the distribution direction is parallel and consistent, the line segment with an angle within the threshold deviation is most likely to be the lining seam to be demolished. For any straight line segment l in the set of straight line segments, this paper first calculates its length and direction angle. lengt h l is used to represent the length of the straight line segment l , and l is the direction angle of the straight line segment. The calculation formula of angle and length is as follows: where (x 1 , y 1 ), and (x 2 , y 2 ) are the two endpoints of the line segment obtained by LSD algorithm, and Δx = |x 1 − x 2 |, Δy = |y 1 − y 2 | are used in Equation (1). At the same time, for the calculated direction angle of the straight line segment, if there is l < 0 situation, this paper carries out the l = l + correction to the direction angle, so that the direction angle of all the straight line segments in the set of linear segments is within the unified range [0, ), which is convenient for the subsequent unified processing. After getting the information of each line segment, is used to represent the weighted direction angle of the image, and S is used to represent the weighted angle variance of the image.
The calculation formula of angle and deviation parameters is as follows:̄=  Figure 5(b) represents the image after setting the adaptive angle threshold. After obtaining all the information of the image, the red line represents the main direction in the image, and its angle is . We use the angle increase and decrease step size Δ , and change the angle threshold to Δ , 2Δ , 3Δ step by step. Each time, we remove the lines with direction angle outside the threshold, such as the yellow line in Figure 5, and retain the line within the threshold, as shown in the blue line in Figure 5. Then, the angle deviation coefficients of all lines within the threshold range are calculated by Equations (3) and (4), and compared with the deviation coefficients of the original image. The threshold with the minimum deviation coefficient is the final threshold of the image. In this way, the possible direction of lining seams in the picture can be determined according to the inclination direction of each picture, reduces the marking of line segments in the direction of unlined seam. In the subsequent processing, only the lines within the threshold range of the main direction deviation are marked with the rectangle, which can shorten the time of subsequent rectangle marking.

Rectangular marking rules
The line segments in the area to be removed are traversed. By using the Bresenham straight line scan conversion algorithm, extend extentNum in opposite directions from the start and end of the line segment to cover the undetected line on the seam, where the line segment to cover the undetected line on the seam, where extentNum represents the length of the extended pixel. According to the experimental results of recall rate and noise rate of different elongation in Figure 9. When the value of extentNum is less than 15, the coverage of lining seam is not complete, which will leave a lot of noise. When the extentNum is greater than 15, the recall rate will decrease and the result will not improve significantly. The head and tail of the line are extended by 15 pixels respectively, and the total length is increased by 30 pixels. The second step is to calculate the coordinates of the points to be tested according to the direction characteristics of the line segment. We set the coordinates of the pixel to be marked as P (x, y). [P Z ] N (x ± Δx, y ± Δy) indicates that the point to be detected in the mark is the point perpendicular to the line segment where the point P is located. When N is negative, it means that the point is on the left side of the line segment. When N is straight, it means that the point is on the right side of the line segment. The value of N represents the pixel offset of the detection point relative to the pixel P, The calculation formula of the angle and distance of point P relative to the starting point is as follows: = arctan The third step is to calculate the grey level of the pixels perpendicular to the line according to the direction characteristics of the line segment, and mark the relevant pixels around the line segment. As shown in Figure 6, the white area represents a straight line segment with width, p start represents the detection starting point on the left side of the line segment edge, and the remaining blue pixels are the pixels to be detected. The characteristics of lining seams and cracks are analysed, and the rectangular marking rules are formulated. According to this rule, a marked rectangle with a certain width is formed in the vertical direction of the line segment. The blue rectangular area marked with 1 pixel means the lining seam, which needs to be eliminated, while the red rectangle area of pixel marked with 2 means that the junction between the lining seams and the crack needs to be retained. For segments on the right edge of a line segment, use the same rules to form a rectangular marker on the left. The detailed rules of rectangle marking in algorithm 1 are as follows: where G ray[p start ] represents the grey value of the start point, and G ray[p start ] −2 represents the second point on the left side of the start point in the direction perpendicular to the line segment. Through experiments and analysis of rough extraction images of cracks on the surface of the tunnel lining, the displacement offset 2 is set in this paper to represent the offset increment of 2 unit pixels. LSD is the detection of the edge contour of a line. When the second point and the third point on the left of the start point are both background points, it indicates that the line segment is located on the left side of the lining seam. A marking rectangle is formed to the right by the following rule to indicate whether the lining seam is eliminated or reserved.

Rule 2. For the first 5 points and the last 5 points of
where G ray[p 1 ] 9 ] represents the grey value of the ninth point on the right side of p 1 in the direction perpendicular to the line segment. The lining seam far from the crack is composed of some long continuous lines. When p 1 and its first five points and the last 5 points satisfy Rule 2, p 1 and those points which are continuous with p 1 in the direction perpendicular to the line segment will be eliminated. The average width of the lining seams is 3 pixels. In order to make the elimination completely, we set the value of tag to 1 from the second points on the left of p 1 to the fifth points on the right of p 1 . The lining seams far from the crack are marked.
when p 2 and its first 1 point and the last 1 point satisfy Rule 3, we set the value of tag to 1 from the second points on the left of p 2 to the fifth points on the right of p 2 . The lining seams close to the crack are marked.

Rule 4. For point p3,
where p 3 + 8 is the eighth point behind p 3 on the line segment. The average length of the junction between the crack and the lining seams is 8 pixels. When the fourth point on the left side of p 3 and the fifth point, sixth point and seventh point on the right side of the eighth point behind p 3 are crack points, it indicated that p 3 is located in the junction of the lining seam and the crack. In order to retain the complete crack and prevent the fracture, for p3 and the next 8 points behind p 3 , we set the value  Figure 7 shows the results of the rectangular mark.
There are many factors that affect the results of this paper, such as the illumination intensity in the process of image acquisition, the image exposure effect in the process of image preprocessing, the binary processing method, the number of seed points in the percolation algorithm, the pixel extension length and the pixel distance in the rectangle marking rule. But all these factors will ultimately directly affect the image effect or the number of various types of pixels detected. In this paper, the extended pixel length value of the line segment that has a greater impact on the experimental results is discussed in detail.
We use recall (R) and the noise rate (NR) to verify the extentNum = 15. R and NR are defined as follows: Figure 8 is the diagram of the evaluation index, where TP is the number of crack pixels detected correctly in experimental results, FN is the number of background pixels detected incorrectly and is the number of crack pixels detected incorrectly in     experimental results. TP + FN is the number of crack pixels in the artificially extracted images. TP + FP is the number of crack pixels detected in experimental results. The higher recall means that more real crack pixels are recalled in the results. The high noise rate indicates that the probability of false detection of the crack as background in the experimental process is high. Figure 9 shows a quantitative analysis of experimental results of the different extension lengths of the line segments. It can be seen from Figure 9(a) that the recall rate is the highest when extentNum = 15. Figure 9(b) shows that the noise rate is lower when extentNum = 15. When the value of extentNum is less than 15, the noise rate is high. The coverage of the lining seams is incomplete, and there are many lining seams remained on the crack which can't be removed during the process of percolation denoising. When the value of extentNum is greater than 15, the recall rate is reduced and the results are not improved significantly. It will also increase the calculation time and reduce the efficiency of the algorithm. Therefore, in order to achieve the best experimental results, we select extentNum = 15 as the extension length of the line segments considering the impacts on the recall rate and noise rate in this paper.

THE EXPERIMENTAL RESULTS AND ANALYSIS
The image set of this experiment is provided by a real project, and about 100 images of tunnel concrete lining surface are used in the experiment. In addition, because of the external factors such as illumination, different complexity of lining surface, different crack shape and direction, different image background noise and so on, we divided the experimental atlas into six groups for experiments. The experimental computer configuration is as follows: 3.5-GHz Intel Core i7 4790 CPU, Microsoft Visual Studio 2013, opencv3.0.0, C + + is the programming language. At the same time, in order to evaluate the performance of our proposed method, we compare the performance and time with the LS-T&E algorithm.
In this paper, we use the classification accuracy evaluation method based on confusion matrix [43][44][45] to quantitatively analyse the experiment, precision (P), R, NR, false positive rate (FPR), accuracy (Acc) and weighted harmonic mean (F) are used for quantitative analysis. The definitions are as follows: where TN is the number of background pixels detected correctly in experimental results. FP + TN is the number of background pixels in the artificially extracted images. TP + TN is the total number of crack pixels and background pixels which are detected correctly in experimental results. FN + TP + FP + TN is the total number of crack pixels and background pixels in the artificially extracted images. A higher precision means more correct detections are in the result. F 1 can reflect the performance of the algorithm relative completely when taking the parameters =1.
The distribution characteristics of the experimental images we selected are different, so we selected a representative image from six groups of images to show the experimental effect. As shown in Figure 10, the results of marking the straight line segment of the lining joint using the rectangular marking rule. The selection range of rectangle mark is the line segment selected by the adaptive threshold setting method. It can be seen clearly from Figure 10 that the setting of this method can effectively distinguish the position of cracks and lining joints. Figure 11 shows the final result of the algorithm proposed in this paper, and Figure 12 shows the result of LS-T&E algorithm. In order to verify the performance of the algorithm, this paper manually extracts the result image by using engineering technology to draw the crack position on the original image and set the crack pixel to 0, as a comparison reference of the real crack situation in the experiment, as shown in Figure 13. Through comparison, we can see clearly in the figure that the results of our algorithm are closer to the real crack information, while LS-T&E FIGURE 10 The experimental results of lining seam elimination algorithm, the rectangular mark results of Figure 4(b)

FIGURE 11
The experimental results of the proposed method algorithm is connected with the crack in some locations with less crack information and some small burr. Table 1 shows the average performance parameters of six groups of experimental images collected using our proposed algorithm and LS-T&E algorithm. It can be seen from Table 1 that the average p, R and F1 of this algorithm are higher, the average accuracy is more than 90%, and the NR is lower. Compared with the P and R parameters of LS-T&E algorithm, because P and R are the most commonly used indexes, F1 can better reflect the comprehensive index of the results. Generally speaking, the larger the value of P, R and F-measure, the better the effect of the representation method. In a word, the

FIGURE 12
The results of LS-T&E algorithm

FIGURE 13
The experimental results of the lining seam elimination algorithm, the artificially extracted crack results  performance parameters of the two algorithms proposed in Table 1 are consistent with the experimental results. Our proposed algorithm has better performance, better lining joint removal effect and higher accuracy of crack extraction. As shown in Table 2, the algorithm time proposed in this paper is compared with the average processing time of LS-T&E algorithm for six groups of experimental images. It can be seen that the time of this algorithm is much less than that of LS-T&E algorithm. We can also analyse it here, because this paper adopts the adaptive threshold setting method to distinguish cracks and lining cracks. When marking rectangles, only the key areas are marked, but not the edges Not all the lines in the image are processed. Although there will be time to set the threshold, the overall time of the algorithm is significantly shorter.
It can be seen from Figure 14(a)-(c) that the algorithm has higher recall rate and accuracy rate and lower noise rate compared with LS-T&E. Figure 14(d) shows that the accuracy parameter F-measure of this algorithm is higher than that of LS-T&E. As a conclusion, the algorithm in this paper can eliminate the lining seams with different morphological distribution features accurately and efficiently by using the rectangular marking rules and keep the real cracks. The algorithm has the characteristics of high specificity, high sensitivity and strong robustness.

CONCLUSIONS
In this paper, the characteristics of grey, edge and morphological distribution of the lining seam in the tunnel lining surface crack image are fully considered. The rectangular marking rules are adopted in the direction perpendicular to the lining seams to eliminate the lining seam and keep the complete cracks.
The experimental results show that the algorithm can quickly and effectively eliminate the lining seam in any direction, especially the lining seam in the inclined direction. The effect is obviously better than the previous algorithms, and has the characteristics of strong specificity, high sensitivity and strong robustness. This algorithm improves the existing tunnel lining crack detection technology, and enriches the related theoretical research.
In the future, we will continue to study the method of detecting straight-line segments in all angle direction to further improve the accuracy of detection. In addition, we will improve the rules of rectangle marking, so that it has practical value in other detection fields.