Toward developing tangling noise removal and blind inpainting mechanism based on total variation in image processing

Intheﬁeldofimageprocessing,tanglingnoiseandartefactseliminationofobjectsaretwoessentialtasks.Tanglingnoiseandlackofintensity in certain applications also occur at the same time. In this paper, a new variational model is proposed based on total variation and l 0 the norm for simultaneously removing the tangling noise, estimating the location ofmissingpixels,andﬁllinginthem.Tobespeciﬁc,thetotalvariationisusedtoregularizetheestimatedimageandusethe l 0 norm to make the missing pixel to be sparse. Moreover, the data ﬁdelity term is given by a new forward description about the degraded process and the gamma noise assumption. Finally, an algorithm based on the alternating direction multiplier method is exploited to solve the model. By conducting simulated and real experiments, the damaged images can be effectively restored by the proposed method. In qualitative and quantitative terms, this approach works better.


✉ Correspondence
Email: jkim@dongguk.edu In the field of image processing, tangling noise and artefacts elimination of objects are two essential tasks. Tangling noise and lack of intensity in certain applications also occur at the same time. In this paper, a new variational model is proposed based on total variation and l 0 the norm for simultaneously removing the tangling noise, estimating the location of missing pixels, and filling in them. To be specific, the total variation is used to regularize the estimated image and use the l 0 norm to make the missing pixel to be sparse. Moreover, the data fidelity term is given by a new forward description about the degraded process and the gamma noise assumption. Finally, an algorithm based on the alternating direction multiplier method is exploited to solve the model. By conducting simulated and real experiments, the damaged images can be effectively restored by the proposed method. In qualitative and quantitative terms, this approach works better.
Introduction: Problems with image inpainting [1,2] have been highly attractive for modern applications. Noise is classified as additive and multiplicative noise in images. In this paper, we emphasize mainly the problem of tangling noise removal [3]: When f: → R + is the observed image ( ⊂R 2 ) is a connected bounded open subset with Lipschitz compact boundary), v ∈ L 2 ( ) denote tangling noise with a mean 1, and K ∈ l(L 2 ( )) is a known linear and continuous bubble operator as mentioned in Equation (1). The aim is to find the true unknown image u from the degraded image f [4].
The image quality is also diminished by the pollution of noise, creases, and marks in image generation and preservation. Some photographs (like old pictures, medical images etc.) are often introduced to varying degrees of noise pollution because of photographic equipment and technology shortcomings. Also, there are creases or scratches in photos because of improper preservation, resulting in missing some pixels. In some medical images [5], radiologists always make some artificial marks in their focus area, which leads to some difficulties in further computer-aided processing and evaluating these images. Any pixels in an image can be lost or corrupted by impulse noise due to a defective image sensor or bit error during transmission. Therefore, we need to restore the damaged images as clearly as possible. This paper focuses on two problems of degradation: Tangling noise and artefacts in images; for this, we propose a model for removing noise and pixel filling. Since we do not know where the missing pixels are in the damaged image, this pixel filling problem is called blind inpainting.
To describe the degraded image with tangling noise and missing pixels, let Ω represent the entire image region, and Ω 0 denote the missing pixel region that needs to be filled. Because the tangling noise only exists in the image region, we can express the damaged image as, where f is an image corrupted by tangling noise and partial pixel missing, u [0, 255] represents the ideal clear image, and η denotes the tangling noise. In region Ω 0, where the real data is missing, the pixel value is 255. Our problem is to fill the pixels in region Ω 0 with information from ΩࢨΩ 0 while removing the tangling noise.
Over the previous few years, there have been various studies on image restoration using digital inpainting and noise removal [6,7], a novel denoising model for low-level image restoration [8], removing tangling noise [9], or filling missing pixels. However, few methods tackle both the removal of tangling noise and the filling of missing data when its location is unknown. Blind painting is more realistic because, for most corrupted images, the area of missing pixels is not known in advance. Several researchers used various deep learning approaches for image inpainting [10,11]. In [12], Dong et al. proposed a tight frame model for blind inpainting and removing additive Gaussian white noise. The l 1 norm was used to regularize missing pixels sparsely. In [13], Yan used l 0 norm instead of l 1 norm to better detect pixels damaged by impulse noise. In [14], Afonso multiplied Rayleigh or Poisson noise on an image with a mask which was a random matrix with pixel values of 0 or 1, to represent the damaged image. They then used the log transformation to change the multiplicative problem into an additive one. They replaced the variables to be sought with new variables (log transform of the original variables). Similarly, the total variation on the log image and the l 0 norm on the log matrix of the mask is employed to regularize them.
In this paper, based on [14], we use a new expression to represent the damaged image and propose a new model for the energy functional minimum problem. The model is composed of three parts. One is the total variation regularization of the estimated image to smooth it; one is the sparse regularization using the l 0 norm of missing pixels, which is utilized to constrain the number of pixels whose pixel values are not 0. The third term is a fidelity term, which is different from [14] in the following aspects: (I) Our data fidelity term is based on the gamma multiplicative noise, but the data fidelity term in [13] is based on the Rayleigh or Poisson distribution. (II) The second one is that we get the term from the new forward degraded expression other than the multiplicative representation in [14]. (III) Finally, we select an algorithm based on the alternating direction multiplier method (ADMM) to resolve our model.
The rest of the paper is organized accordingly. In Section 2, we present a new technique for removing gamma tangling noise and blind inpainting. Section 3 is designed to solve the proposed problem using an algorithm based on the ADMM. Section 4 displays the comparison experiments between the method used in [14] and the proposed model. We use synthetic and actual degraded images for experimental images. Finally, in Section 5, we provide a brief conclusion on the model and the experimental results.
Proposed tangling noise removal mechanism: Similar to the representation in [11], we rewrite the damaged forward process (1) as the new formula, where v represents a random matrix, ηG means the tangling noise obeying the gamma distribution with a mean of one, and its probability density function is.
Based on (3) and (4), we can get the following probability density function The data fidelity term corresponding to (5) is.
So, the following model based on the total variation and l 0 norm to remove the tangling noise and achieved blind inpainting.
Where ||log u|| TV is used for smoothing the tangling noise as [13], ||v|| 0 signifies the amount of non-zero elements in the random matrix v. Observing the log term in data fidelity (6), we introduce a new variable z = log u , and our model can be reformulated as as were, J(u, v) is transformed to: Numerical algorithm: In this section, we plan to use the algorithm based on ADMM to solve (8). However, we can see that the two variables z and v to be solved in (8) cannot be separated. Therefore, we introduce two auxiliary variables ω and d to replace u in the total variational regularization term and ν in the l 0 regularization term, respectively. The model is transformed into a minimum problem with linear constraints: subject to ω = z, d = v Then, the augmented Lagrange algorithm is used to transform (10) into an unconstrained minimization problem: Finally, our model can be solved by the following subproblems: When solving each subproblem, the other variables are fixed as constant by the results of the last iteration. Therefore, by alternating iteration, we have the following iteration scheme to solve (11). When solving each subproblem, the other variables are fixed as constant by the last iteration results. Therefore, by alternating iteration, we have the following iteration scheme to solve (11): where (18) is the Chambolle's projection algorithm for solving TVl 2 problem (12), (19), and (21) are the results of Newton's method for solving (13) and (15), and (20) is the hard thresholding for solvingl 0 − l 2 problem (13). and In this paper, Chambolle's projection algorithm is adopted to solve (11). Specifically, Through the following iteration scheme, we can acquire the optimal solution p* where γ is the step size and (.) = ( x , y ) is the gradient. A more detailed solution process can be seen in [3]. In conclusion, the complete algorithm to solve (10) is as follows Experiment results: This section aims to validate the proposed technique dominance compared to Afonso [14]. The [14] model is equipped for tangling noise from Rayleigh or Poisson but not gamma noise. Therefore, we apply their approach as the approach of contrast to the gamma noise. The peak signal to noise ratio (PSNR) and structural similarity index measure (SSIM) are chosen to measure image restoration quality. The PSNR is calculated as follows: Where u, u * are the original image, restored image respectively, and | | denotes the size of the image. Our experiment consists of a simulation experiment and a real experiment. Firstly, the damaged synthetic images used in the simulation experiment are combined with the tangling gamma noise with the mean of one and a mask. The intensity of the missing pixel position in the damaged image is 255, and the gamma noise contaminates the intensity of the remaining pixels. The final restoration  [13]; (e) v got by [14]; (f) u got by [15]; (g) v got by [15]; (h) u got by the proposed method; (i) v got by the proposed method  [14]; (e) v got by [14]; (f) u got by [15]); (g) v got by [15]; (h) u got by the proposed method; (i) v got by the proposed method  [13]; (e) v got by [14]; (f) u got by [15]; (g) v got by [15]; (h) u got by the proposed method; (i) v got by the proposed method    results are described in Figure 1 and 2 using the methods [14,15] and the proposed method. Our first experiment added a tide mask image featuring different text and evaluated it with Set-14 dataset zebra and monarch images. We observed that the method of [14] and [15] could not fully recognize the location of missing pixels, and some pixels have not been filled in. Also, it can be realized from Figure 1(d), (f) and Figure 2(d), (f) that the method of [14] and [15] over-smoothed the details and edges and failed to protect them well. In our simulated test, we choose images of "monarch" of size 768 × 512, "zebra" of 586 × 391, and "coastguard" of size 352 × 288, which are shown in Figures 1-3, and their PSNR and SSIM values are shown in Tables 1 and 2. In the second experiment, the coastguard image from the Set14 dataset is tested with a triangle shape mark, as shown in Figure 3(a). We can see that our proposed model has a better restoration effect. Similar to the simulation experiment's problems, the method in [14] and [15] fails to recognize and fill in the missing pixels fully, and some details are missing. The proposed technique can correctly recognize the location of missing pixels and paint them to some extent. Furthermore, our model kept better edges while removing the tangling noise. All experiments were performed on MATLAB 2019a on a window-based server with 64 Gb of RAM. The λ_2 is susceptible to studying the missing pixel values and l 0 norm regularisation parameters to see the mask in our proposed method.

Conclusion:
In this paper, based on a new expression of the degraded forward problem, we proposed a variation model to eliminate tangling noise and introduce blind painting by regularizing total variation on estimated image u and l 0 norm on the estimated artifacts matrix. By the new expression, the data fidelity concept was naturally given by the gamma noise probability density function. We adopted the algorithm based on ADMM to make the model easier to solve. Finally, the experiments were carried out using synthetic and realistic images. The experiments showed that our approach is better at eliminating tangling noise and blind images simultaneously than with the method used in [14] and [15].