A novel weighted total variation model for image denoising

Image denoising is a very important problem in image processing ﬁeld. In order to improve denoising effects and meanwhile keep image structures, a novel weighted total variation (WTV) model is proposed in this paper. The WTV model consists of data ﬁdelity and 𝓁 1 norm based regularisation terms. In the WTV model, a weight function w in exponential form is incorporated into the regularisation term, which only depends on the given image itself without extra parameters. The nonlinearly monotone formulation of w helps to increase gaps between lower and higher frequencies of images, which is effective to high-light edges and keep textures. For solving the proposed model, the alternating direction method of multipliers is explored and the according convergence is analysed. Compared experiments of TV, HOTV, ATV and TV p models are conducted and the results show the effectiveness and efﬁciency of the proposed model.


INTRODUCTION
Image denoising is a fundamental yet still challenging problem in the field of image processing. Images are often polluted by noise, which results in quality degradation and visual discomfort. In order to address this issue, image denoising methods are widely explored. The goal of image denoising is to remove noise from observed images and simultaneously maintain as many image features as possible. A noisy image is often formulated by where u is the clean image, n is additive noise. The problem of restoring images from noisy versions is ill-posed [1] and hence mathematical techniques are required. Many models were proposed to deal with noisy images, such as total variation based models [2], transformation based methods [3], partial differential equation (PED) based models [4] and learning based methods [5] etc. Among them, the total variation (TV) based models is one of the most popular categories. TV based models are widely used in various image applications, such as image restoration [6][7][8][9][10], image deblurring [11][12][13], image decomposition [14][15][16][17], image segmentation [18][19][20][21] etc. TV based models are powerful due to its ability This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. © 2021 The Authors. IET Image Processing published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology of preserving edges while there is a main disadvantage named staircase effect [22,23]. Many modified models are proposed to improve it. In [24,25], higher order extensions of the total variation are proposed, in which high order gradient is used for image smoothing. However they may result in blur edges. Authors of [26,27] consider fractional order derivative and incorporate it into their models due to its effectiveness in denoising. There are also some models apply different orders of differentiation to denoising model, such as TGV [28], hybrid TV [29], nonlocal TV [30] etc. These models are able to reduce the staircase effect arises in TV model.
In sense of calculation, anisotropy and isotropy are studied based on their different performance. In [31][32][33][34], anisotropy is applied and obtains better results than the isotropic ones. Besides, some authors explore the weights on regularisation in different ways in hope of improving denoising effect. In [35], the weighted nuclear norm is designed. The weight is with respect to singular values and different singular values are assigned with different weights, which improves the capability and flexibility of image processing. In [36], they employ the regularisation in form of L p − L q , where the weight controls the balance between p and q norms. This provides new idea for the design of regularisations and performs well in image problems. There are some other regularisations designed for image processing, such as graph Laplacian-based regularisation [37], structure tensor based regularisation [38], adaptive pbased regularisation [39], non-local self-similarity based regularisation [40], sparsity based regularisation [41] etc.
The models mentioned before utilise different regularisations to approximately describe image properties. Then denoising problem is converged into maintaining image properties mathematically described by regularisations. In this sense, it can be seen that proper regularisations are essential to denoising problem. Based on the existing study, in order to improve denoising effect, a novel weighted total variation (WTV) model is proposed in this paper. The contributions are highlighted as follows.
• In this model, an adaptive weight function w is defined according to the effect of noise and incorporated into the regularisation term. The weight function in the WTV model is defined with respect to image gradient without extra parameters. The nonlinear formulation of weight w makes it better to keep images structures and textures effectively. • In order to solve the proposed WTV model, alternating direction method of multipliers (ADMM) [42] is explored, whose convergence is guaranteed [43]. • In order to demonstrate the advantages of our model, compared experiments on both gray and colour images are conducted. Visual effect and quantitative evaluation demonstrate the effectiveness and efficiency of the proposed model.
The rest of this paper is organised as follows. In Section 2, we describe some notations and brief review of some related models; in Section 3, the proposed model is formulated, which is solved by ADMM and the convergence is analysed. In Section 4, numerical experiments are conducted to demonstrate the effectiveness and efficiency of the proposed model. Finally, we conclude the paper in Section 5.

PRELIMINARIES
In this section, some notations and related models are introduced briefly.

Notations
where x u, y u are gradients of u in x and y directions respectively. They are computed by difference as follows.
x and * y are adjoint operators of x and y respectively. They are computed by *

TV model
The well-known TV model [44] was proposed by Rudin et al. in 1992. It is formulated by where > 0 controls the trade-off between data fidelity and regularisation, u is the desired image. Larger helps to maintain features of images while smaller is convenient to remove noises. In this model, 1 norm of image gradient is used to regularise images and performs well in maintaining edges during denoising. It is widely applied in many image problems even though it can generate staircase effect during processing.

HOTV
The HOTV [25] model utilises high order differential of image gradients for regularisation and is formulated as High order gradients combined with 1 norm are used to regularise images, which improves the staircase of TV model. However, high order differential may cause excessive smoothness, which will result in blur edges. In addition, the calculation of high order differential is more complicated and costs longer computational time.

ATV model
Recently, Z.Pang et al. proposed a new anisotropic totalvariation-based model (ATV) [34] to remove noises. The ATV model is described as In this model, the authors consider anisotropic property and incorporate an adaptively weighing function T into the regularisation. The weight function is defined by T aims to describe local features of images. However, k andâ re additional parameters, which may increase experiments for values selection.

TV p model
In [39], the authors proposed a novel p -regularisation based total variation model (TV p ) for image restoration. In denoising application, the TV p model is formulated as follows In the TV p model, an adaptive exponent weight p(x) is defined and incorporated into the regularisation. Local information of images is utilised in the definition of p(x), which is effective to keep image structures. However, it may result in blurring for details.

THE PROPOSED MODEL
In this section, a new weighted total variation (WTV) model is proposed for denoising. The WTV model is designed for additive white Gaussian noise (AWGN). It consists of fidelity and regularisation terms and is formulated as where is a parameter of fidelity term, f ∈  M ×N is a noisy image, w is the weight defined by an exponential function as follows In order to clearly explain the WTV model, in the following firstly the motivation of modelling is analysed. Secondly, solving method of the proposed model is explored. Finally, convergence analysis is provided.

Motivation
In the denoising problem, it is natural to consider the effect of noise on images. Denoting the standard deviation of AWGN as , we take image Lena as an example to show the effect of AWGN. Gradient amplitudes of clean and noisy images are observed in Figure 1. For convenience, a single row of Lena is taken and displayed in different noise levels. In Figure 1(a,b), the horizontal axis represents the number of pixels, the vertical axis represents amplitude, standard deviations of f are 0.05 and 0.1 respectively. It is seen in Figure 1(a) that smaller gradient amplitudes, such as those less than 0.05, are greatly increased to 0.1 or more with the exist of noise. Compared Figure 1(a,b), it is also can be seen that noise increases image amplitudes of areas that are originally lower in amplitude, even noise with larger standard deviation causes more severe disturbance to originally lower amplitudes. While the larger amplitudes of ∇u are less affected by noise. Based on the fact above, the regularisation term E R in Equation (11) is designed to deal with the effect of AWGN on images. The incorporated weight function w is nonlinearly increased with respect to amplitudes of image gradients. Larger weights are assigned to higher gradient amplitudes (or higher frequencies) and smaller weights are assigned to lower ones. As is shown in Figure 1(c,d), w can widen the gap between higher and lower frequencies, and hence the higher frequencies are highlighted in the whole image domain. Therefore w is helpful to suppress noise in lower frequencies and enhance edges in higher frequencies.
In order to show the advantage of the proposed regularisation intuitively, Figure 2 draws gradient amplitudes of Lena with and without noise. Figure 2(a) is for the clean image, Figure 2(b-d) are for images polluted by AWGN with standard deviation 0.05. In Figure 2(a), the object inside the image domain is clearly contoured. In Figure 2(b), noise is around all the region and seems to be as important as the character. In Figure 2(c), T ∇ f from the ATV model in [34] is visually similar to Figure 2(b). In Figure 2(d), w∇ f from the proposed model, the character is more prominent than noise, which means noise is suppressed to some degree. Therefore w is effective to suppress noise and emphasise edges.

Solving method
In order to solve model (10), the ADMM is explored. For convenience, let h 1 = w 1 x u, h 2 = w 2 y u. Model (10) can be reformulated as the following constrained optimisation problem.
The Lagrangian function of model (13) is given by where r 1 and r 2 are Lagrangian multipliers, ‖(h 1 , h 2 )‖ 1 = ‖h 1 ‖ 1 + ‖h 2 ‖ 1 . Problem (13) is equivalent to the following problem max Problem (15) can be split into the following subproblems: Now we solve them successively. Minimising Equation (16) with respect to u, the optimality condition is derived as follows Utilising periodic boundary condition and forward difference, the left side of (19) is circulant matrix, which can be diagonalised by fast Fourier transform (FFT) [45]. Model (19) can be efficiently computed by where * x is the adjoint operator of x ,  is fast Fourier transform (FFT) and  −1 is the inverse transformation.
As for subproblem (17) and (18), they can be conveniently calculated by a generalised shrinkage [31]: where and The stop criterion for iterations is defined by the relative error between the kth and k + 1-th iterations as where tol is the tolerance. The calculation will stop if err reaches to tol within finite iterations. The maximum number of iterations is denoted by MaxIter.
The procedure of solving model (13) or model (15) is summarised in Algorithm 1. The convergence analysis will be discussed as follows.

Convergence analysis
In this subsection, we mainly analyse the convergence property of solutions generated from Algorithm 1.
Proof. In fact, {u k , h k 1 , h k 2 } is a minimiser of Problem (15). Hence it satisfies Considering the second order Taylor expansion of , combining Equation (25) we can approximate the following result Since that therefore it can be deduced Summing Equation (29) from k = 1 to k = +∞ in both sides and noticing the boundedness of the left side, we can obtain the following results It is deduced that there exists unique (u * , h * 1 , h * 2 ), such that the sequence {u k , h k 1 , h k 2 } converges to (u * , h * 1 , h * 2 ). Then the proof is completed. □

NUMERICAL EXPERIMENTS
In this section, the proposed WTV model is compared with TV [44], HOTV [25], ATV [34] and TV p [39] models. All the numerical experiments are performed via MATLAB (R2016a) on a windows 7 (64bit) desktop computer with an Intel Core i5 3.20 GHz processor and 4.0 GB of RAM. The compared models are programmed following related references of them by ourselves. All the test images from [46] and Berkeley BSDS500 (https://www2.eecs.berkeley.edu/Research/Projects/CS/ vision/grouping/resources.html) are normalised into [0,1]. In order to evaluate the quality of denoised images, two evaluation indicators are computed, i.e. the structural similarity index (SSIM) and peak signal to noise ratio (PSNR) [27]. Larger PSNR means less noise after denoising progress and larger SSIM reflects higher similarities between denoised and clean images. SSIM and PSNR are computed by MATLAB functions SSIM and PSNR respectively. In order to show the efficiency of these models, computational time is also compared.

Experimental results
In Figure 3, test images for denoising include both gray and colour ones. The AWGN level denoted by presents the standard deviation of noise, which takes 0.05 and 0.1 in experiments. For calculation, the max number of iteration MaxIter is 500, the tolerance tol = 10 −5 for the five compared models. In the proposed WTV model, is tunable, which is important to control the structure similarities of images.
In order to show the effectiveness of the proposed model, compared experiments are conducted on both gray and colour images. In Figures 4 and 5, test images are gray and noise levels are 0.05 and 0.1 respectively. Among them, the five rows from top to the bottom are results of TV, HOTV, ATV, TV p and the proposed WTV models respectively. It is seen that WTV and ATV obtain more sharp edges than the others. This is easy to observe from image rose. With the increasing noise level, WTV model obtains more clean results, which is more apparent for image man, lady and castle.
In Figures 6 and 7, noisy images are colour and noise levels are 0.05 and 0.1 respectively. Among them, the five rows from top to the bottom are results of TV, HOTV, ATV, TV p and the proposed WTV models respectively. It can be seen that the five methods perform well for colour noisy images.
For better observation of the results by these compared models more clearly, enlarged local regions of denoised images contoured in Figure 3. We take noise level 0.1 as examples. In Figure 8 enlarged local regions of them drawn in Figure 3 are displayed to show the details. It is seen that in Figure 8, the 8 columns from left to right are local regions of denoised man, monarch, rose, lady, zebra, castle, building and baboon. The first row is local regions of clean images, the last five rows are local regions from the results of TV, HOTV, ATV, TV p and the proposed WTV models. Seen from these local regions, TV and TV p are vulnerable to blurring details. While WTV, ATV and HOTV are effective to maintain image details for gray images. In Figure 9, enlarged local regions of denoised colour images are shown. The first row is original and the last five rows are from results of TV, HOTV, ATV, TV p and WTV respectively. It is seen that local regions from WTV results are more sharp than those of the others, especially for image lena and bird.
Besides visual effect comparison, quantitative measures are compared to show the effectiveness and efficiency. Two representative image quality metrics, PSNR and SSIM, are provided in Tables 1 and 2 in case of noise levels = 0.05 and = 0.1 respectively. It can be seen that the proposed WTV model results in higher PSNR and SSIM among the compared models. It means the results by WTV model are more clean and have larger similarities between denoised images and the clean ones. This demonstrates that the WTV model is more effective to remove noise and meanwhile maintain image structures. This benefits from the properly designed weighted function w in our WTV model. Its exponential formulation regards gradient makes it powerful to outstand edges and maintain details. Therefore the recovered images are relatively clean and sharp.
In order to show the efficiency, computational time of the compared models is provided in Table 3. It is seen that the proposed WTV model costs less computational time than TV p , ATV and TV models while HOTV costs the most computational time. This is caused by the high calculation of high order gradient. Based on the comparison above, the proposed WTV model gets better image qualities with the least computational time.

CONCLUSION
In this paper, a novel weighted total variation model is proposed for image denoising problem. There are three main points to be summarised. First, an adaptively defined weight function w is incorporated into the regularisation term to enhance edges and meanwhile suppress noise. Second, ADMM is explored for solving the proposed model and the convergence is briefly analysed. Finally, compared experiments demonstrate the effectiveness and efficiency of the proposed WTV model. In our future work, the idea of WTV model may be applied to other image problems, such as deblurring, bias correction and image segmentation etc. In addition, since the proposed model is test on synthetic data, experiments on real image data will be also explored.