Texture and exposure awareness based reﬁll for HDRI reconstruction of saturated and occluded areas

High-dynamic-range image (HDRI) displays scenes as vivid as the real scenes. HDRI can be reconstructed by fusing a set of bracketed-exposure low-dynamic-range images (LDRI). For the reconstruction, many works succeed in removing the ghost artefacts caused by moving objects. The critical issue is reconstructing the areas which are saturated due to bad exposure and occluded due to motion with no ghost artefacts. To overcome this issue, this paper proposes texture and exposure awareness based reﬁll. The proposed work ﬁrst locates the saturated and occluded areas existing in input image set, then reﬁlls background textures or patches containing rough exposure and colour information into located areas. Proposed work can be integrated with multiple existing ghost removal works to improve the reconstruction result. Experimental results show that proposed work removes the ghost artefacts caused by saturated and occluded areas in subjective evaluation. For the objective evaluation, the proposed work improves the HDR-VDP-2 evaluation result for multiple conventional works by 1.33% on average.


INTRODUCTION
In the field of image processing, the dynamic range measures the luminance difference between the darkest and brightest region of an image. The standard digital cameras have an insufficient dynamic range comparing with the dynamic range of the natural scene. It is common for the standard camera to produce extremely over-or under-exposed images for a backlighting scene, such as a sunset. To revolutionise the LDRI produced by standard cameras, the high-dynamic-range image (HDRI) has been developed. HDRI has a wider dynamic range because of the capability of storing much richer luminance information. Wider dynamic range means better visibility of both shadow and highlight areas which can benefit photography and entertainment industry. To obtain HDRI, both hardware and software solutions have been proposed. Researchers usually customise the optical and sensor system of a camera such as Nayar et al. [1] and Tocci et al. [2] to achieve the HDRI capturing. Customisation enables the camera to capture images at two or more different exposure configuration simultaneously, but also rapidly raises the cost of the device. High cost keeps the hardware solution away from popularisation. About software This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. © 2021 The Authors. IET Image Processing published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology solutions, some works are trying to use a single LDRI to reconstruct HDRI such as Eilertsen et al. [3], but their results are in low-quality because of limited source image. Many works try using the standard camera to take several LDRIs of a scene at different exposure, forming a set of bracketed-exposed LDRIs then reconstructing HDRI. Because the luminance information between multi-exposure images is complementary, the camera response of these images can be determined by method like Debevec et al. [4] or Grossberg et al. [5]. With the restored camera response, methods like Debevec et al. [4] manages to merge luminance information and reconstruct HDRI.
HDRI reconstruction by a set of bracketed-exposed LDRIs requires that the input images are taken at different exposure and time, and assumes that the coordinate of the same object among different input images is not changed in the set. This assumption results in an issue: if moving objects are existing in the input images or input images are captured by a handhold camera, the ghost artefacts appear. This issue prevents HDRI from wide application in portrait photography, since it is hard for a human model to stand still naturally for several seconds.
Many works are published to make HDRI technology suitable for reflecting a vivid scene that human, animals, or other To guarantee that no ghost artefact appears in reconstructed HDRI, it is important but challenging to find accurate correspondence between different images. The reason for being challenging is that the material images of HDRI reconstruction are at different exposures. According to correspondence, ghost removal works are supposed to rearrange each image's structure and luminance information then reconstruct HDRI.
This paper proposes refill algorithms to tackle a remaining problem in the ghost removal topic. Chapters of the paper are arranged as followed: Chapter 2 introduces the representative ghost removal approaches, discusses the challenge problem which remains unsolved, and states the contribution of proposals; Chapter 3 introduces the framework of proposed work and two refill strategies: (1) texture and spatial restrictions based texture awareness refill, and (2) surrounding area analysis based exposure awareness refill; Chapter 4 reveals the experimental results on the sets containing saturated and occluded areas; Chapter 5 concludes the proposed work.

RELATED WORKS
Many ghost removal works are proposed to remove ghost artefacts. These works usually require three LDRIs at different exposure as input, because inputting more than three images increases the computation complexity significantly. Ghost removal works can be categorised into three groups: rejection-based, alignment-based and end-to-end neural network. Figure 1 compares the fundamental idea of first two types.

Rejection-based ghost removal methods
The rejection-based methods consider middle-exposed image as the basement of reconstruction because this image has fewest under-and over-exposed pixels, then find the relatively moving objects in other input images. Basing on the findings of moving objects, the rejection-based methods reject the merge for these objects during the reconstructing procedure. Many works are proposed with different approaches to detect the motion. Jacobs et al. [6] measure the local entropy between different input images to indicates the motion existing in the set. Gallo [7] first adjusts the exposure of input images to the same level, then compotionthe pixel values between different image. Wu et al. [8] observe the values of pixels across the set are monotonically increasing or not, and then they compare the value after increasing or decreasing the exposure of the pixel to estimate the motion. Pece and Kautz et al. [9] first compute median threshold bitmaps for each input image then generate a motion map. Heo et al. [10] use the Gaussian-weight distance and energy minimisation to determine ghost regions smoothly. Zhang and Cham et al. [11] propose measuring the motion by estimating the image gradient. Lee et al. [12] and Oh et al. [13] introduce rank minimisation to reject the moving objects and reconstruct the HDRI. However, methods in this category waste the luminance information of the moving object, especially when the motion of a set is complicated and significant.

Alignment-based ghost removal methods
The alignment-based method is good to preserve all luminance information. Works in this category align all the input images before merging them into a single HDRI. Normally, the alignment can be broken down into three steps: (1) Adjust the exposure of reference image to the same level of nonreference image,(2) match the similar pixel patches between reference image (usually the middle-exposure image and other images. 3) Rearranging the location of each pixel patch of the non-reference images according to match pairs. After the rearrangement, the lower-exposed and higher-exposed images look exactly like the reference image (middle-exposed one), except these images are at different exposure. The alignment-based methods regard the luminance information of non-reference images as raw material, then assemble the structure information of reference image. Procedure after the alignment is merging the aligned images into a single HDRI by utilising method such as Debevec. This workflow makes alignment result profoundly affect the quality of reconstructed HDRI.
Methods in this category usually use optical flow or patchbased synthesis to find the matches and align between reference and non-reference LDRI. Bogoni et al. [14] firstly introduce the optical flow to alignment the input LDRIs of a set. Kang et al. [15] propose global registration to against the flaw in alignment. Jinno and Okuda et al. [16] bring the Markov random field to solve the alignment problem. Zimmer et al. [17] use an energybased optical flow optimisation to make different images aligned. Sen et al. [18] and Hu et al. [19] introduce patch-based mechanism into deghost. Hu et al. [19] has optimisation on reconstruction of saturated areas. The results of Sen et al. [18] are considered as a milestone. They propose a patch-based method to achieve robust matching and alignment, then use a variant of PatchMatch to reconstruct HDRI with maximised similarity with the reference LDRI.

Deep learning based ghost removal methods
Since deep learning technology has the magical improvement on several topics of image processing in recent years, this technology is also introduced to simulate the procedure of alignment-based works, such as Kalantari et al. [20] and Deng et al. [21]. Kalantari [20] firstly introduce convolutional neural network (CNN) for modelling the HDRI merge process with ghost removal ability. To improve the Kalantari et al. [20], Deng et al. [21] make efforts on modelling both alignment and merge process to achieve high-quality ghost-free HDRI. In recent years, end-to-end neural network for HDRI reconstruction offers a breakthrough. Wu et al. [22] use translation network to learn how input LDRIs become HDRI. By doing so, they formulate the HDRI reconstruction as an image translation problem to neutralise foreground motion. Yan et al [23]. design attention modules based end-to-end HDRI reconstruction neural network. Attention modules also learns the relationship between input LDRI and reconstructed HDRI, it automatically highlight the helpful details in the non-reference image and block the undesired parts which result in misalignment and ghost artefacts.

Problem in conventional works and the solutions
It is difficult for conventional works to prevent ghost artefacts from appearing in saturated and occluded areas and accurately reconstruct these areas.
As Figure 2(a) shows, for an input set of three images, if an area from the set is occluded by moving objects in the lower-exposed image (image with mark 1), and simultaneously saturated due to exposure changing in other images (image with mark 2 and 3), structure and luminance information is entirely absent. However, like the discussion in previous section, the structure of reconstructed HDRI is the same as reference image. Since the area is not occluded in reference image, deghost works is supposed to reveal its structure and luminance information in reconstructed HDRI. This contradictory circumstance leads to the misalignment for alignment based method or wrong mapping for end-toend neural network based method, further causes the appearance of ghost artefacts on the reconstructed HDRI like Figure 3.
To tackle the HDRI reconstruction issue brought by saturated and occluded areas at its roots, this paper proposes refilling image fragment to provide the structure and luminance information for this kind of areas. As shown in Figure 2(b), the refilled structure and luminance information is supposed to persist in these areas, then improve the alignment and reconstruction result. The target of this paper is removing 99% of ghost artefacts brought by the reconstruction of saturated and occluded areas. To achieve this target, this paper makes the following contributions:  • This paper introduces the first research on improving the reconstruction of saturated and occluded areas. This paper proposes texture and exposure awareness based refill -refilling adequate textures or rough colour patches in the saturated and occluded areas to improve these areas' reconstruction. • The proposed work is capable of integrating and improving the performance of multiple conventional ghost removal works. From the aspect of other ghost removal works, this paper proposes a pre-processing algorithm, which means proposed work can improve various ghost removal research, especially alignment-based works.

FIGURE 4
The framework of proposed work, and the relationship with conventional works 3 PROPOSED WORK

Framework
The main idea of proposals is altering the saturated and occluded areas by two refill strategies. Figure 4 explains the concepts of proposed methods. For a set of three images, the locating method filters out the areas have motion in lower-exposed image (image with mark 1 in Figure 4) and simultaneously saturated in middle-exposed image (image with mark 2 in Figure 4). The proposed work refills adequate textures or patches with rough exposure and colour information into the located areas of lower-exposed image by two different strategies after locating saturated and occluded areas. Table 1 lists the notations used in this paper. Figure 4 also shows the input, output and the relationship between proposed methods and ghost removal methods. The proposed methods require a bracket-exposed set containing three LDRIs as input and output the LDRIs after refill. As a result, from the aspect of other HDRI ghost removal works, this paper proposes a pre-processing algorithm.

Saturation and motion filters based saturated and occluded area locating
To minimise the problem brought by saturated and occluded areas, these areas' location information is essential. The paper designs saturation and motion filters based saturated and occluded area locating approach to finding out these areas. The locating method is designed to extract the motion information from lower-exposed image to middle-exposed image and the mask of saturated areas on middle-exposed image, then do filtering.
The motion from lower-exposed image to middle-exposed image is provided by optical flow-based tool. This kind of tool can estimate each pixel's rough motion direction and distance Average pixel value of a target area before refill at coordinate (x, y) P 2 x,y Average pixel value of a target area after texture awareness refill at coordinate (x, y) D x,y Colour distance of pixels at (x, y) between result of texture and exposure awareness refill T D Threshold of colour distance by analysing two sequential images. Optical flow output is a vector map containing the vertical and horizontal motion amount of each pixel. Although these amounts are not in high precision, but they are efficient enough for the locating method to determine an area is static or dynamic. This method measures the shift amount by the scalar function 1 to reveal the motion amount in scalar:

FIGURE 5
Filters applies in lower-and middle-exposed image to locate the saturated and occluded areas where M represents motion amount in vector, ΔM stands for scalar motion amount, x and y are the coordinates of pixel. Pixels which have significant motion are filtered out by Equation (2): where L (x,y),M represents the filter output at coordinates (x, y) of lower-exposed image, 1 and 0 is the value of mask, correspondingly, T M is the threshold of motion amount. As for the saturation filter, it operates on middle-exposed image by two steps: first is converting the RGB value to greyscale for each pixel of middle-exposed image by Equation (3): where P stands for pixel value, r, g and b is the value of red, green and blue channel correspondingly. After that, it filters out the saturated areas by equation 4: where L (x,y),S represents the filter output at coordinates (x, y) of middle-exposed image, G x,y means greyscale value, T G is the threshold of greyscale value. Next is intersecting the filtered area by Equation (5): where L x,y is the intersected result of two masks. Figure 5 shows the intersection in concept level. This locating method assumes lower-and middle-exposed images are backgroundaligned. With this assumption, the intersection does not require coordinate converting. The relationship between the value of L x,y and mask of saturated and occluded areas (target areas) is shown in Table 2.
There are two post-processes for the intersected result. First is removing areas smaller than T A to prevent the set from being damaged (refilling too much) as much as possible. Second is Overall of texture and exposure awareness refill dilating the remaining located areas for eliminating the small internal hole, besides, it is also beneficial to following procedure (refill) because dilation makes the edge smooth.

3.3.1
Overview of refill methods After locating the target areas, proposed work is ready for refill. As shown in Figure 6, texture and exposure awareness based refill represents two different refill strategies. The absence of structure and luminance information is responsible for the existence of ghost artefacts related to saturated and occluded areas. If the appropriate information is present in these areas, the issue is supposed to be solved at its roots.
With the purpose of providing appropriate information for the type of areas which this paper cares about, this paper proposes refilling new image fragment in target areas as the solution. Refilling new image fragment in target areas requires borrowing fragments from other areas of the same image. The refill is applied in lower-exposed image only rather than all input images. This design is because lower-exposed image has a relatively good exposure in a set of images containing saturated and occlude areas. Good exposure means that there are enough wellexposed image fragments can be borrowed by the refill method. The refill exploits the match estimation mechanism, which alignment-based method usually has. Despite the refill, the target areas are still unmatchable due to saturation at middleexposed image. Like Figure 2 shows, thanks to this mechanism, the refilled image fragments are supposed to persist in alignment result and remain at just where they are before alignment. Consequently, the refilled image fragment has a significant influence on both alignment and reconstruction result. This theory can also be applied in deep learning based methods, since methods in this category also aim to find correspondence between different input image.
Taking this theory into account, this paper proposes two types of refill methods as Figure 4 stating: (1) Texture and spatial restrictions based texture awareness refill; (2) Surrounding area analysis based exposure awareness refill.
The texture awareness refill is the main force. It provides adequate background texture for the target areas to reveal the appearance of occluded areas. Since the refilled fragment is supposed to persist in the target areas, texture awareness refill can provide rich and realistic details for target areas and complement the structure and luminance information which is totally lost. This capability improves the result of alignment and HDRI reconstruction.
The surrounding analysis based exposure awareness refill is the supplement of the first refill strategy. It replaces the result of texture awareness refill when necessary. To determine the necessity, this paper proposes two check mechanisms as shown in Algorithm 1: for a located area, checking its exposure similarity between original pixel values and result of texture awareness refill, and checking the colour distance between the direct output of two proposed refill strategies. Two checks judge if the exposure awareness refill has the ability to provide different exposure and colour information. If both judgements are positive, the exposure awareness refill replaces the result of previous refill strategy. This paper uses the following equation 6 to check the exposure similarity: ALGORITHM 1 Workflow of texture and exposure awareness refill where S stands for exposure similarity. P 1 and P 2 mean the average pixel value of a target area before and after texture awareness refill. The colour distance between pixels of two fragments is calculated by Equation (7): where D x,y represents colour distance of pixels at (x, y). The distance between the direct output of two proposed refill methods D A is an averaged distance. T S and T D are the thresholds of similarity and colour distance, respectively. When S is lower than T S and D A is larger than T D , the exposure awareness refill covers the target area. In the following paragraphs, the detailed concept and implementation of two refill methods are revealed.

3.3.2
Texture and spatial restrictions based texture awareness refill Figure 7 shows the main aim of this proposed refill strategy. This proposal utilises an image completion tool to borrow the textures coming from the neighbourhood of target areas, then refill the picked textures in the target areas. The borrowing and refilling are operated within lower-exposed image. By doing so, the appearance of occluded areas is revealed to the surface.
The critical issue is to guarantee the adequacy of refilled textures. Therefore, this paper introduces texture and spatial restrictions into the proposal. Figure 8 shows the reason why restrictions play a vital role in this refill strategy. Since there are two restrictions: texture and spatial restrictions in this proposal, this paper states that only areas covered by both types of restriction are the textures' borrowable source.
Texture restriction hints the locations of possibly reasonable textures. As shown in Figure 9, it restricts the areas which turn saturated in middle-exposed image. Basing on the definition of saturated and occluded areas, for a three-image set containing these areas, the areas occluded in lower-exposed image always turn saturated in other images (Figure 9 as example). Therefore, for purposes of borrowing the adequate textures and refill in saturated and occluded areas, it is logical to restrict the image Generation method of texture restriction completion tool to borrow textures belong to areas saturated in middle-exposed image but not occluded in lower-exposed image. The reason for doing so is because these areas have one thing in common: turning saturated in middle-exposed image. Considering that input images are background-aligned, this proposal applies texture restriction in the areas that turn saturated in middle-exposed image. The saturated areas are marked by Equation (4). The restricted areas form texture restriction.
Spatial restriction guarantees the visual consistency by ensuring that only the nearby texture could be considered a source by image completion tool. This proposal is designed to draw four boundaries around each target area to circle the spatial restriction for borrowing textures. Figure 10 demonstrates how four spatial restriction boundaries are drawn outside the target area. One boundary is the perpendicular line of motion direction, and three other boundaries are drawn at a designated distance from the mass centre of the area.  The perpendicular line of motion direction blocks the textures from the destination of the moving object being refilled into target areas. The following steps generate this boundary: figuring out the overall motion direction first, then calculating the gradient of the line which is perpendicular to overall motion direction by Equation (8).
where G stands for the gradient of the line which is perpendicular to overall motion direction. By making this line across the mass centre of the target area, the reference linear Equation (9) is revealed: by inputting coordinate of the mass centre (X, Y), b is revealed. The perpendicular boundary is drawn by following Equation (10): where shift means the adjustment factor for distance from the mass centre. As for three other boundaries, it is drawn at the designated distance D from the mass centre.

3.3.3
Surrounding area analysis based exposure awareness refill As a supplement of texture awareness refill, this refill method focus on providing only exposure and colour information for the saturated and occluded areas. It refills the rough patches with no texture in the target areas. This feature is the crucial difference with texture awareness refill. As shown in Figure 11, exposure awareness refill helps the alignment mechanism   Figure 12 shows the procedure of exposure awareness based refill. The refilled rough patches are also borrowed from somewhere else of the lower-exposed image. This refill strategy performs a surrounding areas analysis on the target areas to find out the patch which is efficient enough to be borrowed and refilled in the target areas. The surrounding areas analysis aims to find the highest exposed patches. After the analysis, this proposal smooths and tiles the highest exposed patch. Tiling means repeating the patch itself both vertically and horizontally to form an image fragment. Tiled image fragment is supposed to be large enough to replace target areas' pixels.

Detail of experiment environment
The paper designates 7 image sets coming from backgroundaligned Kalantari dataset [22] to evaluate how much difference the proposed work can make on reconstructed HDRI. All 7 sets contain saturated and occluded areas. Table 3 lists the index number of selected sets. Figure 13 shows the typical and challenging parts of these areas. Lower-and middle-exposed image are presented.
Since the proposed work focuses on the refill, its output is still a set of LDRIs. To evaluate proposed work can bring HDRI reconstruction improvement or not, it is workable that reconstructing HDRI by input the refilled image set, then evaluate the reconstruction result. This paper sets treatment and control groups to reflect the difference which is brought by the proposed work. As shown in Figure 14, input sets are the only different between control and treatment groups. Ghost removal  works in control groups input 7 test sets directly to reconstruct HDRIs, ghost removal works in treatment groups input the test sets which are refilled by the proposed work. This paper designates Sen [18], Hu [19], Deng [21] and Wu [22] to reconstruct the HDRI and verify how much improvement can proposed work bring. Sen [18] reconstructs HDRI with a milestone quality. Hu [19] is very closely related to our work. Deng [21] and Wu [22] are both deep learning based but in different network architecture, it is necessary to figure out that if proposed work can provide deep learning based works with improvement or not. Just as Figure 14 clarifies, this paper applies subjective and objective evaluations to evaluate reconstructed HDRIs. The subjective evaluation means observing HDRI reconstructed by same deghost work but in different groups. For objective evaluation, this paper designates HDR-VDP-2 [31] to detect the remaining artefacts and judge the visual quality of reconstructed HDRIs of different groups. Both reconstructed HDRI and ground truth are required to input into HDR-VDP-2 as Figure 15 drawn. This evaluation tool outputs a score representing the quality of reconstructed HDRI. A higher score means higher quality and fewer ghost artefacts. This paper selects Flownet 2.0 [24] implemented by [25] as the designated optical flow tool of the proposal of locating to providing the optical flow. Flownet 2.0 provides a more precise motion estimation comparing to the conventional optical flow implementation, such as [26] which is adopted by [20]. For the texture awareness refill, this paper chooses patchmatch [27] implemented by [28] as the designated image completion tool to do the refill. The reason of not choosing Generative adversarial network based image completion works like [29], [30] is that the outputs of these works are coming from their training datasets rather than neighbour areas. This design is not compatible with our proposals.
All the proposed methods are implemented in Python language. The implementation uses Python 3.7 as runtime with the following framework: OpenCV 4.1, skimage 0.16.2 and SciPy 1.4.1. For there are many integrated works and tools inside implementation and experiments, this paper builds the same environment as their reference source specified.
The setting for parameters in experiment groups is listed as Table 4. Table 5 lists the parameters used in objective evaluation.

4.2
Results of locating and refill Figure 16 shows the result of locating. The areas with white mask in the result are the located saturated and occluded areas. The masks of located areas correspond to the lower-exposed image. Figure 17 shows the results of texture awareness refill. The refill is taken place in lower-exposed image. Generally, this strategy refills the target areas with the texture coming from nearby background. Figure 18 shows the result of exposure awareness refill and the difference between two refills strategies. The exposure awareness refill also operates in lower-exposed image. Generally, this strategy refills patch coming from the brightest surrounding patch into target areas.  [18] and Deng [21] in control groups, ghost artefacts are persisted in the saturated and occluded areas. For their treatment groups, the reconstruction of saturated and occluded areas is free from ghost artefacts and has better visual consistency than control groups. Hu [19] leaves heavy ghost artefacts in saturated and occluded areas in both experiment groups, but the ghost artefacts of treatment group results is different due to input set difference. Wu [22] has already reached a relatively high-quality level, but the proposed work still provides an improvement in visual consistency. As for objective evaluation, treatment groups Sen [18], Hu [19] and Deng [21] also have better performance. Table 6 shows that proposed work brings improvement to most of the existing  ghost removal works. The improvement percentage is minimal since the size ratio of saturated and occluded areas to whole image is small. Treatment group Wu has an improved result in subjective evaluation but decreased score in objective evaluation. The reason of decreasing is that, as an end-to-end neural network based research, its reconstruction is more close to ground truth in numerical level rather than visual level. One detail which may make readers wonder is that, although proposed refill strategies always cover the foreground objects such as hands or arms, but reconstructed HDRI still holds the almost intact luminance information of those objects. This integrity is preserved thanks to the matching mechanism of alignment. The match is not exactly finding the same object but the same patch, which means as long as similar patches existing in the image, the covered object can be uncovered in alignment result, then merged into HDRI.

CONCLUSION AND FUTURE WORKS
Ghost removal is an eternal topic in the field of HDRI reconstruction. This paper proposes two refill strategies for improving the reconstruction of saturated and occluded areas. To locate the saturated and occluded areas, motion and saturation filters are applied to filter out the target areas. To improve the reconstruction of the saturated and occluded areas, texture and exposure awareness refill which operate in lower-exposed image are proposed. The texture awareness refill borrows the background textures, then refill these textures in saturated and occluded areas to reveal the appearance of the occluded area. The exposure awareness refill provides rough colour and exposure information. Seven sets coming from background-aligned Kalantari dataset [22] are adopted to evaluate the proposed work. For the aspect of existing ghost removal methods, the proposed work is a pre-process procedure. Experimental results show that the proposed work removes the ghost artefacts caused by saturated and occluded areas in subjective evaluation. For the objective evaluation, the proposed work improves HDR-VDP-2 evaluation result for multiple conventional work Since this paper encounters a new topic of HDRI research, there are two directions can be improved in the future. First, improving the refill at the pixel level. Although the proposed texture awareness refill is capable of refilling the texture to make reconstruction of saturated and occluded areas more vivid, there is room for improving the consistency of refilled pixel value. Second is using deep learning tools to execute two proposed FIGURE 19 Subjective evaluation of the testing data from background aligned kalantari dataset (provided by [22]). the top part shows the input LDRI, HDRI reconstructed using the LDRI processed by proposed work, and the LDRI patch. We compare the saturated and occluded areas of HDRI reconstructed by different ghost removal works. control group means inputting test set directly into ghost removal works. Treatment group means processing test sets by proposed work, then input them into ghost removal works. The proposed work can remove ghost artefacts related to saturated and occluded areas refill strategies. This direction requires creating more LDRI sets containing saturated and occluded areas for training and test.