Automatic Waterline Extraction and Topographic Mapping of Tidal Flats From SAR Images Based on Deep Learning

This study presented an intuitive approach to derive large‐scale tidal flat's Digital Elevation Model (DEM). We first developed an automated method for accurately extracting the waterline from Synthetic Aperture Radar images acquired in Subei Sandbanks along the Yellow Sea coast of China between 2015 and 2020 based on deep convolutional neural networks. The statistical results show this method has appreciable accuracy for efficient waterline extraction even under complex imaging conditions with a mean recall and precision of 0.90 and 0.80, respectively. Then the pixel‐level extracted waterlines are calibrated with a global tide model to construct the large‐scale tidal flat's DEM in the study region. The comparison against in situ topographic data shows an error of 29 cm, demonstrating the usefulness of monitoring the morpho‐sedimentary evolution in intertidal areas. Furthermore, the Subei Sandbanks remained stable from 2015 to 2020, while the coastal region changed drastically due to human activities.


Introduction
Coastal zones are ecologically essential and present dramatic dynamic changes.Monitoring these areas is critical in sustainable development and environmental protection.The waterline, also called coastline in the coastal zones, is defined as the instantaneous boundary between land and the water body (while the shoreline is defined as the intersection of mean high water and the shore).Its geographic knowledge is the fundamental work for coastal zone monitoring, which has paramount importance in a broad range of applications, including analysis of land/water resources, navigation, beach-face slope (Vos et al., 2020), coastal erosion monitoring (Ding & Li, 2011, 2014;Ding et al., 2015), and the sea-level rise caused by global warming (McNamara et al., 2011;Quataert et al., 2015;Straub et al., 2015).Furthermore, it is also the fundamental element for topographic mapping of intertidal zones based on the waterline method (Mason & Davenport, 1996).
Remote sensing data play a vital role in coastal waterline monitoring.Microwave and optical sensor images are complementary to each other for this task.Infrared radiation is significantly reflected over the inland area while strongly absorbed by sea waters, making the waterline clear and relatively convenient to extract.However, the optical-based detection is severely contaminated by cloud cover, solar illumination, and other adverse meteorological conditions.Waterline extraction from synthetic aperture radar (SAR) imagery has become popular due to its all-weather, day and night observing capability.Several automatic or semi-automatic waterline extraction (WE) methods for SAR images have been proposed mainly based on two traditional approaches: edge detection (Lee & Jurkevich, 1990;Liu et al., 2017;Mason & Davenport, 1996;Modava & Akbarizadeh, 2017;Wei et al., 2021) and image segmentation (Dellepiane et al., 2004;Heygster et al., 2010;Liu & Jezek, 2004;Nunziata et al., 2016;Salameh et al., 2020).Edge detection methods suffer from the edge pixels produced by edge detectors and are hence quite discontinuous and seldom characterize a waterline completely (Lee & Jurkevich, 1990;Liu & Jezek, 2004;Mason & Davenport, 1996).In contrast, image segmentation methods require more processing steps to delineate the boundary pixels and face difficulties determining reliable threshold values (Liu & Jezek, 2004).Whether based on edge detection or image segmentation, these methods more or less require postprocessing for an accurate WE result from SAR images (Heygster et al., 2010;Salameh et al., 2020).The speckle noise generated by the coherent signal-scattering complicates the WE problem for SAR images.Besides, the radar backscatter signal from wind-roughed and wave-modulated waters can be frequently equal or exceed the return from a nearby land area, resulting in an inadequate contrast for unambiguous land-sea separation.Affected by the moisture of the sandy sediments, the backscattering is more complicated in tidal flat areas (Gade et al., 2014(Gade et al., , 2018;;Wang et al., 2017).
In recent years, machine learning methods have proved their advantages for image pattern recognition (Krizhevsky et al., 2017;Lee et al., 2019).Deep convolutional neural networks have widely been used to extract robust high-level information from satellite images (Liu et al., 2019).Xie and Tu (2015) proposed a holistically nested edge detection network (HED) to perform image-to-image detection for the first time.The results were far better than that of the conventional Canny edge detector.Baumhoer et al. (2019) developed a U-Net model to extract the Antarctic coastline for tracking glacier and ice shelf front movement from European Space Agency's Sentinel-1 data.By combining the HED and U-Net, Heidler et al. (2021) recently successfully extracted the Antarctic coast's waterline and completed the sea-land classification simultaneously for Sentinel-1 SAR imagery.These studies show that the DCNN-based methods have great potential and application value in the automatic extraction of waterlines in high-spatial-resolution SAR images.However, unlike the typical land or ice surface, the radar backscatter signal of the tidal flat area changes dramatically under different sea conditions, bringing new challenges to the automatic WE task.
The purpose of this study is: (a) to develop a deep-learning framework for accurate WE of tidal flats from Sentinel-1 SAR images; (b) to extract the pixel-level waterlines at different tidal phases to construct a multi-year digital elevation model (DEM) series; (c) to understand the morpho-sedimentary evolution in the study area between 2015 ∼ 2020.This paper is organized as follows: we first describe our study area and data in Section 2. Next, the DCNN-based method for WE and its performance are described in Section 3.Then, in Section 4, we develop a processing chain for automatic constructing the tidal flats' DEM and analyze the interannual topographical evolution of Subei Sandbanks.Finally, we presented discussion and conclusions in the last two sections.

Study Area and Data Sets
The Subei Sandbanks is the intertidal zone of the Radial Sand Ridges along the Jiangsu coast in the Southern Yellow Sea (Figure 1a).It is also the largest tidal flats in China, covering an area of approximately 2300 km 2 owing to the active tidal processes and abundant sediment supply (Wang et al., 2019).The vast amount of sediment brought by terrestrial runoff also leads to poor visibility of seawaters, resulting in the limited ability of optical remote sensing technique to detect underwater topography in this area (Figure 1b).The study area is controlled by a semi-diurnal tide, dominated by a progressive Poincaré wave from the East China Sea and an amphidromic system in the Yellow Sea (Ni et al., 2014;Xu et al., 2016).The complex hydrodynamic condition makes the area's topography changeable, including the shallow tidal flat regions exposed at low tide.As shown in Figure 1c, Subei Sandbanks comprises seven tidal flats of different sizes and shapes.These tidal flats emerge and submerge as the tide fluctuates, with an apparent horizontal position change of the waterlines.
With the support of Google Earth Engine (Gorelick et al., 2017), we collect a total of 193 pre-processed Ground Range Detected Sentinel-1 SAR imagery from 2015 to 2020 for the WE analysis and topographic mapping of Subei Sandbanks.These images were acquired in interferometric wide-swath mode, with dual-polarization (VV and VH) and 10 m spatial resolution.We use the VV-polarized images and downsample them to a resolution of 50 m to save training time.In addition, 52 of them in 2019 are used for training and testing our DCNN model and the remainder is used for constructing the large-scale tidal flats' DEM of Subei Sandbanks.

WENet
The U-Net (Ronneberger et al., 2015) is a modified fully convolutional network (Long et al., 2015) initially developed for biomedical image segmentation.Its architecture is designed to work with fewer training images but can yield more precise segmentation.In 2018, U-Net++ was proposed as a modified U-Net by developing a more robust architecture with nested and dense skip connections (Zhou et al., 2020).Recently, Huang et al. (2020) continued to improve the U-Net++ and presented the U-Net 3+ by taking full-scale skip connections, which incorporate low-level details with high-level semantics from feature maps in different scales.As a result, U-Net 3+ uses fewer parameters but has higher accuracy than other state-of-the-art semantic segmentation networks.
However, due to our task's extreme foreground-background class imbalance: the background samples' numbers are much higher than those of waterline samples (the waterlines account for less than one percent of the entire image), directly calling U-Net 3+ will be very time-consuming or even unable to extract the waterline effectively.Therefore, using the U-Net 3+ as a baseline, we proposed a model specifically for automatic WE from SAR images of a large-scale tidal flat: WENet (Figure 2a).Compared with the original model, we made two improvements for this arduous task: (a) a waterline-related feature map was added as another input channel (the number of channels is the depth of matrices involved in the convolutions); (b) the loss function was adjusted to solve extreme sample imbalance problem.
We first used the traditional waterline detection method, bilateral filtering with Canny edge detector (H.Liu & Jezek, 2004) and dilation in this study, to return to the approximate location of the waterlines in the SAR images.An example result is shown as the Channel two image in Figure 2a.The additional channel will help the model focus more on areas with noticeable edge features during training, thereby improving the training efficiency and extraction accuracy.
The waterline signature extraction problem is essentially a binary, pixel-wise classification problem.Therefore, as shown in Figure 2a, the WENet's last layer is 1 × 1 convolution with Sigmoid activation.Traditionally, the loss function is cross-entropy.However, in this case, the samples are incredibly unbalanced.Therefore, motivated by Lin et al. (2018), we use the α-balanced cross-entropy in this study.To put more weight on the waterline samples in the loss function, we have to set α to a higher value.In this study, we set α to 0.99.

Labels Preparation and Training
Ground truth labels are necessary when we train a deep-learning-based model.Since there is no corresponding waterline product in this study area, we manually extract waterlines from the 52 SAR images acquired in 2019 as ground truth values.In practice, we first created a transparent layer over the SAR image, then represented the waterlines' position with white lines by a stylus accurately, and finally filled the layer with a black background.The depicted result is taken as the output image in Figure 2a.In this way, we get 52 pairs of input and output SAR images, among which we randomly select 42/10 pairs for model training/testing (see Text/Figure S1 in Supporting Information S1 for details).
We crop the images and their corresponding ground truth into 256 × 256 sub-images and obtained 3024 pairs of sub-images for training the WENet model.In addition, as a common practice to compensate for the limited number of training samples, we performed data augmentations (Perez & Wang, 2017), including random contrast, brightness change, image rotation, and horizontal/vertical flipping.This process helps the networks learn more tidal flat waterline features with protean brightness and shapes in the SAR imagery.
The training and testing of the network are implemented by Keras/Tensorflow framework.We use Keras default Adam optimizer, the α-balanced cross-entropy as the loss function (α is set to 0.99), and the classification accuracy as the performance metric.Furthermore, the batch size is set to 32, and the number of epochs is 4,000.
The training samples are divided into 80% for training and 20% for validation in the training process.Finally, the classification accuracy of the 20% validation images is 96.06%.

Model Performance
As mentioned before, 10 SAR images are randomly selected to test the classification accuracy of the trained WENet model.It is estimated by calculating the precision P (also called positive predictive value) and recall R (also known as sensitivity) of the automatically extracted waterlines.Precision and recall are defined as follows (Goutte & Gaussier, 2005), respectively: (1) where S TP stands for the sum of true positives (yellow lines in Figures 2b and 2e),  and 2d were acquired under three typical tidal conditions: high, medium, and low, which can also be judged from the exposed area of the tidal flats.Under the medium tidal level, the waterlines in Figure 2c exhibit more detailed features than the other images, especially at Dongsha and Zhugensha tidal flats.The tidal fluctuation also affects the moisture of the sandy sediments, which leads to the difference in brightness of tidal flats both in time and space.The longer the tidal flat is exposed in the air, the lower the moisture, resulting in stronger single-bounce radar backscattering.Conversely, higher moisture can cause a much weaker single-and double-bounce backscattering (Gade et al., 2018).
In addition to the tidal flats themselves, the surrounding seawater also shows significant brightness differences in SAR images under diverse sea conditions (Li et al., 2010;Zheng et al., 2012).For example, the four images in Figure 2 are more or less contaminated by the features of the shallow water topography of Subei Bank (Zhang et al., 2017).The eastern part of Figure 2b  In the few cases, as shown in Figure 2e, where the tidal flats are characterized by darker patterns caused by high humidity, the automatic WE model has a relatively low accuracy compared with other cases.The precision and recall are 0.87 and 0.80, respectively.The relatively low recall value indicates that the model has produced more leak detections (waterlines misclassified as non-waterlines).One possible reason is that there are few images with darker tidal flats in the training data set.

Waterline Method
Conventional topographic mapping of tidal flats techniques such as ground-based and airborne remote sensing surveying (e.g., LiDAR, stereo-photogrammetry) provide accurate measurements but are constrained by logistical difficulties and high costs (Mason et al., 2000).The direct topography retrieval methods based on optical remote sensing are unsuitable in intertidal zones with high water turbidity (Tseng et al., 2017).Mason et al. (1995) first introduced the waterline method, becoming the most common technique for intertidal topography generation nowadays.The waterline method combines remote sensing and hydrodynamic modeling techniques (Salameh et al., 2020).The general idea of this method is to make use of the ever-shifting borderline between tidal flats and adjacent water areas.The tidal level of seawater and elevation of the tidal flat is usually based on the same datum (mean sea level in this study).Therefore, it makes the waterline can be regarded as a quasi-contour line of the topography (not precisely a contour line because the tidal level varies horizontally, especially for large-scale tidal flats).Combined with the corresponding water level information, a series of waterlines at different tidal levels can be assembled and interpolated to construct a gridded DEM.The waterline method has been proved to be one of the best methods that provide an excellent trade-off between accuracy and cost-effectiveness for the DEM generation of tidal flats (Kang et al., 2017;Mason et al., 2000;Salameh et al., 2020).
The flowchart of this method is shown in Figure 3a.With WENet, the DEM automatic generation process can be roughly divided into four steps: 1. Extracting waterlines automatically from time series of Sentinel-1 SAR images acquired at different water levels with the trained WENet 2. Discreting the waterlines into points and evaluating the geographical position of each point 3. Evaluating the tidal level at each point of the waterline at satellite overpassing time by using the calibrated ocean tide model 4. Generating the DEM map by interpolating the resulted grid of quasi-contour lines In this study area, due to tidal phase differences arising from tidal wave propagation and tidal deformation caused by the complexity of underwater topography, the uneven spatial distribution of the instantaneous tidal level field cannot be neglected.After weighing the model accuracy and running time, we use a global ocean model, the TPXO tide model (Egbert & Erofeeva, 2002), to obtain the tidal level value in this DEM generation chain.The previous study (Zhang et al., 2017) shows that the TPXO model performs well in the tidal phase of the study area but presents a systematic underestimation of tidal amplitude.In situ water level data from two tidal gauge stations were used to calibrate this tide model (see Zhang et al., 2017 for details).Then the Tidal Model Driver software and the calibrated TPXO tide model outputs are used to estimate the tidal level at each point of the extracted waterlines.
We first use the manually extracted waterlines of 2019 to verify the accuracy of the waterline method in measuring tidal flats elevation in our study area.The waterlines derived from SAR images acquired at wind speed higher than 10 m/s will have a large offset from their original location caused only by tidal fluctuation (Tong et al., 2020).Therefore, five scenes with wind speed greater than 10 m/s are eliminated after checking the satellite wind products (National Oceanic and Atmospheric Administration/National Centers for Environmental Information's sea surface wind data, derived from operational weather model).Then each point of the waterlines from the remaining 47 images is assigned a tidal level value using the calibrated ocean tide model (Figure 3b).Finally, these points/lines are interpolated to obtain continuous gridded DEM of the large-scale tidal flats (Figure 3c).In situ topographic data along the transect in Figure 3b obtained from the field survey in May 2019 are used to evaluate the accuracy of the derived DEM, and the mean absolute error is 0.29 m (Figure 3d).The relatively high accuracy indicates the waterline method based on SAR imagery and regionally calibrated global tide model can be used for preliminary topographic mapping of large-scale tidal flats such as the Subei Sandbanks.

Morpho-Sedimentary Evolution of Tidal Flats
Annual DEMs of Subei Sandbanks were efficiently generated in other years from 2015 to 2020 by our developed approach (Figures 4a and 4f, see Text/Figure S4 in Supporting Information S1 for WENet performance in years other than 2019).The six waterline-derived DEMs were interpolated over the intertidal area specified by the same mask, which was created manually according to the Sentinel-1 SAR image acquired at the lowest tide (−3.14 m) during 2019.
The significant features of the intertidal zone are well detected by the derived DEMs.All of them show clear sand ridges and tidal channel features.For the northern and offshore tidal flats (zones ABC, see Figure 1b for location), each sandbank exhibits a similar spatial distribution of elevation decreasing from vertices to tidal channels between sandbanks.However, the tidal flats in zone D were often separated by small tidal channels and have shown apparent changes over the past 6 years.This phenomenon is more evident in the difference maps generated by subtracting the DEMs of consecutive years, with the positive values indicating deposition area and the negatives indicating erosion (Figures 4g and 4k).
The tidal flats along the coast and in zones ABC showed relatively stable behavior during the 6 years.However, in the southwestern part of Subei Sandbanks, zone D showed significant erosion and deposition for the whole period, which corresponds well with previous study from Wang et al. (2019), who generated the DEMs of Subei Sandbanks in the past few decades by using the waterline method with optical data.There are two reasons for the drastic change of this part of Subei Sandbanks: (a) zone D (Tiaozini area) is located at the intersection of the  progressive Poincaré wave from the East China Sea and the amphidromic system in the Yellow Sea.Therefore, the strong tidal currents there make the evolution of this area is more complicated than the surroundings (Chen et al., 2007;Ni et al., 2014;Xu et al., 2016); (b) The rapid reclamation around the Jianggang Port (see Figure 1b) since the 21st century has accelerated the outward expansion of the coastline.At the same time, the boundary conditions of tidal currents are also changed, leading to faster swings of tidal channels and more drastic topographical changes (Ding et al., 2011).
For monitoring the interannual morpho-sedimentary changes in the intertidal areas, we further calculated the deposition and erosion volumes from the difference maps.The results show that apparent topographic evolution of this intertidal area was detected for all years.Furthermore, the deposition is more remarkable than erosion almost every year.The mean deposition and erosion volumes of the whole Subei Sandbanks are 0.46 and 0.37 km 3 , respectively.The erosion-deposition balance shows a net deposition of 0.46 km 3 from 2015 to 2020 due to the changed tidal conditions and human activities.

Discussion
In addition to the location of the waterlines, another pivotal factor that affects the accuracy of the generated DEM is the number of satellite images acquired within 1 year.The acquisition time of these images should preferably cover the entire tidal cycle to ensure that the extracted waterlines can cover the whole intertidal zone as much as possible before interpolation.Liu et al. (2013) found that the waterline coverage (the percentage of waterlines in the entire intertidal zone) depends linearly on the accuracy of the DEMs of tidal flats derived using the waterline method.Therefore, the use of an insufficient number of images may lead to inaccuracies.This study tried to overcome this limitation by using SAR, which is not affected by clouds, to ensure the utilization of each image.However, the two Sentinel-1 satellites did not provide enough SAR images in the previous 4 years due to their early stages of operation.There are more than 50 images in 2020 (54) and 2019 (52), while less than 30 in the other four years (29 images in 2018, 27 in 2017, 19 in 2016, and only 12 in 2015).After deleting the ones under high wind speeds or extremely contaminated, fewer images can be used for constructing the DEMs (Figure 4).Lower waterline coverage in the previous years may increase error in this study (but can be solved by using available optical imaging as a supplement during this period).
The tidal level information associated with each point in the waterlines is another critical factor affecting the generated DEM's accuracy.In this study, the tidal level values are from the TPXO ocean tide model, which can quickly export the result at any time in any sea area worldwide.TPXO performs relatively well compared to other popular global tide models in the coastal waters of China with the assimilation of lots of tide gauge data (Egbert & Erofeeva, 2002;Stammer et al., 2014).The accuracy of the output of tide model is closely related to the underwater topography.However, due to the variability of the coastal topography and the lack of in situ bathymetric data, the tide model results have a relatively high error in the coastal areas.The global tide model is incapable of providing very accurate sea level heights over the intertidal regions, especially in our study area with very complex and changeable underwater topography.Therefore, it will inevitably affect the tidal flat DEM accuracy derived by the waterline method.Theoretically, more accurate tide level values at satellite imaging time would be obtained with the help of ocean numerical models (such as the Regional Ocean Modeling System).However, it usually takes much more time to rely on such ocean models due to the complexity of the model, unknown boundary and bathymetric conditions, and its demand for computing resources.It means we need to balance efficiency and accuracy when generating the DEM of tidal flats.Promisingly, the next-generation wide swath altimetry mission Surface Water and Ocean Topography, which measures high-resolution sea surface height, will render the waterline method independent from ground-based measurements (tide gauges) or complicated hydrodynamic models (Fu et al., 2009;Salameh et al., 2020).It implies the developed method may maintain high efficiency and accuracy simultaneously in the near future.

Conclusions
Remote sensing data archives reach several tens of petabytes, and massive satellite data are acquired worldwide daily (Li et al., 2020).Therefore, an automatic extraction method should be given priority with the unprecedented amount of data containing waterline information available.Deep learning has demonstrated its significant superiority over traditional algorithms for image-information extraction in various ocean remote sensing applications within the last few years.In this study, we propose a DCNN-based model to extract waterlines automatically from high-spatial-resolution SAR images.Our approach simultaneously shows high accuracy and efficiency for waterline detection in complicated large-scale tidal flats of Subei Bank.The average precision and recall are 0.80 and 0.90, respectively.It indicates that the deep learning method can be promising in mining helpful information such as waterlines submerged in massive ocean remote-sensing data sets in the big-data era.
By combining this model and the waterline-based DEM generation method, we also present the first attempt to construct the DEM of the tidal flats from time series of SAR images intuitively.There is a good agreement between the derived elevation and in situ topographic data.Multi-year DEM series analysis implies that the proposed workflow for automatic topographic mapping of tidal flats has excellent potential for the rapid study of intertidal topography evolution, even for large-scale tidal flats such as Subei Sandbanks.

Figure 1 .
Figure 1.(a) Overview of the study area.The elevation data are from the ETOPO1 (Amante & Eakins, 2009).(b) Cloud-free Sentinel-2A Multi-Spectral Instrument true-color image of the study area acquired at Greenwich Mean Time (GMT) 02:37, 17 February 2021.(c) Sentinel-1B synthetic aperture radar image of the study area at low tide, acquired at GMT 09:54, 26 November 2019.Four cluster analysis zones: zone A consists of Liangyuesha; B of Niluoheng; C of Dongsha, Gaoni, and Zhugensha; D of Tiaozini and Jiangjiasha.
FP for the sum of false positives (red lines) and S FP for the sum of false negatives (blue lines).Precision and recall are suitable metrics to evaluate classifier output quality when the classes are highly imbalanced (see Text/Figure S2 in Supporting Information S1 for details).The mean precision and recall of the model are 0.80 and 0.90, respectively.The statistical result shows that the automatic extraction results from the DCNN-based model are highly consistent with the manually annotated ground truth maps, even for large-scale tidal flats like Subei Sandbanks.Four examples of the WE results from the 10 testing SAR images under different sea conditions are shown in Figures 2b and 2e.As can be seen from both Figures 1b and 2, the fluctuation of ocean tides causes drastic changes in the shape and distribution of the waterlines.The tidal levels at Dongsha tide gauge indicate that Figures 2b and the northern coastal area of Figure 2c show unusual dark patterns, which may be caused by the oil spill, low sea surface temperature, or low wind speed.All of the above factors have brought challenges to the automatic extraction of waterlines in the study area.However, as shown by the yellow lines in Figures 2b and 2e, most of the waterlines obtained from the DCNN-based model are in good agreement with the manually annotated ground truth values (see Text/Figure S3 in Supporting Information S1 for details about the comparison with traditional extraction methods).

Figure 3 .
Figure 3. (a) Flowchart of the method for automatic topographic mapping of the tidal flats.(b) Assembled tidal levels at each point of the waterlines extracted manually from 47 Sentinel-1 synthetic aperture radar (SAR) images in 2019; (c) The derived Digital Elevation Model (DEM) of Subei Sandbanks in 2019 overlaid on Sentinel-1A SAR image acquired at GMT 09:55, 20 November 2019; (d) The derived DEM (orange) and in situ measurement along the transect (solid black line in c).
The work was supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (CAS) (XDB42040401), the Key R&D project of Shandong Province (2019JZZY010102), the National Natural Science Foundation of China-Shandong Science Foundation (U2006211), the Key deployment project of Center for Ocean Mega-Science, CAS (COMS2019R02), the CAS Program (Y9KY04101 L), and the National Natural Science Foundation of China under Grant 41976163.