Real time refractivity from clutter using a best fit approach improved with physical information



[1] Refractivity from clutter (RFC) retrieves the radio frequency refractive conditions along a propagation path by inverting the measured radar sea clutter return. In this paper, a real-time RFC technique is proposed called “Improved Best Fit” (IBF). It is based on finding the environment with best fit to one of many precomputed, modeled radar returns for different environments in a database. The method is improved by considering the mean slope of the propagation factor, and physical considerations are added: smooth variations of refractive conditions with azimuth and smooth variations of duct height with range. The approach is tested on data from 1998 Wallops Island, Virginia, measurement campaign with good results on most of the data, and questionable results are detected with a confidence criterion. A comparison between the refractivity structures measured during the measurement campaign and the ones retrieved by inversion shows a good match. Radar coverage simulations obtained from these inverted refractivity structures demonstrate the potential utility of IBF.

1. Introduction

[2] Efficient algorithms exist to model the radio waves propagation taking into account the refractive conditions. A commonly used propagation model is based on a Split-Step Fourier (SSF) Parabolic Wave Equation (PWE) [Kuttler and Dockery, 1991; Levy, 2000]. These methods are useful to predict the coverage of radars in a maritime environment, where the medium is fluctuating. A problem in predicting radar coverage is the 3-D refractivity structure to be used in a propagation model. When the refractivity structure is unknown, a standard atmosphere is often used in the modeling, which may entail significant errors. If this structure is known, the radar coverage can be well estimated by propagation models for all azimuths.

[3] Abnormal propagation events occur when the atmospheric conditions are different from the standard atmosphere. These events are caused by the presence of different gradients of temperature, humidity, and pressure [Paulus, 1990]. Four typical kinds of abnormal propagation events can be distinguished: the subrefractive layer, the evaporation duct, the surface-based duct, and the elevated duct. A classical method to measure the refractive index at lower altitudes is the use of an airplane or a helicopter following sawtooth flights, on which a refractometer can directly measure refractivity [Briggs, 2005], or it can be deduced from radiosondes. Combined with buoy measurements, the refractivity for all heights can be deduced. However, these methods require expensive equipment and are not practical for real-time operational use.

[4] RFC exploits the radar itself to predict its coverage [Krolik and Tabrikian, 1997; Rogers et al., 2000; Gerstoft et al., 2003a; Yardim et al., 2008]. The idea is to extract the information contained in the sea clutter. This information allows retrieving the refractivity, which allows modeling the radar coverage. This method is attractive because it only requires an additional signal-processing module linked to the radar. As no equation gives the refractivity from the sea clutter, RFC is an inverse problem.

[5] To perform RFC, the refractive conditions have to be parameterized. The principle of RFC is to retrieve the parameters of the atmospheric ducts. The evaporation duct is classically modeled using Jeske's description [Jeske, 1973]. The evaporation duct height can be extracted easily using real-time methods quite accurately for simple environments [Rogers et al., 2000; Douvenot et al., 2008b]. For surface-based or elevated ducts, the number of parameters describing the refractivity conditions is larger and the inverse problem is more difficult. This paper focuses on the inversion of clutter to noise ratio (CNR) maps measured in the presence of surface-based ducts during the 1998 Wallops Island, Virginia, measurement campaign [Rogers et al., 2000].

[6] In the last decade, several RFC methods have been proposed [Gerstoft et al., 2000, 2003a, 2003b; Barrios, 2004; Yardim et al., 2006, 2007, 2008]. These are based on accurate Bayesian methods. Actually, the inverse method is nonlinear and ill posed. These methods are computer intensive and do not permit retrieving the refractivity parameters for all azimuths in real time. Yardim et al. [2008] used particle filters to extract the refractivity parameters across several azimuths. Here, a less precise but real-time inversion method is preferred as refractivity conditions can significantly change in short time [Stapleton et al., 2003; Douvenot et al., 2008b]. In this context, methods based on learning machines have also been introduced [Douvenot et al., 2008a, 2008b, 2008c]. These methods, based on the Least squares Support Vector Machine (LS-SVM) [Suykens et al., 2002] and on the Multitask Least squares Support Vector Machine (MuLS-SVM) [Argyriou et al., 2006], respectively, have both been validated on simulated data [Douvenot et al., 2008a, 2008c]. MuLS-SVM method has been applied on measured data in the presence of evaporation ducts [Douvenot et al., 2008b].

[7] This paper introduces an RFC method based on finding the environment with best fit to the radar return to one of many precomputed, modeled clutter returns for different environments in a database. Without improvements, this method does not work well for the RFC problem because of the many local optima [Douvenot et al., 2008a]. However, many of these local minima can be removed using physical considerations. The improved technique is called improved best fit (IBF). This method gives real-time information for the radar coverage prediction for all azimuths simultaneously. It takes advantage of the spatial continuity of the refractivity.

2. Physical Modeling

2.1. Modeling of the Atmospheric Conditions

[8] To describe the refractive conditions, the modified refractivity M is preferred to the refractive index n:

equation image

where z is the altitude in m. The first term expresses the part per million of the difference of the refractive index with the one of the void (n is close to 1) and the second term takes into account the Earth's curvature.

[9] The data studied here have been measured in the presence of surface-based ducts and very low evaporation ducts, less than 5 m. To simplify the inversion problem, only the surface-based ducts that have a strong impact on propagation are modeled. These ducts are modeled by a trilinear variation in refractivity with respect to the altitude [Douvenot et al., 2008c]. Three parameters zb, c2, and zthick describe the classical trilinear vertical surface-based duct profile. zb denotes the height of the duct base in m, c2 denotes the slope of modified refractivity into the duct in M unit/m, and zthick denotes the duct thickness in m. For surface-based ducts in a coastal environment, the most common variation of the modified refractivity is a rising duct height with range [Stapleton et al., 2003]. A parameter δzb is added to this classical model to express the variation of the height of the duct base from 0 to 60 km, in m, resulting in a four-parameter model (see Figure 1).

Figure 1.

Three-parameter surface-based duct varying with the distance. The parameter δzb gives the distance variation.

2.2. Radar Equation

[10] RFC retrieves the duct parameters from just the received clutter power PR versus range. These variables are linked through the radar equation in monostatic configuration [Cherniakov and Sizov, 2006] with the sea surface as the target:

equation image

where PE is the power emitted by the radar, G is the antenna gain, F0 represents the propagation factor, σ is the monostatic radar cross section (RCS) of the sea surface, and λ is the wavelength.

[11] As F0 takes into account all the refractive effects and is independent of the sea state, the inversion retrieves the refractivity from the propagation factor. The propagation factor is assumed independent of sea state because in the case of very low grazing angles, the rough surface tends to be seen as a flat surface by the radio waves, and only the crests of the sea waves are illuminated. This behavior can be explained studying the forward reflection coefficient of rough surface [Fabbro et al., 2006]. However, note that the surface must be sufficiently rough that enough power is reflected and not too rough to make the signal random. Other dependencies as the spray production due to sea roughness are neglected. The propagation factor must be deduced from the received radar power. The values PE, G, R, and λ are known. Thus, to obtain F0 from the received power, the RCS σ of the illuminated sea surface is required. If the received signal is normalized, only the dynamics of σ is required.

[12] The sea state and the wind conditions are assumed constant along the propagation path and the shadowing effect is neglected for the forward propagation and for the RCS computation (a θg0.4 dependency due to the shadowing effect is assumed in the GIT model [Horst et al., 1978]). When computed just above the sea surface at altitude z, the propagation factor is expressed as ∣F(z, θg)∣ = ∣1 − exp(−2jk0zsin θg)∣F0 due to multipath. Then, at low grazing angle, ∣F(z, θg)∣ ∝ F0θg. This factor models forward propagation and induces a variation with grazing angle θg. So the variation of the normalized power at grazing angle is assumed in θg4. Depending on the author, this variation is taken into account in the NRCS [Barrick, 1998; Tatarskii and Charnotskii, 1998] or in the propagation factor [Rogers et al., 2000]. For the 1998 Wallops Island data, the obtained propagation factor is normalized by its value at 10 km, so the inversion is carried out on the normalized propagation factors Fn2, not on the real propagation factors F02.

3. IBF Inversion Method

[13] This IBF method requires a precomputed database, composed of 30000 pairs “duct parameters/normalized propagation factor” vector. The duct parameters values are chosen using a Latin hypercube sampling [McKay et al., 1979], and then the corresponding elements of the normalized propagation factor vectors are computed using an SSF-PWE propagation method [Barrios, 1992]. The database generation for RFC is optimized by applying a Latin hypercube process [Douvenot et al., 2008c]. This sampling method is a generalization of Euler's magic square. To compute a N-sized Latin hypercube, the basic idea is to divide each dimension into N intervals, usually equal. Then, the sample of the data is forced to be more representative than using random sampling by forcing variability: if a Latin hypercube is projected on one of its dimensions, each of the N single points belongs to one single interval. This point is randomly chosen into that interval. This sampling method is a classical one more efficient than regular or random samplings because it simultaneously stratifies on all input dimensions. For more details, see Loh [1996].

[14] Using this precomputed database, the IBF inversion finds the minimum error over all azimuths:

equation image

where naz is the total number of azimuths on which the inversion is carried out. eij denotes the quadratic difference between the measurement on the jth azimuth and the ith normalized propagation factor of the database, defined by:

equation image

where (Fn k meas2)j represents the kth point of the measured (“meas”) normalized propagation factor (Fn2) to invert on the jth azimuth, and (Fn k DB)i represents the kth point of the ith normalized propagation factor of the pregenerated database (“DB”). P is the dimension of the propagation factor vectors.

[15] The IBF is very fast and can be implemented in real time, but in order to obtain good inversion results it is necessary to constrain the propagation factors to have similar variations (section 3.1), to constrain the variation in base height (section 3.2), and to smooth the parameters over azimuth (section 3.3).

3.1. Constraining the Propagation Factor Variations

[16] The quadratic distance (4) is not a sufficient criterion to select the best fitting propagation factor. Sometimes two propagation factor vectors will be close with respect to the quadratic distance but will present high differences in their dynamics. In order to select a propagation factor vector with dynamic close to the one to invert, a constraint on the Mean of the Absolute Value of the Derivative (MAVD) is added. The MAVD measures the absolute slope of the propagation factor over the range:

equation image

where Rmin and Rmax are the minimum and maximum distances, here 10 km and 60 km, respectively. In practice, the propagation factor is sampled over distance, and the integral is substituted by a sum. By adding the constraint on the MAVD, the system becomes:

equation image

where MAVDmeas is the MAVD of the measured normalized propagation factor, MAVDj is the MAVD of the normalized propagation factor that was found to minimize the quadratic error on the jth azimuth. ΔMAVD is a parameter of the inversion system here arbitrarily chosen as 0.05 dB/km.

3.2. Limiting Variations in Duct Height

[17] The second improvement of the method relies on the continuous variation of the refractive ducts with azimuth. If the variations are too disorderly, the inverted profiles can be considered as nonphysical. As the refractivity measurements used for verification have been carried out on one single azimuth during the 1998 Wallops Island campaign (see section 4), this hypothesis only relies on the physical sense of the problem.

[18] A new constraint is added in the inversion. Using (6) for the inversion, it has been observed that the parameter δzb is highly variable with the azimuth, too much to be physically realistic. The new constraint introduced here forces the variations of the parameter δzb on all the azimuths to be in a fixed interval Δδzb. By applying this constraint, the system becomes:

equation image

where δzbmax and δzbmin denote the maximum and minimum values of δzb on all the azimuths. Δδzb is a parameter of the inversion system arbitrarily chosen as 25 m. The best fitting propagation factors on each azimuth fulfilling (7) are searched for in the database, and the corresponding refractivity parameters are obtained. Note that the inversion is carried out on all azimuths at once.

3.3. Azimuthal Smoothing

[19] The inverted parameters are smoothed over the azimuth to reduce the effect of choosing incorrect parameters. The coefficients αi are introduced which quantify the fit of an inverted result from the corresponding normalized propagation factor. An inverted profile is considered as accurate (αi = 1) if the mean quadratic error equation image between the inverted and the measured normalized propagation factor is inferior to 5 dB2, and inaccurate (αi = 0) if this value is superior to 24 dB2. Between these two extreme values, the coefficient αi varies linearly with the mean quadratic error:

equation image

[20] The limits 5 dB2 and 24 dB2 are arbitrarily chosen. The limits to describe an accurate or an inaccurate propagation factor comparison can vary from one application to another.

[21] For one inversion, M is the vector giving one of the four parameters describing the atmospheric duct (Figure 1) with respect to all the azimuths. Once M is obtained from (7) it is smoothed with respect to the azimuth considering the continuity of the refractivity. The smoothing gives less importance to the less well determined ducts, characterized by a larger quadratic error as quantified by the coefficients αi(8). The smoothed value of the inverted parameter equation imagei at the ith azimuth is calculated as:

equation image

[22] Notice that when the three adjacent refractivity profiles are considered as accurate (αi = αi−1 = αi+1 = 1), the smoothing is equivalent to a low-pass triangular filter.

[23] The IBF technique consists in solving (7) and then applying the smoothing as described in (9). The system constants are empirically chosen. This point is one of the further improvements to be studied for the IBF method.

4. The 1998 Wallops Island, Virginia, Data

[24] The 1998 Wallops Island measurement campaign was conducted by the Naval Surface Warfare Center, Dahlgren Division and took place in the Wallops Island, Virginia. The CNR signals were measured using the Space Range Radar (SPANDAR) [Ingwersen and Lemnios, 2000]. The radar works at 2.84 GHz with beam width 0.4°, height 30.78 m, and maximum range 267 km. The data were measured on 2 April 1998, in the presence of surface-based ducts [Gerstoft et al., 2003a] and very low evaporation ducts. To keep the CNR above zero, only data from 10 to 60 km are used. Figure 2 shows an example of measured CNR data.

Figure 2.

SPANDAR Clutter to Noise Ratio (CNR) map (1653 UTC) from the 1998 Wallops Island campaign.

[25] During the radar measurements, a helicopter equipped with meteorological instrumentation from The Johns Hopkins University Applied Physics Laboratory (JHU/APL) was used to measure air temperature, relative humidity, and atmospheric pressure. It was flying along azimuth 150° in a vertical sawtooth trajectory to characterize the atmospheric medium on the first 60 km. Each atmospheric measurement lasted about 30 min. Five flights were carried out during the measurement of the studied CNR data. A buoy on this azimuth was used to measure the meteorological parameters to characterize the evaporation duct. Then the refractivity was obtained by postprocessing the data. Figure 3 displays an example of the postprocessed refractivity profiles along the 150° azimuth, which includes the appended evaporation duct profile derived from the buoy measurements. As the refractive conditions can change in the 30 min it took to collect the vertical profiles, the actual conditions during the SPANDAR data collection were likely to be different from the measured one, but they give an idea of the ducting.

Figure 3.

Refractivity profiles derived from processed measurements from the 1998 Wallops Island campaign (2110 UTC).

5. Validation of the IBF Method for RFC

[26] During the measurement campaign, CNR signals were measured on several azimuths. The inversions using IBF has been applied simultaneously on all the azimuths not covered by land (85–170°). The inversions is carried out for three CNR maps measured at 1810 UTC (Figure 4), 2110 UTC (Figures 5 and 6) , and 1832 UTC (Figure 7).

Figure 4.

Inversion by IBF (1810 UTC). The constraints on the MAVD and on the variations of the parameter Δδzb are applied, and the inverted parameter values are smoothed. (a) Measured Fn2, (b) inverted Fn2, (c) inverted parameters with respect to the azimuth, and (d) modified refractivity profiles with respect to the azimuth.

Figure 5.

Inversion by IBF (2110 UTC). The constraint on the MAVD only is applied, and the inverted parameter values are smoothed. The constraint on the variations of the parameter Δδzb is not applied. See Figure 4 for descriptions of Figures 5a–5d.

Figure 6.

Inversion by IBF (2110 UTC). The constraints on the MAVD and on the variations of the parameter Δδzb are applied, and the inverted parameter values are smoothed. See Figure 4 for descriptions of Figures 6a–6d.

Figure 7.

Inversion by IBF (1832 UTC). The constraints on the MAVD and on the variations of the parameter Δδzb are applied, and the inverted parameter values are smoothed. See Figure 4 for descriptions of Figures 7a–7d.

[27] The results are displayed: the measured CNR data (Figures 4a, 5a, 6a, and 7a), the inverted normalized propagation factor Fn2 is displayed with respect to the same azimuths (Figures 4b, 5b, 6b, and 7b), the values of the parameters retrieved by inversion are plotted with respect to the azimuth, from 85 to 170° (Figures 4c, 5c, 6c, and 7c), and the inverted refractivity profiles are plotted with respect to the azimuth (Figures 4d, 5d, 6d, and 7d). As the ducts vary with distance, the refractivity profiles at the antenna are displayed. The parameter δzb controls the duct variation with range. The color of each profile corresponds to its accuracy on the propagation factor, defined by a quadratic error following equation (8). Accurate profiles (αi = 1) are in blue, and inaccurate ones (αi = 0) are in red.

[28] The profiles at the antenna should be all the same as they are on the same physical point. However, the four-parameter profile is an approximation of the real refractivity structure and the inversion does only use information from range 10 km. If the profiles were all equal at the antenna, the parameter δzb would be the only one to vary with azimuth. This constraint is not included to keep sufficient degrees of freedom in the inverted refractivity structures. Applying this constraint could be useful if dealing with more complex refractivity duct models. The method gives the equivalent four-parameter profiles for each azimuth that fits best to the NRCS.

[29] The result in Figure 4 represents a good fit. The retrieved normalized propagation factor is close to the one obtained from the measurements, and the continuity in azimuth of the inverted duct parameters gives confidence in the result.

[30] The importance of the constraint on the parameter δzb is illustrated in Figures 5 and 6. Figure 5 depicts a result of the IBF technique on second data without applying the constraint on the parameter δzb. The result when applying this constraint is shown in Figure 6. The improvement can be observed in two ways. First, the inverted Fn2 map (Figure 6b) is much closer to the measured Fn2 (Figure 6a) when the constraint is applied. Second, the inverted parameters (Figure 6c) are smoother with respect to the azimuth. When δzb is smoothed over azimuth, all the parameters tend to be smoothed as well. Actually, the ill-posedness of the problem is very pronounced on the parameter δzb. Constraining this parameter significantly reduces the ill-posedness for all the parameters and makes the inverse results smoother. Last, in Figure 6d, according to the color scale, the retrieved CRM gives much more accurate normalized propagation factors when the constraint on δzb is applied.

[31] When a new constraint is applied the parameters often becomes more stable due to smoothing. If the inversion finds very different parameter values from one azimuth to the next, the smoothing highlights incorrect results, and the smoothed refractivity conditions gives an inaccurate Fn2 value. The parameter values in Figure 6 are continuous with respect to the azimuth. When these values are smoothed, the accuracy on the propagation factor is good. Around 120–125°, the parameter c2 changes fast. Due to the smoothing, this change is gradual in the inversion (Figure 6c) and the propagation factor is inaccurate on these two azimuths (Figure 6b). This is, however, a little loss compared to the efficiency of the smoothing, and these two inaccurate azimuths can be easily identified from Figure 6c. On other hand, the parameters obtained without the constraint on δzb (Figure 5) are uneven with respect to the azimuth. In such cases, the parameters after smoothing give wrong ducts, and the propagation factor in Figure 5b is inaccurate. This smoothing method gives a confidence criterion: if the normalized propagation map is accurate, it implies that the inversion gives realistic parameter values for that azimuth.

[32] Another example in Figure 7 illustrates the efficiency of the IBF RFC technique. The inverted normalized propagation factor obtained in Figure 7b is close to the measured represented in Figure 7a and the continuity in azimuth of the inverted parameters (Figure 7a) is strong. This result gives confidence in the IBF RFC technique.

[33] The differences between the measured and inverted propagation factors are plotted for the three cases studied in Figure 8. For the 1810 data, the error occurs mainly between 90 and 115°. When compared to Figure 4c, one can suppose that c2 is overestimated for azimuth 105°. The other errors mainly correspond to overestimated interference patterns. There are few errors for the 1832 data. One can notice few discrepancies around 150° as the interferences patterns in the simulated data are too pronounced. There are more errors on the 2110 data. Between 90 and 100° around 28 km, the error is due to an overestimated interference pattern. The errors on the two following azimuths are due to the smoothing as explained above. In general, for these three cases, the error is below 5 dB. Making the duct model more complex could increase the accuracy, but it would also decrease the resolution of the parameters.

Figure 8.

Inversion by IBF. Differences between the measured and inverted normalized propagation factors in dB for times 1810, 1832, and 2110 UTC.

[34] During the CNR measurements, five refractivity maps with respect to the distance and the altitude have been deduced from atmospheric measurements carried out by a buoy and a helicopter along the azimuth 150° and merged using specialized postprocessing software tools at JHU/APL. The inverted and measured modified refractivity with respect to the distance on this azimuth are compared on these five data, see Figure 9. The inversion is somewhat reasonable for a method giving an approximation of the refractivity in real time on all the azimuths visible by the radar. The retrieved ducts are close to the measured ones and give a good idea of the surface-based duct that is important for the radar coverage prediction at low altitude.

Figure 9.

Modified refractivity extracted from five data sets (1740, 1800, 1832, 1900, and 2110 UTC). Inversions (dashed gray) accompanied by the refractivity measurements (solid).

[35] Using the extracted refractivity profiles the radar coverage can be obtained. In Figure 10, the one-way propagation losses are simulated for two measured refractivity conditions and for the associate inverted ducts. The case at 1832 UTC is shown in Figure 10a. It corresponds to the less successful case (see Figure 9) and however implies a reasonable approximation of the propagation losses. The mean error is 4.8 dB with a standard deviation of 1.9 dB for the overall coverage. The error exceeds 20 dB only in the domain around 100 m of height and 50 km of range. For the case at 2110 UTC in Figure 10b, the mean error is also 4.8 dB with a standard deviation of 0.6 dB. Notice that the approximate refractivity is good for this case but small discrepancies still remain for the radar coverage propagation. For this case, the error is localized on narrow peaks. If the coverage obtained with the inverted ducts is not exactly as the ones from the measured refractivity, these are convenient approximations. One must keep in mind that this inversion is real time and could be improved considering the time evolution of the data.

Figure 10.

One-way propagation losses computed with the measured and inverted profiles at 150° for times (a) 1832 and (b) 2110 UTC.

6. Conclusion

[36] Previous RFC methods have focused on using the information along a single azimuth. The RFC method presented here explores a new approach: the results are rough approximations obtained in real time on all the radar azimuths, as previous developed methods give more precise results with higher computational times. The information from all azimuths is combined, i.e., the whole clutter map is used at once to obtain a solution. In order to use all azimuths at once a simple algorithm is used augmented with physical constrains.

[37] A real-time RFC method based on finding the best fit from a precomputed database is introduced. This method is improved using physical considerations: the mean dynamics of signal to be inverted and continuity in azimuth of refractivity. It is named improved best fit method (IBF). It can be applied in real time on all the azimuths visible by the radar. This method was applied on data from 1998 Wallops Island measurement campaign. The normalized propagation factor maps retrieved after inversion are close to the ones obtained from the measurements, and the retrieved refractivity structures give acceptable radar coverage.

[38] Some physical parameters were chosen empirically (the limits of Δδzb and ΔMAVD, and the limits of the definition of αi). They are related to physical properties of the problem but also to the 1998 Wallops Island data. Therefore, this method should be tested and tuned for other measurements in different conditions with other radar band and antenna, refractivity conditions, and sea states. In conclusion, this work demonstrates that azimuthal information improves an RFC system.


[39] This work was sponsored by DGA-CNRS (Direction Générale de l'Armement–Centre National de la Recherche Scientifique), and the authors would like to thank Yvonick Hurtaud from DGA, Célar (Centre d'Electronique de l'Armement), for his support.