A two-stage tunable detector with enhanced rejection capabilities

This paper focuses on the method of an adaptive detector which works against Gaussian background with unknown parameters, such as covariance matrix. The authors design a tunable detector for point-like targets by mixing the Kelly’s generalized likelihood ratio test and the enhanced Rao test. The analytical expression of the false alarm probability and detection probability of the proposed detector are derived. The performances of the new scheme are assessed, and analysed in comparison with its natural counterparts. The results show that it can provide enhanced rejection capabilities of mismatched signals, at the price of a limited detection loss for matched signals.


INTRODUCTION
In the signal processing community, design of adaptive detecting schemes that can effectively detect point-like targets in the presence of Gaussian background has considerable interest in recent years, especially, when the covariance matrix (CM) is unknown. There are some classical detectors proposed in open literature. For example, Kelly develops a scaling factor for detection by using Generalized Likelihood method [1], which is wellknown as the Kelly's GLRT (generalized likelihood ratio test). Robey et al. develops a two-step GLRT scheme called adaptive matched filter (AMF) [2], while in [3] Kraut and Scharf derive a fully-adaptive detector by applying the GLRT, called an adaptive coherence estimator (ACE) for partially homogeneous environment. The whitened adaptive beamformer orthogonal rejection test (WABORT) is proposed for rejection signals in [4]. More recently, in [5] De Maio devises a selective receiver based on the Rao test, and in [6] Orlando and Ricci improve the selectivity properties of the Rao test by introducing a noise-like interferer in the cell under test, called the enhanced Rao (ERao). All above schemes have the property of the constant false alarm rate (CFAR). It is worth mentioning that all above detectors require the pre-knowledge of the array normal vector. When a mismatched signal originating from mainlobe target or sidelobe interferer occurs, their performances will degrade greatly in practice.

Mismatched signals can occur for several reasons, such as
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. © 2021 The Authors. The Journal of Engineering published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology imperfect antenna shape, array calibration errors as well as multi-path propagation etc. It is not easy to design a solo scheme capable of rejecting sidelobe targets and ensuring a good detection performance under matching condition at the same time. In order to solve the dilemma, the two-stage tunable detectors have been introduced [7][8][9][10][11][12][13] which are obtained by mixing two detectors with different behaviours, for instance, the capabilities of detecting a matched signal and rejecting a mismatched one. The adaptive sidelobe blanker (ASB) [9] and the K-WABORT [11] are two well-known two-stage tunable detectors. Precisely, the ASB is realized by cascading the AMF with the ACE, while the K-WABORT is formed by the GLRT mixed with the WABORT. The above detectors have the CFAR performance, and gain the probability of detection (P d ) under expected probability of false alarm (P fa ). And more recent efforts in CFAR detection are shown, for instance, in [14][15][16][17][18][19].
In present work, we move a further step to improve the selectivity of two-stage tunable detectors. Precisely, we design a tunable detector by mixing two detectors with different performance in detection, called the K-ERao. We choose the GLRT and the ERao detector as first and second stage, respectively, to obtain the new scheme. The new detector has the CFAR performance for the unknown CM of background clutter. The analytical expression of the P fa and the P d of the K-ERao are derived. Simulation results show that the K-ERao can balance the performance of the K-ERao under matching conditions and the rejection ability of mismatch situations.

PROBLEM FORMULATION
Considering a uniform linear array of N sensors, z ∈ ℂ N ×1 is used as a complex vector (primary data) to find the useful signals. We assume the referenced data set z q ∈ ℂ N ×1 , q = 1, … , Q (Q ≥ N ), contains no signal component and has the same CM as z.
The problem of detection to be solved can be written as: where • H 0 is referred to as null hypothesis and H 1 as the alternative hypothesis; • n 0 , n q ∈ ℂ N ×1 , q = 1, … , Q, are complex vectors, which have zero mean and the same CM, R ∈ ℂ N ×N , which is an unknown positive definite matrix, given by, where E[⋅] and (⋅) H denote the statistical expectation and the conjugate transpose operation, respectively; • v ∈ ℂ N ×1 denotes the nominal steering vector; • a ∈ ℂ is an unknown scale parameter accounting for signal propagation and reflectivity.
The decision statistics of the GLRT and the ERao, h K and h E say, are given as follows [5]: where h ACE denotes the decision statistics of the ACE.
Cascading the GLRT and the ERao, we obtain a new twostage tunable detector, referred as the K-ERao in what follows.
It can be summarized to be where K and E denote the thresholds of the first and second stages, respectively. The two thresholds should be chosen according to an expected P fa .

PERFORMANCE ASSESSMENT
The analytical expressions of the P d and the P fa for the K-ERao detector are derived under matched and mismatched signals in this section. Considering the definition of the transformations of the maximal invariants in [10], we recall the decision statistic as follows:h and Observe that h E can be rewritten as follows:

P fa of the K-ERao
Assuming the H 0 hypothesis, we can get [13]: • Given ,h K is a vector that corresponds to the complex central F -distribution. It has a parameter of freedom degrees Then, we obtain the P fa of the K-ERao, given by ))] We can see the P fa of the K-ERao depends on the thresholds pair ( K , E ), but not the unknown CM. It is apparent that the K-ERao possesses the CFAR property with respect to R. It is important to note that the P fa of the two-stage detector depends on two thresholds. Therefore, there exist infinite number of threshold pairs to ensure the same P fa value as shown in Figure 1. Contour plots corresponding to different P fa values versus ( K , E ) are given in Figure 1. The simulation results illustrate that the K-ERao provides a balance performance between target detection and the interference rejection scenarios. The K-ERao degenerates to the GLRT as E = 0, and the ERao with K = 0. The operating point (threshold pairs) can be selected on the curve based on a specific application requirements.

P d of the K-ERao
Assuming two independent variate ′2 1 from a non-central chisquare distribution with n degrees of freedom and parameter and 2 2 from a chi-suqre distribution with n degrees of freedom. The non-central F -distribution F ′ with parameters (m, n, ) is defined by [20] F ′ (m, n, ) = ( ′2 1 ∕m)∕( 2 2 ∕n).
Considering a case for mismatched scenario, namely, the actual steering vector v m does not match the nominal steering vector v. Under the H 1 hypothesis, denote by the angle between v m and v, it can be shown that [13]: •h K is a vector that corresponds to the complex non-central F -distribution, when is given. It has parameter of freedom degrees 1, Q − N + 1. And the non-central parameter is defined by where we define the signal-to-noise ratio (SNR) here, and with ∈ [0, 2 ]; • is a random variable that obeys the complex non-central beta distribution. It has a parameter of freedom degrees, Q − N + 2, N − 1. The non-central parameter is where Thus, it is easy to show that the P d of the K-ERao is given by where P 1 (⋅) is the CDF of the random vectorh K ∼  1,Q−N +1 ( ), when the is given under H 1 . p 1 (⋅) is the probability density function (PDF) of the random vector that obeys  Q−N +2,N −1 ( ).

ILLUSTRATIVE EXAMPLES AND DISCUSSIONS
We present numerical examples to illustrate the detection performance of the new scheme. The results include simulations under matched signal and interference rejection scenarios. In particular, we compare the K-ERao with the ASB and the K-WABORT. The interference background is modelled as an exponentially correlated complex normal vector. We set the CM of background with one-lag correlation coefficient , which means that (i, j )th element of the CM is | j −i| , with = 0.9. Moreover, we set parameters N = 64, Q = 96 and P fa = 10 −4 and obtain the curves by means of numerical integration techniques. It should be noted that the term cos 2 is a measure of the mismatch between v m and v. Its value equals one for the matched case where v m = v and less than one otherwise.
As mentioned in Section 3.1, there exist infinite numbers of threshold pairs for two-stage detectors to ensure a given P fa value. Accordingly, these threshold pairs ensure different performances, such as the detection performances for mainlobe signal, and the rejection performances for sidelobe interference. In this section, the curves of P d versus SNR and the contours curves of P d as a function of cos 2 and SNR are used to highlight the different detection performances of two-stage detectors.
In Figure 2, the performances of the ASB and the K-ERao are analysed. We plot P d versus SNR for the detectors, for both the case of a matched signal and mismatched interference in subplot 2(a). The results show two curves for each of them. We choose P d = 0.5 and P fa = 10 −4 , and the threshold pairs correspond to the rejection scenarios, while the schemes have a loss about 0.5 dB relative to Kelly's GLRT. Moveover, in subplot 2(b), we report the contour curves of P d as function of cos 2 and SNR. Considering the parameters N , K and P fa , it can be seen from 2(b), the K-ERao has better performance of rejection than the ASB, and also has the same performance for detecting matched signals.
In Figure 3, we compare the proposed detector to the K-WABORT. We choose P d = 0.6 and P fa = 10 −4 , and the threshold pairs correspond to the rejection scenarios, while the schemes have a loss about 1 dB relative to Kelly's GLRT. We plot the P d versus SNR for both matched target and mismatched interference scenarios in 3(a), and report the contour curves of P d as function of cos 2 and SNR in 3(b).
As it can be seen from Figures 2 and 3, considering the same parameter N , K and P fa , the proposed detector has better performance of rejection than the other two detectors, and also has the same performance for detecting matched signals. In particular, the K-ERao can reject a signal with SNR = 30 dB with higher probability than the ASB and the K-WABORT. From another point of view, the proposed detector provides a compromise between the detection robustness and the rejection performance. Comparing with the natural competitors, the proposed detector can obtain a better rejection capability with high value of E under mismatched case, while it has the same detection performance with the ASB and the K-WABORT at about P d = 0.5 in case of perfectly matched signals. It is also important to notice that the K-ERao outperforms the other analysed detectors in presence of a weak matched signals with small SNR values.

CONCLUSIONS
We design a tunable detector in this work, called the K-ERao, which has two stages, the Kelly's GLRT and the ERao, with different performance under matched and interference rejection scenarios. We derived the analytical expressions of P fa and P d . The K-ERao detector has the property of CFAR for unknown CM of the background. Through choose one from the threshold pairs with the same expected P fa , the K-ERao can balance the performance under matching conditions and the rejection ability of mismatch situations. The Monte-Carlo simulation results illustrate that the K-ERao can ensure better interference rejection performance than the traditional schemes at a limited loss price under matching scenarios. It is worthy of analysing the new detector in different scenarios for future work.