The 21 cm multibeam receiver on the Parkes telescope has been used for many neutral hydrogen (HI) imaging projects, for example, the HI Parkes All Sky Survey (HIPASS) and the Southern Galactic Plane Survey. In many experiments to date, basic radio frequency interference (RFI) rejection has been accomplished on a per-feed basis, using the median statistic throughout the processing, which eliminates outliers in a nonparametric fashion at the expense of increased statistical noise. The classic 21 cm baseline ripple problem has been largely ignored, except for the subtraction of a coarse template which is calculated independently of feed, time, telescope elevation, and receiver rotation. The ripple is especially strong during the daytime, when the Sun acts as a broadband RFI noise source. Here we report on new techniques that we are developing, for application to raw, postcorrelation multibeam data, which handle RFI rejection and ripple suppression in more sophisticated ways. RFI can be identified by the presence of coincident outliers on multiple feeds, by sharp increases or decreases in flux time series that are inconsistent with a beam-smeared sky, and by the presence of highly polarized flux. We present our progress with this approach and highlight some remaining difficulties. We also present a novel technique for modeling the variation during telescope scans of 21 cm baseline ripple caused by stray continuum radiation undergoing multiple reflections in the telescope cavity and demonstrate its application to HIPASS data; this may increase the usefulness of daytime spectral line observing.
If you can't find a tool you're looking for, please click the link at the top of the page to "Go to old article view". Alternatively, view our Knowledge Base articles for additional help. Your feedback is important to us, so please let us know if you have comments or ideas for improvement.
Barnes et al.  described the neutral Hydrogen (HI) Parkes All Sky Survey (HIPASS), the observations and the processing techniques developed and implemented for data reduction and robust imaging. To remind the reader, HIPASS is a survey for HI emission from galactic and extragalactic objects over the entire sky south of declination +2°, with an effective integration time of 450 s per beam. Since Barnes et al. , the survey has been extended north to +25°, and for the rest of this paper we consider the enlarged survey. The raw data comprise 34640 declination scans of length 8.6°, with the telescope moving at 1° min−1. The primary data product of HIPASS is a set of 538 data cubes, arranged in declination bands centered on −90°, −82°, −74°, …, +22°. The data cubes are presented in the orthographic projection [Calabretta and Greisen, 2002] with pixels measuring 4 arcmin by 4 arcmin on the sky, and 1024 channels extending from 1362.6–1426.5 MHz in the barycentric reference frame.
 HIPASS primarily images the hyperfine HI line emission from galactic and extragalactic sources having radial (recessional) velocities in the range −1281 < v⊙ < 12727 km s−1. However, additional line signals fall in the frequency coverage of the survey, such as galactic HI recombination lines and OH maser lines in galaxies at intermediate redshift. Continuum emission at 1.4 GHz from tens of thousands of galactic and extragalactic sources is also present in the data.
 The HIPASS data have provided a number of important results, including the first detection of the Leading Arm of the Magellanic Stream [Putman et al., 1998], the best determination of the HI mass function in the local Universe [Zwaan et al., 2003], and the largest and most complete homogenous catalogue of extragalactic HI sources (HICAT) [Meyer et al., 2004; Zwaan et al., 2004]. HIPASS spectra have been available via a web interface since 2002, and HICAT has been recently published by the Australian Virtual Observatory (http://www.aus-vo.org).
 A number of defects are known to be present in first-generation HIPASS images. The sliding window bandpass estimator performs poorly near bright sources, and at the start and end of scans, leading to poor dynamic range. In lieu of parametric methods for detecting and excising bad data (e.g., radio frequency interference), the robust but nonlinear median statistic has been used liberally throughout the processing pipeline, leading not only to increased noise but also to peculiar image artifacts such as an image beam shape that depends on source distribution and brightness and the (related) property that the point source image flux scale is different to the integrated flux scale. Additionally, baseline ripple has been largely ignored, except for the subtraction of a very basic spectral template which is calculated after the nonlinear combination of spectra collected over a few years, by up to 26 independent feeds, over many directions in the Parkes Alt-Az space.
 We have embarked on a project to reprocess the HIPASS data and address all known defects of the first-generation images. In this paper, we describe the simple but effective parametric, multibeam radio frequency interference (RFI) mitigation techniques that we are developing with a view to adopting a standard Fourier transform gridding algorithm in place of the nonlinear median gridding used previously. We also describe our efforts to characterize and suppress the classic 21 cm baseline ripple which is ubiquitous in HIPASS data, and whose most common causes are the daytime, broadband RFI noise source known as the Sun, and the population of 1.4 GHz radio sources brighter than ∼50 mJy.
2. Interference Mitigation
2.1. Status Quo
 The HIPASS project acquired data for a total integration time of ∼240 days gathered over five years. Consequently, the postcorrelation data set represents an extensive sampling of the 1.4 GHz radio frequency interference (RFI) environment at the Parkes telescope during the period 1997–2002. The data are corrupted by a wide range of sources, including Global Positioning System (GPS) satellites passing through and broadcasting in the sidelobes of the telescope, distance measuring equipment beacons from the Parkes regional airport some ∼10 km distant and from overhead planes, the 43rd harmonic of the 33 MHz clock of a computer located in the telescope visitor center, closed-circuit TV and networking systems in the telescope tower, and a few internally generated, spectrally unresolved birdies. Some of the on-site RFI sources were eliminated during the survey.
 First-generation HIPASS processing made extensive use of the median statistic, especially in the bandpass estimation and gridding steps, to suppress the impact of RFI. In all but pathological cases, the median is more robust to outliers than the mean. Median processing worked moderately well for short-term transient RFI such as the GPS L3 beacon which transmits at high power (it can saturate the samplers) for short periods (∼2 min) with a low duty cycle (∼2–5%), but less well for long-term persistent RFI. This is simply because for individual pixels on the sky the median gridder combines data acquired at different times, and ephemeral RFI is easily discarded as outlying data, whereas enduring corrupting signals can lead to genuinely good data being discarded by the median.
 To illustrate the effectiveness of the median, we show in Figure 1 a comparison of image noise as a function of channel number for a single HIPASS image cube gridded using mean and median statistics. Please note that the plots are offset along the y axis for clarity. While the baseline noise level in the median gridded cube is higher, significant suppression of RFI features is achieved. (This is partly because the median statistic standard error is ∼1.2 times that of the mean statistic [Kendall and Stuart, 1963], and partly due to scaling that is applied to the pixels to correct the flux scale.) For example, the GPS L3 beacon corrupts the mean gridded cube significantly in channels 720–740, but barely perturbs the median gridded cube. This is shown explicitly in Figure 2, where the mean gridded map for channel 734 is ruined by the presence of GPS flux, while the same channel map gridded with the median statistic is mostly unaffected by GPS RFI. Note that the per-channel noise statistic used to generate Figure 1 is half of the interquartile range, which is itself robust, and RFI features that look quite mild in the plot are actually corrupting an appreciable fraction of the respective channel map(s).
 Despite our best intentions and the use of the median statistic, RFI has had a serious influence on the efficiency of constructing and the completeness limits of the HIPASS catalogue. Figure 3 shows the impact of a number of known (and a few unknown) RFI sources on the “second-check” source candidate list on which HICAT is based. RFI sources are variously responsible for a few dozen (e.g., the CCTV line at ∼10,500 km s−1) to a few thousand (e.g., the GPS “line” at ∼8,600 km s−1) false sources, which have subsequently been (mostly) cleaned out of the final catalogue. At this point, the reader might wonder why such an excess of candidate sources is present near RFI frequencies, given that Figures 1 and 2 suggest the median gridded cubes are actually pretty clean. We believe there are two answers to this question, both related to the standard error on statistics such as the mean or median. The standard error on the mean is proportional to the standard deviation of the sample distribution (σs) and inversely proportional to the square root of the sample size (Ns). For the purposes of this discussion, the standard error on the median exhibits similar dependencies.
 In regions where the median has (implicitly) excised many outlying data points, leaving a sample that otherwise just looks like a smaller sample of perfectly good data points, the error on calculated pixel values simply increases with , resulting in elevated HIPASS image noise. This is likely to be the case in regions of transient, infrequent interference such as GPS signals: if ∼20% of the data is determined to be bad (e.g., implicitly by the median), then the image noise will increase by ∼12%. In other cases (where, for example, the median statistic is unable to discriminate between “good” and “bad” data) the standard deviation of the sample itself will be elevated, and so the noise on the calculated pixel values increases with σs. The 1408 MHz correlator line appears to have corrupted the data in this latter way. In reality, both effects are present in any sample, and since the HIPASS catalogue second-check list is constructed using a flat threshold, image regions with higher noise will produce more candidate sources above the selection limit. Even if GPS-afflicted data were manually flagged and the median estimator replaced with a more “well-behaved” statistic (e.g., the mean), there would still be an overpopulation of HICAT second-check sources near the GPS frequency, due to increased image noise in the channels where data was explicitly removed. Future algorithms for generating candidates from reprocessed HIPASS data will certainly consider the local noise and flagging levels in an attempt to circumvent this problem.
2.2. Doing Better
 We have shown that the median is good at suppressing RFI (Figures 1 and 2. However it does this at the expense of increased noise in the bulk of RFI-free channels (Figure 1) [Kendall and Stuart, 1963]. As well, extensive simulations were needed to calibrate the integrated and peak flux scales of HIPASS images, which vary for point and extended sources and bright and faint sources. Indeed, the beam shape itself is variable because of the median gridding statistic. With this background, our aim is to minimize the impact of RFI and lower the general noise floor closer to the theoretical best value and produce more well-behaved images. Stevens  has demonstrated that a Fourier transform gridding approach can produce images with a noise floor around ∼25% lower than median gridding. Our approach therefore is to replace the implicit robustness of the median with simple parametric techniques for identifying and excising RFI, so that more traditional gridding techniques, such as Fourier transform gridding, can be used. We now describe some of the parametric techniques we are developing toward this goal.
2.2.1. Explicit RFI Tagging
 There exists a small collection of RFI sources which have corrupted HIPASS data and can be removed explicitly. This is possible because they are single-channel RFI spikes, and they were present for known epochs of the survey. For example, the 11th harmonic of the multibeam correlator clock was ever present at 1408 MHz until the correlator was shielded in early 1999. Lines like this can be dealt with explicitly provided we know the start and end times, and the precise frequency, of the corrupting signal. When reading data for processing, we simply calculate which channels to flag and mark them as bad. We keep track of flagged data by storing a vector of flags for each and every spectrum. The flag vectors are initialized to zero (meaning all good data) at the start of the processing pipeline and are set to a nonzero value to indicate bad data. Subsequent processing and gridding steps simply ignore data that is marked bad in this way. The flag vectors are translated and dilated as necessary when their corresponding spectra are Doppler tracked and/or smoothed. Provided observations of the same part of sky are spread over a few months, in many cases we have an excellent chance of still retaining good data for the flagged channels, thanks to the motion of the Earth which effectively Doppler shifts most ground-based RFI sources across a few HIPASS channels throughout the year, depending on the celestial position being observed. In the same way that RFI-corrupted data in a single topocentric channel “bleeds” into neighboring channels in the (nontopocentric) HIPASS images, we can effectively bleed good data back into otherwise completely flagged channels in the HIPASS images. An example of explicit RFI tagging is shown in Figure 4, where a channel map adjacent to the 1408 MHz line is shown with mean statistic gridding, without and with explicit RFI tagging. The improvement is clear, and this very simple flagging technique has reduced the RMS image noise by 30% in this case.
2.2.2. Coincidence RFI Detection
 There are many ways to detect RFI, especially with a focal plane array system like the Parkes multibeam receiver. Bright, off-axis interfering signals will generally be detected in multiple feeds, and a very simple coincidence detection technique can be employed to identify such signals. We have initially implemented a single-level coincidence discriminator, which flags points in (channel, time) space. For each spectral channel, we evaluate the median Mp and median absolute deviation from the median mp within a ∼100 cycle scan, for each spectral product p. Points in cycle-channel space are flagged as bad when the spectral flux deviates by more than (xc · mp) from Mp for more than X products p. The two parameters for this simple coincidence flagging technique are xc, the threshold scale which defines highly outlying data, and X, the threshold count which defines how many independent feeds must exhibit highly outlying data for it to be considered a coincidence event.
Figure 5 provides an example of the usefulness coincidence flagging. In this example, we show the same channel map as in Figure 2, but gridded in both cases this time with the mean statistic. We apply coincident flagging with xc = 8.0 and X = 6; that is, we flag data where at least six feeds show highly outlying data. The improvement is substantial, showing nearly complete rejection of the GPS signal on the left of the image, and complete preservation of good signal elsewhere in the map.
2.2.3. Time-Varying RFI Detection
 The combination of the smooth primary beams of the telescope and a slow scan of the receiver across the sky means that receiver voltages will vary relatively smoothly and gradually with time. Put another way, the convolution of the smooth telescope beams with the sky brightness distribution produces a smooth result, independent of fine structure in the sky. A time varying signal though, such as an RFI source which turns on and/or off during a scan, can insert non-smooth segments in the time series. These segments can be identified by searching for significant steps in flux in individual channels between adjacent integrations.
 For a scan rate of one degree per minute, the 5 s integration time of most of the Parkes multibeam surveys (including HIPASS) yields an acceptable range of ∼[0.4, 2.5] in the ratio of channel fluxes acquired in timewise-adjacent spectra, in noise-free conditions. This is calculated for the steepest gradient in a 15′ Gaussian beam. We have implemented a simple time-varying RFI detector on the basis of this principle, and parameterize the detector with two simple thresholds—xs and xn—such that timewise-adjacent flux ratios outside the range xs · [0.4, 2.5], where both of the flux values exceed xn · σc, constitute non-smooth events indicative of time-varying RFI corrupting the data. The latter constraint is used to prevent noise-dominated sequences of measurements being identified as non-smooth events, and σc is a robust measure of the instantaneous noise in the channel being considered. The algorithm proceeds by identifying complementary non-smooth events (i.e., upward then downward or downward then upward jumps) in channel and system temperature time series for each product, and flagging all data between the complementary events.
 An example of parametric flagging for non-smooth events (time varying RFI) is shown in Figure 6, for xn = 12 and xs = 6. Note that this is a fairly conservative effort, and some less bright features remain which are clearly non-smooth events.
3. Off-Axis Spectral Baseline Ripple
 Continuum emission sources can introduce important non-flat components into band pass–removed, calibrated Parkes multibeam spectra. In particular, strong continuum radiation entering the telescope cavity can manifest itself as sinusoid-like ripple in the baseline. The baseline undulates with frequency (or equivalently, channel number) in conditions like this because of reflections of strong signals within the cavity between the dish, the focus cabin, and its support legs. While the bulk of a strong off-axis continuum signal is rejected by the focussing geometry of the telescope, a small fraction of the incident energy can enter and be reflected from the receiver in the focus cabin assembly, down to the dish, and back again. The result of autocorrelating such a signal is a spike in the time delay spectrum corresponding to the time taken for the signal to travel this extra path length and re-enter the receiver system. The Fourier transform of this time delay spectrum yields a frequency spectrum with a sinusoidal component, which in some cases can be more than an order of magnitude greater in amplitude than the receiver noise. The ripple spectrum is usually complicated by the presence of many harmonics, corresponding to various reflection paths in the telescope structure. In some sense, baseline ripple is the radio astronomer's optical flare, producing image artifacts and generally lowering image contrast.
 The main reflection path in the Parkes telescope, from the receiver directly to the dish center and returning to the receiver, that is, twice the focal length, is 52 m in length, which corresponds to a light travel time of 0.173 μs. This delay gives rise to a sine wave in the Fourier transform spectrum having this wave number, or equivalently, a period of 5.7 MHz in frequency space or 1200 km s−1 in (HI) velocity space. Off-axis ripple in the baseline is generally not removed by bandpass correction, since in the time the multibeam sweeps less than a beam width on the sky, far field radiation patterns may change substantially, leading to different standing wave patterns. Some form of postprocessing is necessary to deal with severe cases of continuum induced ripple.
 By far, the strongest and most persistent radio continuum source in the Parkes sky is the Sun. The solar continuum flux is so bright that it can lead to spectral baseline ripple regardless of the angular offset of the Sun from the telescope axis. Solar emission is the most common cause of ripple, and is problematic for a high fraction (say ∼40–60%) of spectra acquired during daylight hours. Depending on the epoch spread of observations of a particular part of the sky, solar ripple can be visible in gridded spectral line cubes, or can more generally lead to increased image noise. Other than the Sun, there are hundreds of continuum sources whose locations are static on the sky and whose flux densities are sufficient to generate baseline ripple in multibeam spectra as they move through distant sidelobes of the telescope beams.
3.1. Status Quo
 Baseline ripple has plagued HI radio astronomy for decades. All on-axis parabolic reflector systems are subject to some level of baseline ripple, regardless of observing frequency. HI astronomy though suffers particularly badly, because the principle ripple components are usually similar in spectral width to the HI line widths of galaxies. Newer telescopes, such as the Greenbank Telescope (GBT), attempt to suppress baseline ripple by adopting off-axis receiver geometries, but for existing telescopes like Parkes, other solutions are required. Hardware solutions, such as baffling the underneath of the focus cabin, can be effective, but these approaches generally increase the system temperature, lowering the overall sensitivity of the telescope.
 For telescopes like Parkes, software solutions will be more flexible and probably more successful in the long run. For on-axis ripple (ripple on the spectra of continuum sources), a double beam switched calibration approach as described by Ghosh and Salter  for the Arecibo telescope is likely to work well, and we have acquired suitable measurements at the telescope to implement this approach for 64 MHz multibeam observations at Parkes. For both on and off axis ripple in HIPASS, a template subtraction method was applied in the gridded image cubes, and worked well to first order. However, since time and product information has been lost in the gridded images, this approach is limited to correcting gross ripple effects only.
Briggs et al.  described a novel approach to handling slowly varying ripple, such as due to off-axis continuum sources, for drift scans with the Arecibo and Nancay telescopes. The technique (hereinafter referred to as Briggs harmonic tracking) deduces a model for the amplitude and phase behavior of harmonic components of observed spectra, and directly subtracts the model from the data. The time series of a particular harmonic is subdivided into windows that are long enough to be immune to the presence of a bright source for a short period of time, but sufficiently brief to allow the nonlinear evolution of the harmonic amplitude and/or phase. Within each window, an estimated linear fit is made to the amplitudes directly, and to the phases via a Fourier transform. Where significantly strong harmonics having trustworthy amplitude and phase fits are found, they are accumulated into a model of the baseline ripple. Once a selected range of harmonics has been analyzed in this way, the resultant model is subtracted from the data. Figure 6 of Briggs et al.  shows how very effective this procedure can be. In the following, we describe our modifications to this algorithm to handle Parkes multibeam survey data.
3.2. Briggs Harmonic Tracking for Parkes Multibeam Scans
 The main hurdle in applying the Briggs harmonic tracking technique to Parkes survey data is that scans with the Parkes multibeam are generally short in time (i.e., contain only ∼100 5 s integrations) and are slow enough that point source emission on the sky is generally detected in many (i.e., ∼8) contiguous integrations. Briggs et al.  nominate a window length p = 2N, with N in the range 4–7. Smaller windows allow for shorter (and better) piecewise linear approximations to harmonic evolution, at the expense of signal-to-noise ratio and a certain quantization of the fitted phase rotation rate. Larger windows alleviate the problem of sources polluting the harmonics, and improve signal-to-noise ratio, but will produce incorrect results for rapidly rotating phases. For Parkes multibeam scans, there is almost no flexibility in choosing N: it must be longer than the time a single source may be detected, certainly shorter than one quarter of the scan length, and so we set N = 4 (p = 16).
 For such a small window, there is the chance of significant contamination of the tracked harmonics with genuine HI signal harmonics. To avoid this, we use an iterative, masked, clipped polynomial fit to the amplitude time series to provide the harmonic phase evolution. The mask is provided by the locations of known HI sources on the sky from the HICAT database [Meyer et al., 2004], and ensures that integrations containing genuine HI flux do not contribute to the amplitudes of the harmonic model. This fit has the additional benefit over the smoothing approach of Briggs et al.  in that it provides predictive power and yields improved results at the start and end of scans. A way of addressing the vector averaging of phases to suppress the effect of source flux is less obvious, and work is continuing in this area. Nevertheless, the examples we show exhibit a substantial robustness in the harmonic models to HI sources in the sky simply as a result of using a masked fit to the amplitudes. Briggs et al.  noted the possibility that many different harmonics can be relevant at any one time, being present because of different scatterings in the telescope substructure. We address this by applying the harmonic tracking technique iteratively. We have found that typically after four iterations, there is no further benefit in tracking.
 In Figure 7, we show a simulated example of harmonic tracking. An image of dimensions 1024 × 106 pixels (similar to HIPASS scan data comprising 1024 frequency channels and ∼100 integration cycles) was generated. To this, a bright top hat “source” was added, above a low-level noise field and two harmonics of slowly varying phase. This image was submitted to the phase tracking routine, which generated an excellent model of the two harmonics, with no residual contamination by the strong source. The result of subtracting the calculated model from the input image is shown. In Figure 8, we show the subtle but definite improvement made to a velocity-declination slice of a HIPASS image cube, after individually subtracting harmonic models calculated for all ∼1900 input scans. That is, the raw correlator data were reprocessed, including iterative calculation and subtraction of a harmonic model, for all 13 beams and 2 polarizations of the 75 scan files. After reprocessing the scans were gridded into a single image cube using the standard multibeam gridding software.
 We have developed parametric techniques for identifying and removing interference-corrupted data from Parkes multibeam survey data. The prospects for replacing the median gridding algorithm of the first generation HIPASS processing with parametric RFI mitigation and a mean or Fourier gridding technique are good. We have shown that a modified version of the Briggs phase tracking technique can yield excellent suppression of solar and far-field continuum baseline ripple in Parkes multibeam spectra. The techniques described will shortly be used to reprocess the entire HIPASS raw data set, yielding a deeper and more well-behaved extragalactic HI all-sky image and catalogue.
 In light of the work we have described, we can also make two significant recommendations to future observers using the Parkes 21 cm multibeam system to produce wide-field HI images:
 1. Make active scans as long as practically possible in angular distance (and time) to increase the baseline for phase-tracking ripple removal, thereby improving its likely performance.
 2. Use the highest practically possible frequency resolution correlator and filter configuration to reduce the fractional impact of the numerous, known, narrow RFI lines on the acquired data.
 We express our sincere thanks to the staff at the Parkes Observatory, especially Mal Smith, who worked tirelessly and well beyond the call of duty to identify and remove as many RFI sources as possible during the lifetime of the HIPASS project. Their efforts saw a substantial improvement to the Parkes RFI environment, benefiting all observers.