An evaluation of the performance characteristics of the U.S. National Lightning Detection Network in Florida using rocket-triggered lightning

Authors


Abstract

[1] The performance characteristics of the U.S. National Lightning Detection Network (NLDN) were evaluated using rocket-triggered lightning data, acquired in the summers of 2001–2003 at the International Center for Lightning Research and Testing (ICLRT), at Camp Blanding, Florida. During the 3-year period, 37 flashes, containing a total of 159 (158 negative and 1 positive) strokes, were triggered at Camp Blanding. Flashes consisting of the initial stage only (having no return strokes) were not considered in this study. Directly measured currents were obtained for 122 of these strokes in 29 flashes. Camp Blanding and NLDN events were correlated using GPS time stamps. The NLDN recorded 95 Camp Blanding strokes in 31 flashes. Of these 95, usable directly measured currents were obtained for 70 strokes in 22 flashes. Flash and stroke detection efficiencies were estimated to be about 84% and 60%, respectively. Median location error was about 600 m, with larger location errors (greater than 2 km) being associated with strokes having smaller peak currents (5–10 kA). The NLDN tended to underestimate peak current, with the median peak current estimation error (signed) being about 18%. There was a steady trend of improved stroke detection efficiency from 2001 to 2003.

1. Introduction

1.1. U.S. National Lightning Detection Network

[2] Since 1989, the U.S. National Lightning Detection Network (NLDN) has provided lightning data covering the contiguous United States. During the evaluation period (2001–2003) discussed in this paper, a network-wide upgrade was underway, and was near completion at the end of 2003. A detailed discussion of the NLDN upgrade is given by Cramer et al. [2004] and Cummins et al. [2004]. A total of 106 sensors were in the network in 2001. Six sensors have been added in 2002 and 2003, and all sensors have been upgraded to the latest combined MDF/TOA technology, described later in this section. The sensors are distributed over the contiguous United States and relay data back to a central site where the sensor data are analyzed to locate and characterize cloud-to-ground lightning events. A map showing the locations of NLDN sensors in the Florida region, as of late 2003, is found in Figure 1.

Figure 1.

Map showing the locations of NLDN sensors in and around the Florida region, as of late 2003. The approximate location of Camp Blanding is also shown.

[3] The optimum lightning location is computed using a generalization of the χ2 minimization technique described by Hiscox et al. [1984]. For each stroke location, an error ellipse is computed on the basis of the assumptions about the sensors angle and timing accuracy. Spatial and temporal grouping rules are used to assign detected strokes to flashes before providing the real-time flash data to the end user. The data are processed in real-time and are generally available to users in about 30 s. Reprocessed data, corrected for errors in sensor calibration and communications delays, are generally available within a few days and are archived. It should be noted that the NLDN only reports data if the shapes of the recorded waveforms are characteristic of return strokes in natural, cloud-to-ground lightning.

[4] Cummins et al. [1998], using their detection efficiency model, estimated flash detection efficiency to be on the order of 80 to 90%, depending on the region, for flashes having peak currents of 5 kA and larger. The detection of only one stroke of a flash is required for a flash to be detected. Cummins et al. [1998] estimated stroke detection efficiency to be roughly 50 percent for the overall network, on the basis of a comparison of the average NLDN stroke multiplicity of about 2 (observed for 2 years after the 1995 NLDN upgrade) and the average stroke multiplicity of 3 to 4 reported by Thomson et al. [1984]. Rakov and Huffines [2003] estimated the NLDN stroke detection efficiency to be roughly 40% and 20% (corresponding flash detection efficiencies 78% and 62%) for Florida and New Mexico, respectively. In doing so, they used stroke counts in 1995–2001 NLDN data and “ground-truth” electric field and optical observations found in the literature. Note that the latter detection efficiency estimates are based on relatively small samples and involve an assumption that the detection efficiency for first strokes is the same as that for subsequent strokes, although first strokes typically have larger peak currents than subsequent ones and therefore should be associated with higher detection efficiency than subsequent strokes.

[5] The median stroke location accuracy has been theoretically estimated to be about 500 m over much of the United States, on the basis of the calculated 50 percent error ellipses, which assume that the distributions of angle and time errors are Gaussian [Stansfield, 1947; Cummins et al., 1998]. Detailed discussions of these models, along with assumptions used, are given by Cummins et al. [1995, 1998].

[6] As described by Cummins et al. [1998], return-stroke peak currents are estimated from peak magnetic field signal strengths measured by the individual NLDN sensors. During the 2001–2003 evaluation period, in order to account for propagation effects, a power-function attenuation of signal with distance was assumed, with the exponent (derived empirically by Orville [1991a] and Idone et al. [1993]) equal to −1.13. The raw signal strengths measured by individual sensors were normalized to 100 km using this power relationship. A different model to account for propagation effects is described by Cummins et al. [2004]. The range-normalized signal strength (RNSS) values, from all reporting stations within 625 km (to exclude signals with polarity reversals due to ionospheric reflection), are averaged and converted to a peak current estimate (Ipeak) using the empirical linear relationship (obtained using the triggered-lightning data of Idone et al. [1993])

equation image

[7] Prior to the 1995 upgrade [see Cummins et al., 1998], a different regression equation (also obtained using the data of Idone et al. [1993]) was used which contained an intercept value of 5.2 kA. The intercept of the new regression equation was constrained to zero in order to accommodate strokes with peak currents below 5 kA that were sometimes locatable after the 1995 NLDN upgrade.

[8] Although theoretical models can be used to evaluate the location errors and accuracy of peak current estimates, as well as the probability of detection, ultimately ground-truth data are required to verify the performance of the network. Such data should include the measured time, position, and peak current of lightning events in a specific region. The time and position can be determined using multiple-station video camera observations, as done by Idone et al. [1998a, 1998b]. The time, position, and peak current measurements can be obtained using instrumented towers [e.g., Diendorfer and Schulz, 1998] or rocket-triggered lightning [e.g., Orville, 1991b; Idone et al., 1993].

[9] As stated previously, the years 2002–2003 comprised an upgrade period for the NLDN [see Cramer et al., 2004; Cummins et al., 2004]. A summary of this upgrade is presented in Table 1. Early in 2001, before the Camp Blanding lightning triggering season, LPATS IV sensors replaced the older LPATS III sensors, with both models being time-of-arrival (TOA) sensors measuring the electric field. During the 2001 Camp Blanding triggering season, all relevant sensors were either LPATS IV or IMPACT type. The IMPACT-type sensors employ a combination of magnetic direction finding and time-of-arrival methods. The year of 2002 was a transition period; five sensors were upgraded to the IMPACT-ESP type before the start of the Camp Blanding season, and an additional sensor, located at Marion Junction (see Figure 1 and Table 1), was actually upgraded on 28 August 2002, during the Camp Blanding triggered-lightning season. The year of 2003 is essentially “post-upgrade.” Only one location in Florida, Tampa, remained equipped with an LPATS IV sensor, and it has since been replaced (19 November 2003) with an IMPACT-ESP type. The IMPACT-ESP sensor is an upgraded version of the original IMPACT sensor installed in the NLDN in the mid 1990s.

Table 1. NLDN Configuration In and Around the Florida Region for the Camp Blanding Triggering Seasons In 2001–2003a
NLDN Sensor LocationDistance from Camp Blanding, kmNLDN Sensor Type
2001 20 July to 18 August 20012002 9 July to 13 September 20022003 30 June to 15 August 2003
  • a

    The sensor locations given here correspond to those shown in Figure 1 (except for Jackson, Rockwood, and Russellville).

  • b

    Sensor was upgraded on 28 August 2002, during the Camp Blanding triggered lightning season.

  • c

    Sensor was upgraded and moved to a nearby location in November 2003, after the Camp Blanding lightning season.

Marion Junction, Ala.571LPATS IVLPATS IV/IMPACT-ESPbIMPACT-ESP
Homestead, Fla.517IMPACTIMPACT-ESPIMPACT-ESP
Naples, Fla.422LPATS IVLPATS IVIMPACT-ESP
Ocala, Fla.89IMPACTIMPACT-ESPIMPACT-ESP
Palm Bay, Fla.257IMPACTIMPACT-ESPIMPACT-ESP
Quincy, Fla.256LPATS IVLPATS IV
Tampa, Fla.227LPATS IVLPATS IVLPATS IVc
Wakulla, Fla.228IMPACT-ESP
Beckham Farms, Ga.424IMPACTIMPACT
Bledsoe Farms, Ga.426IMPACT-ESP
Savannah, Ga.255LPATS IVLPATS IVIMPACT-ESP
Lenoir, N.C.660IMPACTIMPACT-ESPIMPACT-ESP
Greenwood, S.C.481IMPACTIMPACT-ESPIMPACT-ESP
Myrtle Beach, S.C.533LPATS IVLPATS IVIMPACT-ESP

1.2. Rocket-Triggered Lightning

[10] In this paper, data from rocket-triggered lightning experiments conducted in the summers of 2001–2003 at the International Center for Lightning Research and Testing (ICLRT) at Camp Blanding, Florida, are used to evaluate the performance characteristics of the NLDN. The ICLRT occupies an area of approximately 1 km2 located 45 km north-east of Gainesville, home of the University of Florida. A review of the first 10 years of triggered-lightning experiments at the ICLRT is presented by Rakov et al. [2005].

[11] In classical rocket-triggered lightning, a small rocket (about 1 m in length) extends a thin grounded wire at a speed of about 100 to 200 m s−1. The rocket is launched when the quasi-static electric field at ground is sufficiently high (−5 to −6 kV m−1 in Florida). The presence of the wire enhances the electric field produced by the cloud charge source. If the conditions are right, an electric discharge (known as a leader) propagates upward from the top of the wire, resulting in a flow of current in the wire, and when this current is sufficiently high (about 100 amperes) for a sufficient period of time (some milliseconds), the wire explodes and is replaced by a conducting plasma channel [Rakov et al., 2003]. Following this process, a steady current of a few hundred amperes (known as the initial continuous current or ICC) flows to ground for several hundred milliseconds. The upward leader, destruction of the wire, and ICC comprise the initial stage (IS) of classical rocket-triggered lightning. After the cessation of current, a negatively charged downward-propagating dart leader may traverse the gap between the cloud charge source and ground. When this leader reaches ground, a large surge, known as a return stroke, propagates up the channel and neutralizes (lowers to ground) the charge that was deposited onto the channel by the dart leader. The return stroke has a peak current of typically 10–15 kA and a risetime of some hundreds of nanoseconds. This leader/return-stroke sequence (stroke) may be followed by up to 10 or more additional strokes, although the typical number of strokes per flash is 3 to 5.

[12] These triggered-lightning strokes are similar to subsequent strokes in natural negative cloud-to-ground (CG) lightning, [e.g., Uman and Krider, 1989; Le Vine et al., 1989; Fisher et al., 1993]. Hence the conclusions based on triggered-lightning data are thought to be applicable to subsequent strokes in natural downward lightning, but not necessarily to natural first strokes.

2. Data

[13] During the summers of 2001, 2002, and 2003, 37 flashes containing a total of 159 strokes were triggered at the ICLRT. Of these 37 flashes, 36 were classically triggered and one flash (consisting of four strokes) was the result of an unintentional altitude trigger (in altitude rocket-triggered lightning, the triggering wire is ungrounded). Flashes consisting of the initial stage only (no return strokes) were not considered in this study. All strokes lowered negative charge to ground with the exception of one positive stroke having a peak current of about +5 kA. This positive stroke followed the channel of a negative stroke in a two-stroke “bipolar” flash [see Jerauld et al., 2004]. The average number of strokes per flash was 4.3.

[14] A summary of the Camp Blanding flashes and strokes is presented in Table 2. Care was taken to distinguish between small return strokes and M-components (current pulses having a peak up to several kiloamperes and a risetime of a few microseconds or more, and a typical duration of tens to hundreds of microseconds, that are superimposed on continuing currents following return strokes).

Table 2. Summary of Flashes and Strokes Recorded at Camp Blanding During the Summers of 2001–2003, Along With the Corresponding NLDN Detection Efficienciesa
YearNumber of Flashes TriggeredNumber of NLDN Detected FlashesNLDN Flash Detection EfficiencybNumber of StrokesNumber of NLDN Detected StrokesNLDN Stroke Detection Efficiencyb
  • a

    Flashes consisting of the initial stage only (having no return strokes) were not considered in this study.

  • b

    The NLDN flash and stroke detection efficiencies for natural downward lightning are expected to be higher owing to the presence of first strokes that typically have higher peak currents than subsequent strokes.

  • c

    This includes one +5 kA positive stroke which was not detected by the NLDN (see sections 2 and 4.1).

200111982%331752%
2002141286%774457%
2003121083%49c3469%
2001–2003373184%159c9560%

[15] During the 2001 season, lightning was triggered at two launching locations: an underground launcher [see Schoene et al., 2003] and an 11-m-high tower launcher [see Mata et al., 2003]. In 2002 and 2003, the underground launcher was not used and a mobile launcher was employed in addition to the tower launcher. Over the course of the 2002 and 2003 seasons, the mobile launcher was placed at five different locations, so that there were a total of seven distinct launching locations for the 2001–2003 period.

[16] In all cases, the lightning current was measured at the base of the launcher with a non-inductive current-measuring resistor (shunt). Different shunts were used at different launchers, but in all cases the bandwidth of the shunt exceeded 5 MHz. All current data were transmitted to a shielded trailer by fiber-optic links, where they were digitized on digital storage oscilloscopes (DSO). The fiber-optic transmitters and other electronics associated with the current measurements were housed in sealed steel enclosures. Care was taken to account for the variation in gain of the fiber-optic links by measuring calibration signals before and after each storm. The uncertainty of the calibration of the current-measuring system is estimated to be at most about 10%.

[17] For each flash, the lightning current was digitized at two different sampling rates. Depending on the experiment and year, a LeCroy DSO sampled the data at 20 or 25 MHz (−3 dB filtered at 5 or 12 MHz, respectively) while a Yokogawa DSO sampled at 1 or 2 MHz (−3 dB filtered at 500 kHz). The LeCroy digitizers operated in segmented memory mode, such that data were only recorded within a 100 μs to 10 ms window (depending on the experiment and year) for each return stroke whose peak current exceeded the trigger threshold. The Yokogawa digitizers operated continuously for several seconds, and hence data were recorded for all strokes and interstroke periods. In general, the peak current estimates for the Yokogawa data are slightly lower than those for the LeCroy data, owing to the sampling rate and upper-frequency response of the Yokogawa digitizer. Further, since a separate trigger was required for each stroke recorded by the LeCroy digitizer, strokes not exceeding the trigger threshold (ranging from about 4 kA to 10 kA, depending on the experiment and year) were not recorded. Hence the peak currents obtained from the LeCroy data are probably biased toward higher values, while those obtained from the Yokogawa data are not.

[18] For some flashes, the LeCroy and/or Yokogawa digitizers failed; for other flashes, the current-measuring system itself failed. Finally, for a few strokes in 2002, the lightning current split between two instrumented paths and, in those cases, the two current waveforms were summed to obtain the overall current peak value. If the combined waveform appeared severely distorted (possibly owing to malfunctioning of one or both of the individual measurements), the peak current values were not used in this study. Hence the total number of strokes for which peak currents were obtained (122) is less than the total number of strokes recorded (159).

3. Methodology

[19] The time-correlated Camp Blanding and NLDN events were examined to determine the following NLDN performance characteristics: (1) flash detection efficiency, (2) stroke detection efficiency, (3) location accuracy, and (4) errors in peak current estimates. These characteristics were evaluated for each of the three evaluation years (2001, 2002, and 2003), as well as for all data combined.

[20] To establish a correlation between Camp Blanding and NLDN events, the time stamps on each event were compared. The Camp Blanding system is capable of recording GPS (Global Positioning System) times, having about 1 μs accuracy, for individual strokes, although times are typically only recorded for strokes having peak currents exceeding the trigger threshold of the LeCroy digitizers. When GPS timing was not available for individual strokes within a flash, interstroke interval timing was used to calculate the stroke times. The Yokogawa digitizers recorded full-flash current waveforms, and interstroke intervals could be determined with about 1 μs accuracy. Interstroke interval timing was also extracted from the LeCroy records, which were much more precise, but not available for all strokes. For flashes with GPS timing, the search scope in the NLDN database was usually limited to within one second of the Camp Blanding time and within a 20 km radius of the launcher. If a stroke was not found, the radius was increased. In the few cases when Camp Blanding GPS timing was not available for an entire flash, the approximate flash time (usually taken from video records), in conjunction with interstroke intervals, was used to search the NLDN records, although the scope of the search was increased to within 2–3 s of the Camp Blanding time. In general, the Camp Blanding and NLDN times differed by only a few microseconds, although timing differences could be as high as some tens of microseconds, especially for strokes with relatively high location errors. Since the NLDN stroke times are determined from the stroke location algorithm (i.e., they are part of the stroke location solution), a relatively large location error typically results in a relatively large NLDN timing uncertainty. Once correlated strokes were identified, detection efficiency values were computed as ratios of the numbers of NLDN-detected events and all triggered-lightning events.

[21] Location accuracy was evaluated for all strokes reported by the NLDN, including those for which no peak currents were obtained. The location of the rocket launcher was taken as the ground strike point (except in the case of the altitude trigger, where the ground strike point was approximately 50 m from the launcher) and that position (accurate within a few meters) was compared with the NLDN-reported location, which corresponds to the centroid of the NLDN error ellipse. For a given stroke, the distance between these two locations was defined to be the stroke location error. The location errors were compared to the semi-major axis lengths of the corresponding NLDN 50% error ellipses as well as to the stated median location accuracy of the NLDN in the region. In addition to the absolute magnitude of the location error, the individual components (north-south and east-west) of the location error were also examined.

[22] As stated previously, Camp Blanding return-stroke current waveforms were recorded on high-bandwidth (5 or 12 MHz −3 dB upper-frequency response) LeCroy digitizers and low-bandwidth (500 kHz −3dB upper-frequency response) Yokogawa digitizers, the latter of which acquired continuous, full-flash records. For some flashes, only Yokogawa data were obtained, and for a few flashes, only LeCroy data were obtained. When only one type of record existed, that value of peak current was taken as the ground-truth for comparison with the NLDN estimate. When both types of records existed, the two peak current values (which usually agree very well) were averaged. The justification for this is that while the 12-bit Yokogawa digitizer may somewhat underestimate the peak current (owing to the limited bandwidth), the 8-bit LeCroy digitizer often yielded an overestimate of the peak due to the inclusion of bit noise. Figure 2 is a scatterplot of peak currents obtained from the LeCroy digitizers versus peak currents obtained from the Yokogawa digitizers, for all strokes for which data were recorded on both systems (71 strokes total). In general the two estimates are very close, although, as expected, the LeCroy system provided slightly higher estimates, as evidenced by the small positive intercept and the slightly greater-than-unity slope of the regression line. No usable Yokogawa currents are available for 2001. However, we did verify that no small strokes existed in the distorted Yokogawa records that were not recorded by the LeCroy digitizers. Thus there is no bias in our data set due to finite trigger threshold.

Figure 2.

Return-stroke peak currents from the LeCroy digitizers (ILeCroy) versus the corresponding values from the Yokogawa digitizers (IYokogawa), for 71 strokes in 18 flashes triggered at Camp Blanding during 2002–2003. No Yokogawa currents are available for 2001. The regression line is valid only within the range of variation of data, but is extrapolated in either direction to the edge of the plot. Also given is the linear regression equation for ILeCroy versus IYokogawa along with the corresponding coefficient of determination (R2) and sample size (n).

[23] The distribution of peak currents for all triggered strokes with measured current, obtained during 2001–2003 (122 strokes total), is given in Figure 3. Note that this distribution includes strokes that were detected by the NLDN as well as ones that were not.

Figure 3.

Histogram of Camp Blanding triggered lightning return-stroke peak currents, I, for 2001–2003. Shown below the histogram are statistics for the individual years, as well as for 2001–2003. Statistics given are arithmetic mean (AM), standard deviation (SD), geometric mean (GM), standard deviation of the log10 of the parameter (SD(log10I)), minimum value (min), and maximum value (max). Of 122 strokes (in 29 flashes) for which peak currents were measured in 2001–2003, 70 were detected by the NLDN.

4. Results and Discussion

4.1. Flash and Stroke Detection Efficiencies

[24] Table 2 gives the number of Camp Blanding flashes and strokes detected by the NLDN for each of the summers of 2001–2003, along with data for all three years combined. Note that flashes consisting of the initial stage only (having no return strokes) were not considered in this study. Since all strokes in classically triggered flashes are similar to subsequent strokes in natural negative downward lightning, the flash detection efficiency reported here is likely to be an underestimate of the true value for natural negative lightning flashes in Florida, since first strokes typically have larger peak currents than subsequent ones and thus should be associated with higher detection efficiency. However, the stroke detection efficiency reported here is probably representative of the true subsequent stroke detection efficiency for natural lightning in Florida (but not necessarily overall stroke detection efficiency). Also, the flash and stroke detection efficiencies reported here may be applicable to upward lightning (containing return strokes) initiated from tall objects, since upward flashes are thought to be similar to classically triggered flashes [Miki et al., 2005]. The presence of one altitude-triggered flash (see section 2) in the data is unlikely to affect these statements.

[25] As shown in Table 2, the flash detection efficiency was 82% (9 out of 11) in 2001, 86% (12 out of 14) in 2002, and 83% (10 out of 12) in 2003, with an overall flash detection efficiency of 84% (31 out of 37) for the 3-year period. The flash detection efficiency appears to be more or less constant over the 3-year period (although the data sets are relatively small and were acquired only during the summer months of each year), and is consistent with the expected flash detection efficiency of 80% to 90% estimated by Cummins et al. [1998].

[26] The observed stroke detection efficiency was 52% (17 out of 33) in 2001, 57% (44 out of 77) in 2002, and 69% (34 out of 49) in 2003, with an overall stroke detection efficiency of 60% (95 out of 159) for the 3-year period. The 2003 and 2001–2003 data include the one +5 kA positive stroke noted in section 2, which was not detected by the NLDN; omitting this event results in a slightly higher stroke detection efficiency for 2003 (34 out of 48, or about 71%) and virtually no change in the overall 2001–2003 stroke detection efficiency. The expected pre-upgrade stroke detection efficiency, according to Cummins et al. [1998], is about 50%, which is consistent with the value reported for 2001.

[27] The mean stroke multiplicity at Camp Blanding (defined as the total number of strokes divided by the number of flashes triggered) was about 3.0 in 2001, 5.5 in 2002, and 4.1 in 2003, with a combined 3-year average stroke multiplicity of about 4.3. This latter value is close to the average number of strokes per flash, 4.6, reported for natural lightning in Florida by Rakov et al. [1994]. The smallest Camp Blanding stroke multiplicity was 1 and the largest was 17, which is the largest value recorded at Camp Blanding to date. The corresponding mean NLDN stroke multiplicity (defined as the total number of strokes detected divided by the total number of flashes detected) was approximately 1.9 in 2001, 3.7 in 2002, and 3.4 in 2003, with a 3-year average stroke multiplicity of about 3.1. Cummins et al. [1998] report average NLDN stroke multiplicities (for natural lightning), measured over a 2-year period, ranging from 1.9 to 2.1 (2.37 for Florida in 1995–2001 according to Rakov and Huffines [2003]), which is consistent with the value observed for 2001. However, in 2002 and 2003, the average NLDN stroke multiplicity almost doubles, and this is likely related to the observed increase in stroke detection efficiency for those 2 years. It is worth noting that the ratio of the average NLDN stroke multiplicity to the average Camp Blanding stroke multiplicity increases monotonically over the 3-year period (about 0.63 in 2001, 0.67 in 2002, and 0.83 in 2003).

[28] Figure 4 gives the NLDN stroke detection efficiency as a function of peak current measured at Camp Blanding. Note that strokes for which no directly measured currents were obtained are not included, hence reducing the total number of strokes recorded at Camp Blanding in Figure 4 to 122, although some of these strokes without measured currents may have been detected by the NLDN. Since the strokes for which no peak current was measured do not belong to any specific peak current range (since the loss of peak current measurement is due to instrumentation failure as opposed to trigger thresholds or saturation), the results presented in Figure 4 should not be biased.

Figure 4.

NLDN stroke detection efficiency as a function of peak current measured at Camp Blanding. For each peak current range (bin size of 5 kA), the ratio given inside the column indicates the number of strokes detected by the NLDN (numerator) and the number of strokes recorded at Camp Blanding (denominator), for that peak current range. The total number of strokes whose currents were measured at Camp Blanding is 122, of which 70 were detected by the NLDN.

[29] The combined data for 2001–2003 suggest that the detection efficiency is probably near 100% for strokes having peak currents in excess of 30 kA, although the data set is quite small for that peak current range. The stroke detection efficiency decreases to 60–70% as current decreases from 30 to 10 kA, and drops to less than 30% for currents in the 5 to 10 kA range. No strokes with measured peak currents below 5 kA were detected by the NLDN. The detection efficiency for strokes in the range of 5 to 15 kA increased between 2001 and 2003.

4.2. Location Accuracy

[30] Figure 5 is a plot of the NLDN stroke location errors for 95 strokes in 31 flashes triggered during 2001–2003 at Camp Blanding. The origin corresponds to the actual strike location that was known within a few meters. The horizontal and vertical axes correspond to the east-west (east being positive) and north-south (north being positive) error components, respectively. For the 2001 and 2003 data, the location errors are distributed more or less uniformly about the strike location. The 2002 data appear to be divided into two major clusters, one of which is very similar to the 2001 and 2003 data (with a slight bias toward the west). The second cluster in the 2002 data appears to have the roughly same westward bias as the first cluster, but also contains a large bias toward the north. The north component of these large location errors ranges from about 2 km to about 11 km.

Figure 5.

Plot of NLDN stroke locations for 95 strokes in 31 flashes triggered during 2001–2003 at Camp Blanding. The origin corresponds to the actual stroke location (lightning triggering location). The horizontal axis corresponds to the east-west component of the location error, with positive values corresponding to east. The vertical axis corresponds to the north-south component of the location error, with positive values corresponding to north. The total area covered by the plot is 576 km2 (24 km × 24 km), with the inset showing the central 16 km2 (4 km × 4 km). Statistics given are arithmetic mean (AM), median, and standard deviation (SD), for each location error component.

[31] Figure 6 is a histogram of the NLDN absolute stroke location errors for the 95 strokes shown in Figure 5. The overall distribution has a long “tail” owing to the relatively large location errors observed in 2002. The inset of the figure shows the distribution of stroke location errors up to 1 km. The median stroke location errors are 0.3 km, 0.8 km, 0.5 km, and 0.6 km, for 2001, 2002, 2003, and 2001–2003, respectively. The corresponding arithmetic means are 0.7 km, 2.4 km, 0.6 km, and 1.5 km, respectively. The relatively large arithmetic mean location errors observed for the combined data are mostly due to the relatively large location errors (>2 km) observed for some of the 2002 events.

Figure 6.

Histogram of the NLDN absolute location errors for 95 strokes in 31 flashes triggered during 2001–2003 at Camp Blanding. All bins are 0.25 km except for the last bin, which includes all errors greater than 5 km, with a maximum of 11 km. The inset shows the histogram with bins of 0.1 km for errors less than 1 km. Corresponding statistics are given below the histogram.

[32] Figure 7 gives the NLDN absolute location error plotted versus the peak current, measured at Camp Blanding, for 70 strokes in 22 flashes. Note that the sample size of 70 in Figure 7 is smaller than the sample size of 95 in Figure 6 because, as stated previously, the number of strokes for which peak currents were obtained is smaller than the number of triggered-lightning strokes recorded at Camp Blanding. The majority of large (>2 km) location errors occur in the 5–10 kA range during 2002 (no strokes were detected with peak currents below 5 kA). Between 10 and 35 kA, most of the 2002 location errors are below 1 km. For 2003, the location accuracy for smaller peak currents appears to have improved, although there are not as much data in that range as in 2002. For 2001, all detected strokes were above 15 kA, so no conclusions can be drawn regarding the location accuracy for small strokes during that year.

Figure 7.

NLDN absolute location error versus Camp Blanding peak current, for 70 strokes with measured peak currents in 22 flashes triggered during 2001–2003.

[33] Figure 8 gives the NLDN absolute location error plotted versus the number of NLDN sensors which were involved in the location solution, for 95 strokes in 31 flashes. The number of reporting sensors ranged from 2 to 14 during 2001–2003. Figure 8 provides crude estimates of the apparent upper and lower location error boundaries for a given number of sensors. For example, for strokes detected by only 2 sensors, the smallest location error observed was above 2 km. For strokes detected with 3 sensor solutions, the majority of location errors fall between 0.1 and 3 km, although some outliers exist beyond 3 km. As the number of reporting sensors increases, the upper bound on location error appears to decrease. It should be noted that all of the two-sensor locations in 2002 were located by the Ocala and Palm Bay sensors (see Figure 1 and Table 1), resulting in large location errors, apparently due to the baseline orientation relative to the source location (Camp Blanding).

Figure 8.

NLDN absolute location error versus the number of reporting NLDN sensors, for 95 strokes in 31 flashes triggered during 2001–2003.

[34] The NLDN 50% error ellipse, calculated for each stroke location solution, is defined as a confidence region for which there is a 50% probability that the actual stroke location lies within the area circumscribed by the ellipse, with the center of the ellipse being the most-probable (reported) stroke location. Hence the semi-major axis of the 50% ellipse is usually viewed as the median (50%) location error. Corresponding error ellipses for any probability level (e.g., 90%) can be derived by multiplying the semi-major and semi-minor axes of the 50% ellipse by an appropriate scaling factor. The two-dimensional Gaussian distribution of errors in latitude and longitude is based on the assumption that the random errors in sensor time and angle measurements are uncorrelated and their distributions are approximately Gaussian [Cummins et al., 1998]. Strokes located within a group of several sensors typically have relatively small nearly circular error ellipses, whereas strokes detected by only two or three sensors typically have very large, elongated ellipses. A stroke detected by only two sensors, when that stroke is located near the line joining the two sensors (base line), typically has an elongated ellipse whose major axis is along the base line.

[35] Figure 9 gives the NLDN 50% semi-major axis lengths plotted versus peak current, measured at Camp Blanding, for 70 strokes in 22 flashes. The majority of large (>4 km) semi-major axis lengths correspond to strokes with measured peak currents in the 5–10 kA range triggered during 2002 (although some large semi-major axis lengths are observed for larger peak currents), and is generally consistent with the observation that the majority of strokes with large (>2 km) location errors have peak currents in this range (see Figure 7).

Figure 9.

NLDN 50% error ellipse semi-major axis length versus Camp Blanding peak current, for 70 strokes with measured peak currents in 22 flashes triggered during 2001–2003.

[36] Figure 10a gives the NLDN absolute location error plotted versus NLDN 50% semi-major axis length, for 95 strokes in 31 flashes triggered during 2001–2003. Data for semi-major axis lengths and location errors less than 2 km (73 strokes total) are given in Figure 10b. Strokes with absolute location errors less than 2 km are typically associated with semi-major axis lengths between 0.4 and 1.3 km. As seen in Figure 10a, no semi-major axis lengths between 2 and 4 km were observed, although 20 strokes were found to have semi-major axis lengths between 4 and 8 km and one greater than 11 km, with most of those strokes having location errors greater than 2 km. The slanted solid line (slope = 1) on each plot is the locus of points for which the NLDN 50% semi-major axis length and corresponding location error are equal, that is, represents the boundary of the 50% error ellipse. If the error ellipses are assumed to be nearly circular, then data points below this line correspond to strokes with ground-truth locations enclosed by the 50% error ellipse and strokes above it are outside the 50% error ellipse. Using only the data shown in Figure 10b, where the assumption of nearly circular error ellipses is most-likely valid, it was found that about 63% (46 out of 73) of those strokes had ground-truth locations enclosed by the 50% error ellipse. Data points below the dashed line (slope = 1.82) on each plot of Figure 10 correspond to strokes with ground-truth locations enclosed by the 90% error ellipse (assumed to be nearly circular). It was found that about 96% (70 out of 73) of those strokes shown in Figure 10b had ground-truth locations enclosed by the 90% error ellipse. If this analysis is extended to all strokes, including those with large (>2 km) semi-major axes and location errors (Figure 10a), and the nearly circular error ellipse assumption is kept, then about 66% (63 out of 95) and 96% (91 out of 95) of stroke locations are enclosed by the 50% and 90% error ellipses, respectively. Hence the results are more or less the same regardless of whether strokes with large semi-major axes (for which the nearly circular ellipse assumption is likely to be invalid) are included, which may be due to the ellipse major axis orientation being consistent with the orientation of the line joining the NLDN-reported and ground-truth locations. These results suggest that the ellipse semi-major axis is a conservative estimate of the NLDN median location error, at least in north-central Florida. In order to perform a more detailed analysis, the actual shape and orientation of each individual error ellipse must be considered.

Figure 10.

(a) NLDN absolute location error plotted versus NLDN 50% error ellipse semi-major axis length, for 95 strokes in 31 flashes triggered during 2001–2003. (b) An expansion of Figure 10a that shows data only for location errors and semi-major axis lengths less than 2 km. The NLDN-calculated stroke location corresponds to the centroid of the error ellipse. The slanted solid line (slope = 1) is the locus of points for which the NLDN 50% semi-major axis length and corresponding location error are equal. If the error ellipses are assumed to be nearly circular, then points below this line correspond to strokes with ground-truth locations enclosed by the 50% error ellipse and strokes above are outside the 50% error ellipse. Points below the dashed line (slope = 1.82) correspond to strokes with ground-truth locations enclosed by the 90% (assumed to be nearly circular) error ellipse.

4.3. Peak Current Estimates

[37] Figure 11 shows the NLDN-estimated peak current plotted versus peak current measured directly at Camp Blanding, for 70 strokes in 22 flashes. There is a strong positive linear relationship between the measured and NLDN-estimated peak currents for all 3 years, which is stronger in 2002 and 2003 than in 2001. It is clear from Figure 11 that the NLDN tends to underestimate the actual peak current. For all data combined, a crude “correction factor” for the NLDN peak current estimates can be obtained by taking the mean of the ratio ICB/INLDN, which is about 1.2 for the combined data, and is presumably applicable to subsequent strokes in natural lightning. Note that this result might be valid only for the configuration and settings used in the NLDN in 2001–2003. No attempt is made in this paper to re-process the raw NLDN data.

Figure 11.

NLDN-reported peak current versus peak current directly measured at Camp Blanding for (a) 2001, (b) 2002, (c) 2003, and (d) 2001–2003. There were a total of 70 strokes with directly measured current in 22 flashes triggered during 2001–2003. The slanted line (slope = 1) is the locus of points for which the NLDN peak current is equal to the Camp Blanding peak current. For each data set, the sample size (n) is given.

[38] Histograms showing the distributions of Camp Blanding and NLDN peak currents, for 70 strokes in 22 flashes triggered during 2001–2003, are given in Figure 12. For all 3 years, the NLDN mean and median peak currents were lower than the corresponding Camp Blanding values, although the median values are very close for 2002 (15.5 kA Camp Blanding median versus 14.9 kA NLDN median). Note that the Camp Blanding arithmetic and geometric means presented in Figure 3 are smaller than those presented in Figure 12, since smaller strokes not detected by the NLDN are not included in the distributions of Figure 12. Note that both the overall Camp Blanding and NLDN histograms are indicative of a lognormal distribution, generally thought to be typical of lightning peak current distributions.

Figure 12.

Histograms of peak currents directly measured at Camp Blanding (black) and estimated by the NLDN (shaded) for (a) 2001, (b) 2002, (c) 2003, and (d) 2001–2003. There were a total of 70 strokes with directly measured currents in 22 flashes triggered during 2001–2003. All bin sizes are 5 kA (not 2.5 kA, as suggested by the histogram column width), with two histogram columns of half width being shown within each bin. Corresponding statistics are given below the histograms for both the Camp Blanding and NLDN data.

[39] Figure 13 gives histograms of the NLDN peak current estimation errors (defined here as ΔI = INLDNICB), both signed (Figure 13a) and absolute values (Figure 13b). Note that only negative events are considered here, since the positive stroke in the Camp Blanding data (see section 2) was not detected by the NLDN. The arithmetic mean values of ΔI are −6.0 kA, −1.3 kA, −2.6 kA, and −2.9 kA, for 2001, 2002, 2003, and 2001–2003, respectively. The corresponding median values are −8.2 kA, −2.1 kA, −2.9 kA, and −2.9 kA, respectively. If the absolute value of ΔI is considered (Figure 13b), the arithmetic mean values are 6.0 kA, 2.6 kA, 2.9 kA, and 3.4 kA, for 2001, 2002, 2003, and 2001–2003, respectively. The corresponding median values are 8.2 kA, 2.5 kA, 2.9 kA, and 2.9 kA, respectively. Note that the mean and median peak current estimation errors are somewhat lower for 2002 and 2003, compared to those for 2001. This observation is consistent with the plots of NLDN peak current versus Camp Blanding peak current shown in Figure 11.

Figure 13.

Histograms of NLDN peak current estimation errors (defined as ΔI = INLDNICB), for 70 negative strokes in 22 flashes triggered during 2001–2003. (a) Distribution for signed errors with a bin size of 2 kA and (b) distribution for the magnitudes of the errors (absolute values) with a bin size of 1 kA. Statistics given are arithmetic mean (AM), standard deviation (SD), geometric mean (GM), median, minimum, and maximum values.

[40] Figure 14 is similar to Figure 13, except that the NLDN peak current estimation error (ΔI) is now expressed as a percentage of the measured Camp Blanding peak current (ΔI% = 100ΔI/ICB). As with Figure 13, histograms are given for both signed (Figure 14a) and absolute (Figure 14b) errors. For all 3 years, the percentage errors never exceeded 50%, with the median signed errors being −25%, −11%, −22%, and −18%, for 2001, 2002, 2003, and 2001–2003, respectively. The arithmetic mean values were generally similar to the corresponding median values. For the unsigned (absolute) percentage errors, the median values were 25%, 17%, 23%, and 20%, for 2001, 2002, 2003, and 2001–2003, respectively. Hence it appears that the NLDN underestimated peak currents by about 20%.

Figure 14.

Histograms of NLDN peak current estimation errors (defined as ΔI = INLDNICB), given as a percentage of the directly measured Camp Blanding current (ΔI% = 100ΔI/ICB), for 70 negative strokes in 22 flashes triggered during 2001–2003. (a) Distribution for signed percentage errors and (b) distribution for the magnitudes of the percentage errors (absolute values). Both histograms have a bin size of 10%. Corresponding statistics are given below each histogram.

[41] The number of NLDN reporting sensors is plotted against Camp Blanding peak current in Figure 15. As expected, there is a positive correlation between measured peak current and the number of reporting sensors since larger peak currents should correspond to larger signal strengths at the NLDN sensor locations, and hence larger strokes can be detected by more-distant sensors. Interestingly, for most peak currents, the range of the number of NLDN reporting sensors is typically quite large. For example, for peak currents ranging from 20 to 30 kA, the number of reporting sensors ranges from 3 to 11.

Figure 15.

Number of NLDN reporting sensors versus Camp Blanding peak current, for 70 strokes with measured currents in 22 flashes triggered during 2001–2003.

5. Summary

[42] Data from rocket-triggered lightning, obtained in the summers of 2001–2003 at Camp Blanding, Florida, have been used to evaluate the performance characteristics of the NLDN. For the 3-year period, the following NLDN performance characteristics have been found: (1) a flash detection efficiency of about 84% (31 flashes detected out of 37 triggered at Camp Blanding); (2) a stroke detection efficiency of about 60% (95 strokes detected out of 159 recorded at Camp Blanding); (3) a median location error of about 600 m (larger location errors (>2 km) are observed for strokes having smaller peak currents (5–10 kA)); and (4) a median peak current estimation error of about −18% (the NLDN tends to underestimate peak current).

[43] Since first strokes in natural lightning typically have larger peak currents than subsequent strokes, and the triggered-lightning strokes studied here are similar to subsequent strokes in natural lightning, the observed values of flash (84%) and stroke (60%) detection efficiencies should probably be viewed as lower bounds. Further, since location error generally decreases with increasing peak current, the lack of typically larger first strokes can be interpreted to indicate that the observed median location error (600 m) is an upper bound. It has been observed that stroke detection efficiency systematically increased between 2001 and 2003, this trend being probably attributable to the 2002–2003 NLDN upgrade.

[44] The observations presented here are, strictly speaking, only applicable to negative subsequent strokes and not necessarily to negative first strokes or positive first or subsequent strokes in natural lightning (one positive stroke in our data set cannot influence this statement). Further, all of the strokes considered here have peak currents less than 50 kA, with the majority having peak currents below 30 kA, and hence the NLDN peak current error estimates presented here may not be applicable to strokes with larger peak currents.

[45] Finally, the results of this study are valid only for the configuration and settings used in the NLDN in 2001–2003. A new configuration and/or settings would generally require another ground-truth evaluation.

Acknowledgments

[46] This work was supported in part by NSF grants ATM-0003994 and ATM-0346164, U.S. DOT (FAA) grant 99-G-043, the Florida Power and Light Corporation, and by the Florida Space Grant Consortium. The authors would like to thank W. Schulz and G. Diendorfer for useful discussions of the results presented in this paper.

Ancillary