Can natural variability trigger effects on fish and fish habitat as defined in Environment Canada's metal mining environmental effects monitoring program?



The Metal Mining Effluent Regulations (MMER) took effect in 2002 and require most metal mining operations in Canada to complete environmental effects monitoring (EEM) programs. An “effect” under the MMER EEM program is considered any positive or negative statistically significant difference in fish population, fish usability, or benthic invertebrate community EEM-defined endpoints. Two consecutive studies with the same statistically significant differences trigger more intensive monitoring, including the characterization of extent and magnitude and investigation of cause. Standard EEM study designs do not require multiple reference areas or preexposure sampling, thus results and conclusions about mine effects are highly contingent on the selection of a near perfect reference area and are at risk of falsely labeling natural variation as mine related “effects.” A case study was completed to characterize the natural variability in EEM-defined endpoints during preexposure or baseline conditions. This involved completing a typical EEM study in future reference and exposure lakes surrounding a proposed uranium (U) mine in northern Saskatchewan, Canada. Moon Lake was sampled as the future exposure area as it is currently proposed to receive effluent from the U mine. Two reference areas were used: Slush Lake for both the fish population and benthic invertebrate community surveys and Lake C as a second reference area for the benthic invertebrate community survey. Moon Lake, Slush Lake, and Lake C are located in the same drainage basin in close proximity to one another. All 3 lakes contained similar water quality, fish communities, aquatic habitat, and a sediment composition largely comprised of fine-textured particles. The fish population survey consisted of a nonlethal northern pike (Esox lucius) and a lethal yellow perch (Perca flavescens) survey. A comparison of the 5 benthic invertebrate community effect endpoints, 4 nonlethal northern pike population effect endpoints, and 10 lethal yellow perch effect endpoints resulted in the observation of several statistically significant differences at the future exposure area relative to the reference area and/or areas. When the data from 2 reference areas assessed for the benthic invertebrate community survey were pooled, no significant differences in effect endpoints were observed. These results demonstrate weaknesses in the definition of an “effect” used by the MMER EEM program and in the use of a single reference area. Determination of the ecological significance of statistical differences identified as part of EEM programs conducted during the operational period should consider preexisting (background) natural variability between reference and exposure areas. Integr Environ Assess Manag 2013; 9: 155–163. © 2012 SETAC


Environmental effects monitoring (EEM) programs are a requirement for most metal mining operations in Canada under the 2002 Metal Mining Effluent Regulations (MMER) of the Fisheries Act. The purpose of metal mining EEM program biological monitoring studies is to determine if mine effluent is having an effect on fish, fish habitat (as measured with benthic invertebrate indices), or the use of fisheries resources (Environment Canada 2002, 2012). The metal mining EEM guidance document prescribes study design frameworks and sampling methodologies to achieve consistency of the approach for all aquatic EEM studies across the nation (Environment Canada 2002, 2012). The approach is effects-based and evaluates a specified set of biological indicators (e.g., fish condition, benthic invertebrate taxon richness) mostly through comparisons of areas exposed to mine effluent and reference areas that are not exposed to effluent. The EEM program is tiered, such that the results from 1 phase of monitoring will, in part, determine the study design for monitoring during the next phase. More intensive monitoring will be prescribed for areas in which effects were found during 2 consecutive studies, with the objective of determining the magnitude, geographical extent, and cause of the effect in subsequent phases. Conversely, when no effect is detected, a less intensive monitoring design or longer time period between programs will be prescribed for the next phase.

The definition of an effect used by the EEM program is “a statistical difference between data collected in an exposure area and in a reference area or sampling areas within an exposure area where there are gradually decreasing effluent concentrations at increasing distances from the effluent discharge” (Environment Canada 2012). Although the metal mining EEM program has recently introduced the use of critical effect sizes (CES) to establish thresholds above which effects may be indicative of a higher risk to the environment, the definition of an “effect” has not changed (Environment Canada 2012). This means that statistically significant differences in EEM-defined effect endpoints, regardless of the direction or magnitude of the difference, can cause mines to move to more intensive sampling programs.

The EEM program has received multiple criticisms, some of which include the use of a single reference area, statistical issues, and influences of natural variability (Munkittrick et al. 2010; Huebert et al. 2011). Because the first phase of EEM does not require monitoring exposure and reference areas before impacts are initiated, natural variation in biological communities between lakes could result in statistical differences that initially are labeled as “effects.” The objective of this article is to provide a case study of an EEM biological monitoring program conducted using a traditional metal mining EEM study design and methods prescribed in the metal mining EEM technical guidance document on lakes that have no prior exposure to mine effluent. At the time of the study, the mine did not exist and no effluent had been released into the proposed exposure area, allowing for an evaluation of natural variation between exposure and reference lakes in site characteristics, benthic invertebrate communities, and fish populations.

The study was completed in 2007 at the Millennium Project, which is a proposed underground U mine that will discharge treated mine water effluent subject to the MMER and hence EEM. All ore processing is proposed to be done off-site at a nearby U mill, thus no mill or tailings facilities are proposed for the Millennium site. At the time of this study, Cameco Corporation, the operator of the Millennium Project, was in the stage of completing environmental baseline studies to inform the environmental impact assessment.

This case study demonstrates weaknesses in the EEM program definition of an “effect” and in the use of a single reference area. The study illustrates the importance of documenting natural variability in the study area while the exposure area is unimpacted so that this information can be used when evaluating the ecological significance and applying CES to statistical results of EEM programs conducted during the operational period.


Study area and sampling design

The Millennium Project is located in northern Saskatchewan approximately 35 km north of Cameco's Key Lake Operation (Figure 1). Moon Lake will receive treated liquid effluent from the proposed mine and therefore was the future exposure lake in this study. Of all accessible lakes in the region, Slush Lake and Lake C were determined to be the best potential candidates for reference lakes for EEM, based on morphometry, sediment characteristics, fish habitat, and fish community structure. Moon Lake, Slush Lake, and Lake C are located within the same drainage basin and are within close proximity to each other. At the time of this study no effluent had been released in Moon Lake.

Figure 1.

Location of EEM study areas at the Millennium Project.

For the benthic invertebrate community survey, a multiple control-impact study design was used. In each of Moon Lake, Slush Lake, and Lake C, one sampling area with 5 replicate stations was sampled in September 2007. In Moon Lake, the large bay at the inflow of Slush Creek was the proposed effluent release area and thus contained the EEM sampling area (Figure 1). The designated EEM sampling areas in Slush Lake and Lake C were chosen based on their similarity to the Moon Lake sampling area in terms of depth and sediment quality, as determined from intensive sampling of the lakes in 2006 (CanNorth 2010). In each sampling area, the 5 10-m2 replicate stations were spaced a minimum of 20 m apart, as recommended in the EEM guidance document (Environment Canada 2002) to allow for statistical independence of samples. The use of 5 replicates is prescribed based on the premise of a sample size of 5 yielding statistical power of 0.90 for detecting a difference of 2 standard deviations when using α = β = 0.10 in all statistical analyses of the data (Environment Canada 2002, 2012).

For the fish population survey, a control-impact study design was used. The future exposure area consisted of the bay in Moon Lake that will potentially receive treated effluent (Figure 1). The reference area consisted of sections of Slush Lake that contained similar habitat characteristics to the future exposure area in Moon Lake. As per EEM guidelines, 2 sentinel fish species were used for the fish population survey (Environment Canada 2002, 2012). Reconnaissance fishing was first conducted in the future exposure area of Moon Lake to determine which sentinel species to use for the survey. Based on the abundance of juvenile northern pike (Esox lucius) and adult yellow perch (Perca flavescens) captured in the bay, it was decided to conduct a nonlethal northern pike survey and a lethal yellow perch survey.

Site characterization

Site characterization information was required to determine the best candidate and/or candidates for a reference lake and/or lakes, to identify similar sampling areas within the future exposure and reference lakes, and to assess whether potential confounding factors were present that may need to be taken into consideration when interpreting results.

Lake morphometry

Bathymetric mapping of Moon Lake, Slush Lake, and Lake C was completed in the summer of 2006 using a GPS MAP 178 depth sounder mounted on a Zodiac portable boat. Lake depth and GPS locations were recorded simultaneously at regular intervals along lake transects. Bathymetric maps were produced using AutoCAD® and Surfer software.

Sediment quality

In September 2007, sediment samples were collected at 5 stations in each sampling area that were co-incident with the benthic invertebrate sampling locations. Sediment samples were collected using a Tech-ops corer. The Tech-ops corer consists of a 10 cm diameter Lexan polycarbonate tubing and a head-piece. The tube is attached to the head-piece that contains a valve that creates a vacuum in the tube. To collect a sample, the corer was lowered, by means of a line, into the sediment by hand to sediment penetration of 40 to 50 cm and then retrieved. Sediment subsamples were collected from the 0 to 2 cm horizon. To obtain sufficient material for analysis, subsamples from 3 or 4 retrieved cores were composited into 1 sample. The cores were collected adjacent to one another and composited cores were placed in a single sample bag and mixed to ensure homogeneity of the sample. All samples were frozen as soon as possible after collection. Composite samples containing the 0 to 2 cm sediment horizon were submitted to the Saskatchewan Research Council Analytical Laboratory (Saskatoon, SK) for particle size analyses and determination of total organic C (SRC 2011a, SRC 2011b). The particle size breakdown followed the Wentworth categorization recommended in the metal mining EEM program guidance document (Environment Canada 2002).

A 1-way analysis of variance (ANOVA) was performed to compare the sediment data between Moon Lake, Lake C, and Slush Lake. When the ANOVAs indicated significant differences, Tukey's post hoc tests were performed to elucidate where differences occurred. If sediment data departed from ANOVA assumptions, they were log or square root transformed before analysis. An α of 0.10 was used to be consistent with the benthic invertebrate and fish data analyses.

Additional information

Seasonal limnology profiles, seasonal water chemistry samples, sediment chemistry sampling (co-incident and colocated with the benthic invertebrate sampling), macrophyte chemistry, fish community sampling, and aquatic habitat characterization were also completed on the future exposure and reference lakes as part of the baseline studies (CanNorth 2010). These data collections provide comprehensive site characterization information for the study areas but are too extensive to present in this article.

Benthic invertebrate community survey

Data collection

The benthic invertebrate community is a standard EEM indicator of the condition of fish habitat (Environment Canada 2002, 2012), and several EEM effect endpoints are derived from a benthic invertebrate survey. At each of the 5 established sampling stations in each study area, 3 subsamples collected using an Ekman dredge (0.052-m2 sampling area) were composited to form a single sample. Sediment samples were rinsed in the field through a 500-µm nitex mesh bag by swirling the bag in the lake water to release fine sediments. The material retained by the bag was placed in a sample jar and preserved in 10% buffered formalin. Preserved benthic invertebrate samples were sorted and keyed according to the latest methods and taxonomic keys by a qualified taxonomist.

Data analyses

Statistical differences in the benthic invertebrate communities were determined by evaluating 4 endpoints defined in the EEM guidance document as indicative of effects on fish habitat (Environment Canada 2002, 2012). The endpoints are total benthic invertebrate density (individuals per unit area sampled), taxon richness (number of families), Simpson's evenness (calculated at the taxonomic level of family), and the Bray–Curtis similarity index (based on family abundance). Two Bray–Curtis indices were calculated based on the median count of each family from each of Slush Lake and Lake C. Additionally, Bray–Curtis indices were calculated from pooling the 2 reference areas to calculate 1 reference median.

The statistical methods used to test differences in the benthic invertebrate community endpoints between reference and exposure areas were the same as those described above for sediment particle size and total organic C content. Because multiple reference areas were sampled, data were analyzed using separate reference area data and by pooling the data from both reference areas.

Fish surveys

Data collection

The populations of 2 sentinel species common to the future exposure (Moon Lake) and reference (Slush Lake) study areas were assessed. A boat electrofisher was used to conduct the sampling in September 2007, which consisted of a modified Zodiac Grand Raid MK II (4.2 m) equipped with a Smith-Root 2.5 GPP Portable Generator Pulsator Electrofisher. All fish were collected under the authority of a Special Collection Permit issued by the Saskatchewan Ministry of Environment, and every effort was made to minimize incidental fish mortality.

Two types of surveys were carried out: nonlethal for northern pike and lethal for yellow perch. Because this was a new site with no existing variability data to use in power analyses to estimate sample size requirements, the sample sizes recommended in the EEM guidance document were used. To calculate nonlethal fish survey EEM endpoints, a minimum of 100 individuals across a range of sizes was required from each lake (Environment Canada 2005); a total of 110 northern pike from Moon Lake and 107 from Slush Lake were caught and processed. All fish were retained in a holding net until the end of the study to ensure there were no recaptures. For the yellow perch lethal survey, the target sample size was 20 adult females and 20 adult males from each lake (Environment Canada 2002). A total of 35 females and 21 males were sampled from Moon Lake, and 20 females and 23 males were sampled from Slush Lake.

All sampled northern pike and yellow perch individuals were measured (fork length) to the nearest millimeter and total body mass was measured to 0.01 g accuracy using an Ainsworth XP-300A scale. In addition, yellow perch were processed for several other measurements. The liver and gonads were dissected from each individual, and the mass of each was measured with a Scientech ZSP 150 scale (0.0015 g accuracy).

All yellow perch were aged and a subsample of northern pike were sacrificed for aging to assist in identifying young-of-year (YOY) northern pike because some EEM nonlethal effect endpoints are specific to YOY (Gray et al. 2002). For aging, otoliths were harvested from each yellow perch and the cleithra was removed from 10 northern pike from Moon Lake and 6 northern pike from Slush Lake. The aging structures were read by at least 2 independent technicians at North/South Consultants (Winnipeg, MB), and then at least 10% of each sample was reviewed.

Data analyses

All data analyses for the northern pike nonlethal survey and yellow perch lethal survey were as prescribed by the EEM guidelines available at the time of the survey (Environment Canada 2002, 2005). For parametric analyses (t tests and ANCOVA), data were square root or log10 transformed as necessary to meet assumptions of the analyses. A Mann–Whitney U test was used when t test data could not be normalized. All analyses of fish data determined statistical significance by comparing p with α = 0.10.

Northern Pike

Environmental effects monitoring effect endpoints for the northern pike nonlethal survey included body mass and length of YOY, length-frequency distribution, and condition. Age data acquired from sacrificed individuals (Moon Lake, n = 10; Slush Lake, n = 6), in combination with length data, were used to establish length cohorts defining YOY and older individuals. The length cohort defining YOY were less than 19.3 cm (Slush Lake, n = 82) and less than 14.5 cm (Moon Lake, n = 47).

A 2-sample Kolmogorov–Smirnov (K–S) test was used to determine whether the length-frequency distributions differed between Moon Lake and Slush Lake. Northern pike YOY body mass and length were compared between lakes with 2-sample t tests. The final northern pike nonlethal endpoint was condition, or the relationship between body mass and length, and this relationship was compared between lakes with the use of ANCOVA.

Yellow Perch

All lethal endpoints were evaluated separately for males and females, and endpoints dealing with body mass used adjusted body mass, which was body mass without liver, gonads, and visibly obvious parasites such as intestinal tapeworms. Fish were included in the analyses only if their gonad mass was greater than 1% of their adjusted body mass, so that immature fish were not mixed with mature fish for analyses (Environment Canada 2002). In Moon Lake, additional female fish (n = 35) were sampled to obtain a sample size of 20 male fish. To balance the sample sizes of females from the 2 lakes and to replicate a typical EEM study design, only the first 20 female yellow perch captured from Moon Lake with a gonad size of greater than 1% of adjusted body mass were included in the statistical analyses. A total of 21 males from Moon Lake and 23 males from Slush Lake were included in the analyses.

There were 5 EEM effect endpoints calculated from the yellow perch lethal survey. The endpoint that assessed survival was age, and the endpoints that assessed energy use included adjusted body mass relative to age and gonad mass relative to adjusted body mass. Two indicators of energy storage, adjusted body mass relative to body length and liver mass relative to adjusted body mass, were the final effect endpoints. All of the effect endpoints, except age, were assessed with ANCOVA to compare slopes and elevations of the relationships between Moon Lake and Slush Lake. Age was compared between lakes with the use of a Mann–Whitney U test after assumptions of a parametric 2-sample t test could not be satisfied.


Site characterization

Moon Lake, Slush Lake, and Lake C are located in very close proximity to each other and are in regions with similar climate, geology, topography, soil, and vegetation (CanNorth 2010). Studies completed as part of a comprehensive baseline program demonstrated that these lakes have comparable fish communities and fish habitat. All 3 lakes have complex shorelines, similar types of substrate and macrophyte communities, and contain abundant northern pike and yellow perch spawning habitat. Yellow perch had the highest catch per unit electrofishing effort of large-bodied fish in all 3 waterbodies and test netting demonstrated abundant populations of lake whitefish (Coregonus clupeaformis), northern pike, walleye (Sander vitreus), and white sucker (Catostomus commersoni) (CanNorth 2010).

Currently, the only anthropogenic influences in the study area are exploration activities located between Slush Lake and Moon Lake and an exploration camp located near the southwest tip of Slush Lake. The exploration camp does not discharge into Slush Lake.

Moon Lake has the largest catchment area, surface area, and volume of the 3 study lakes, as well as the greatest depth (Table 1). Despite differences in lake morphometry, Slush Lake and Lake C are the most similar to Moon Lake out of all lakes in the vicinity of the proposed Millennium mine site.

Table 1. Morphometry of Moon Lake, Slush Lake, and Lake C
Lake/site characteristicMoon LakeSlush LakeLake C
Mean depth (m)5.584.24.33
Surface area (km2)6.322.740.92
Volume (×106 m3)35.3111.523.98
Catchment area (km2)200.5552.935.08
Shore length (km)41.5722.168.88
Shoreline development index4.663.782.61
Volume development index1.141.521.37

Sampling areas in Moon Lake, Slush Lake, and Lake C were located at depths of 5 to 6 m and all contained predominantly fine-textured sediments. The 1-way ANOVA indicated significant differences in clay, silt, fine sand, coarse sand, and organic C between Moon Lake, Slush Lake, and Lake C (Table 2). Organic C content was similar in Moon Lake and Slush Lake and was significantly higher in Lake C. Moon Lake and Lake C sediments were both dominated by silt, but they differed significantly in silt and fine sand content. The study area in Slush Lake contained significantly more silt and less clay than the study area in Moon Lake; however, both were comprised of approximately 70% fine-textured sediments.

Table 2. Statistical analysis results of sediment particle size and TOC data (mean ± SD) collected in Moon Lake, Slush Lake, and Lake C in 2007
EndpointpMean ± SDa (% composition)
Moon LakeSlush LakeLake C
  1. SD = standard deviation; TOC = total organic carbon.

  2. a,b,cMeans with the same superscript are not significantly different from each other, based on Tukey's post hoc tests and α = 0.10.

Clay0.0000315.4 ± 2.74a32.9 ± 6.27b17.7 ± 1.51a
Silt0.000151.1 ± 7.95a38.4 ± 7.63b65.7 ± 2.71c
Fine sand0.00632.0 ± 9.18a24.8 ± 10.40a15.5 ± 2.72b
Coarse sand0.0571.5 ± 0.92a,b3.9 ± 3.15b1.1 ± 0.39a
Organic carbon<0.000019.1 ± 2.46a7.1 ± 1.68a19.8 ± 2.49b

Benthic invertebrate community survey

There were no significant differences in benthic invertebrate community endpoints between Moon Lake and Lake C; however, numerous significant differences were identified between Moon Lake and Slush Lake (Table 3). The study area in Slush Lake had significantly higher richness, significantly lower Simpson's evenness, and significantly different Bray–Curtis indices than the study area in Moon Lake. The differences between the 2 reference areas, Slush Lake and Lake C, were similar except that density also differed between the lakes. However, when the data from the 2 reference areas were combined, there were no significant differences in the EEM effect endpoints between the exposure and reference areas (Table 3).

Table 3. Statistical analysis results of benthic invertebrate community data (mean ± SD) collected in Moon Lake, Slush Lake, and Lake C in 2007
EndpointpaSlush LakeLake CMoon LakeSlush + Lake CpbMagnitude differencec
  • LC = Lake C; ML = Moon Lake; SD = standard deviation; SL = Slush Lake.

  • a

    Comparison of Moon Lake, Slush Lake, and Lake C. Means with the same superscript are not significantly different from each other, based on Tukey's post hoc tests and α = 0.10.

  • b

    Comparison of Moon Lake to the pooled reference areas.

  • c

    Difference in the first lake of each pair relative to the second lake (e.g., ML-SL: (ML mean − SL mean)/SL SD).

Density (organisms/m2)0.0301445 ± 559b526 ± 406a646 ± 158a,b985 ± 6680.821−1.6
Family richness (# taxa)0.0027.6 ± 1.5b4.0 ± 1.2a5.0 ± 0.7a5.8 ± 2.30.737−1.7−2.4
Evenness0.0030.21 ± 0.03b0.39 ± 0.13a0.28 ± 0.04a0.32 ± 0.100.9352.36.0
Slush Lake reference0.0170.16 ± 0.12b0.54 ± 0.27a0.37 ± 0.10a1.83.2
Lake C reference0.0480.59 ± 0.11b0.36 ± 0.26a,b0.28 ± 0.13a−2.8
Pooled reference0.24 ± 0.090.33  ± 0.250.813

The magnitude difference in benthic invertebrate endpoints was calculated for endpoints that were significantly different between study areas (Table 3). The significant differences between Moon Lake and Slush Lake ranged between magnitudes of 1.7 and 2.8 reference area standard deviations, whereas the significant differences between the references areas ranged between magnitudes of 1.6 and 6.0 reference area standard deviations.

Northern pike nonlethal survey

All EEM effect endpoints assessed for the northern pike nonlethal survey demonstrated significant differences between Moon and Slush lakes (Table 4). The length–frequency distributions were statistically different, with Slush Lake individuals, on average, being longer than individuals from Moon Lake (Figure 2). Energy use was assessed by analyzing the effect endpoints of YOY body length and body weight. The YOY northern pike from Slush Lake (n = 82) were significantly longer and heavier than the YOY northern pike from Moon Lake (n = 47). An ANCOVA was used to compare condition of all age classes combined between Moon Lake and Slush Lake and the slopes of the relationship significantly differed between the 2 lakes.

Table 4. Summary of fish population survey EEM effect endpoints for which significant differences were found between Moon Lake and Slush Lake in 2007
SurveyEffect endpoint (units)pValuesaMagnitude differenceb
Moon LakeSlush Lake
  • EEM = environmental effects monitoring; SD = standard deviation; YOY = young of the year.

  • a

    Mean ± SD, adjusted means (for ANCOVA when intercepts were different), slopes, and medians are based on transformed values when data transformations were carried out.

  • b

    Magnitude difference (%) = (Moon − Slush)/(Slush × 100); calculated from untransformed mean values, except for slopes.

NonlethalLength frequency distribution<0.0001
Northern pikeBody length: YOY (cm)<0.000112.7 ± 1.016.4 ± 1.3−22.3
 Body mass: YOY (g)<0.00013.44 ± 0.415.10 ± 0.69−54.5
 Condition0.0943.08 ± 0.023.02 ± 0.032.0
LethalAdjusted body mass vs age: male (g, yr)0.0391.217 ± 0.0251.143 ± 0.0246.5
Yellow perchGonad mass vs adjusted body mass: female (g, g)0.00010.282 ± 0.0090.336 ± 0.009−16.1
 Gonad mass vs adjusted body mass: male (g, g)0.0071.479 ± 0.1290.965 ± 0.12253.3
 Adjusted body mass vs length: female (g, cm)0.0231.473 ± 0.0091.444 ± 0.0092.0
 Liver mass vs adjusted body mass: female (g, g)0.0050.515 ± 0.0190.597 ± 0.019−13.7
Figure 2.

Length-frequency and condition of northern pike for the EEM nonlethal fish population survey in Moon and Slush lakes, September 2007.

The magnitude difference in northern pike endpoints was calculated for all endpoints that were significantly different between lakes and the absolute values of these ranged from 2.0% to 54.5% (Table 4).

Yellow perch lethal survey

The following EEM effect endpoints were significantly different between Moon Lake and Slush Lake: adjusted body mass versus age (male), gonad mass versus adjusted body mass (female and male), adjusted body mass versus length (female), and liver mass versus adjusted body mass (female) (Table 4 and Figure 3). The slopes of the relationship differed between the 2 lakes for male gonad mass versus adjusted body mass. In the other 4 cases, the slopes were equal (p > 0.10) (thus endpoints were not different), but the intercepts differed between lakes. Male adjusted body mass at age and female adjusted body mass at length were significantly higher in Moon Lake than Slush Lake, whereas female gonad mass at adjusted body mass and female liver mass at adjusted body mass were significantly lower in Moon Lake than Slush Lake.

Figure 3.

Significantly different effect endpoints from the lethal yellow perch population survey, September 2007: (a) male adjusted body mass versus age; (b) male gonad mass versus adjusted body mass; (c) female gonad mass versus adjusted body mass; (d) female adjusted body mass versus length; and (e) female liver mass versus adjusted body mass.

The magnitude difference in yellow perch endpoints was calculated for all endpoints that were significantly different between lakes (Table 4); the absolute values of these differences ranged between 2.0% and 53.3%.


According to MMER EEM guidelines, any statistically significant difference between effect endpoints from exposure and reference locations triggers the need for more monitoring to 1) confirm the effect, 2) determine the magnitude and extent of the effect, or 3) determine the cause of the effect (Dumaresq et al. 2002; Environment Canada 2002, 2012). Criticism of the MMER EEM guidance has occurred on numerous occasions with respect to the use of a single reference area and the use of a statistically significant difference, regardless of the magnitude of the difference, as a significant effect (Dubé 2003; Kilgour et al. 2007; Kilgour and CanNorth 2010; Huebert et al. 2011). In this case study of a preexposure lake (Moon Lake) and 2 reference lakes (Lake C and Slush Lake), analyses of benthic invertebrate and fish effect endpoints indicated the presence of significant differences before any exposure had occurred. These results show that metal mining EEM-defined effects (i.e., statistically significant differences) can occur naturally in the environment between unimpacted areas.

Locating suitable reference areas for EEM studies can be very challenging, particularly in remote settings with limited access. The reference areas selected for Moon Lake were considered suitable when the comprehensive suite of site characterization information was evaluated (CanNorth 2010). All 3 lakes contained similar water quality, fish communities, aquatic habitat, and a sediment composition largely comprised of fine-textured particles. Although some significant differences in terms of sediment particle size and organic C composition were found, the extent of the differences does not corroborate with the benthic invertebrate community statistical results. The particle size composition and organic C content of the sediment was more similar between Moon Lake and Slush Lake than between Moon Lake and Lake C; however, there were numerous significant differences in the benthic invertebrate community metrics between Moon and Slush lakes and no significant differences between Moon Lake and Lake C. Therefore, the differences observed in the benthic invertebrate community metrics cannot be solely attributed to site characteristic differences between the study areas. As indicated by Dubé (2003) and Kilgour et al. (2007), knowledge of preexposure variability between study areas is important for determining and understanding postexposure differences between areas; if possible, preexposure sampling should be a part of the formal MMER EEM sampling protocol.

The presence of high natural variability between areas in this empirical case study corroborates Huebert et al.'s (2011) study design-based conclusion that replication of reference areas is essential for understanding the actual impacts of a metal mining operation. When the 2 reference areas were pooled for the benthic invertebrate data, Moon Lake no longer significantly differed from reference endpoints, indicating the value of replicating reference areas to better account for natural variability among areas.

The value of attaining a good estimate of natural variability in the EEM effect endpoints was highlighted in Kilgour and CanNorth (2010). Benthic invertebrate community data from Aline Lake, which is part of the Cigar Lake Project in northern Saskatchewan, was used as an example assessment for a Reference Condition Approach (RCA). Under the traditional EEM study design for the benthic invertebrate community program, the Aline Lake benthic invertebrate community differed from the 2 site-specific reference communities (Mad Dog Lake and Lake B). Using a RCA model derived from 21 reference lakes in northern Saskatchewan, the benthic invertebrate community from Aline Lake was found to be within the range of reference area variability for the region (Kilgour and CanNorth 2010).

As part of the 2012 metal mining EEM guidance document released recently, Environment Canada has listed CES for key effect endpoints (Environment Canada 2012). The CES for most fish endpoints are a 25% difference between the exposure and reference means, with the exception of condition that has a CES of 10% (Environment Canada 2012). Reevaluation of the fish population results from this case study excluding significant differences with magnitudes less than 25% reduced the number of effects. However, some effects between preexposure areas still occurred, with magnitude differences up to 54.5% for the fish population studies; 1 effect endpoint for each of northern pike (YOY body mass) and yellow perch (male gonad mass vs adjusted body mass). Similarly, the magnitude of difference of the benthic invertebrate evenness index between Moon Lake and the Slush Lake reference area exceeded the CES of ±2 reference area standard deviations. Therefore, even if the EEM program used CES to define effects, natural variability needs to be factored in as the magnitude of differences observed in an unimpacted environment could exceed CES.

Huebert et al. (2011) and Huebert (2012) argued that the correct determination of significant effects would require Bonferroni adjustment of α to prevent compounding Type 1 error. This is based on the concept that the EEM guidelines state an effect has occurred if there is a significant difference between an exposure area and a reference area. With multiple endpoints that could each lead to the declaration of a mining effect, Type 1 error would compound to 1–0.9n. In the current case study, there were 5 benthic invertebrate endpoints, 4 nonlethal fish survey endpoints, and 10 lethal fish survey endpoints, or at least n = 19, which would give a total Type 1 error of 86%. Bonferroni adjustment of α = 0.1 to account for 19 comparisons yields a new α of 0.005. Although this does reduce the number of significant effects observed, 2 benthic invertebrate endpoints, 3 nonlethal northern pike effect endpoints, and 1 lethal yellow perch endpoint are still significant between the reference and future exposure areas.

In conclusion, this case study of MMER EEM effect endpoints resulted in the observation of several significant effects on the benthic invertebrate community and 2 fish populations (northern pike and yellow perch) in a preexposure area relative to a reference area. The results demonstrate weaknesses in the current MMER EEM program, which considers statistically significant differences between reference and exposure areas as effects.

This study provided empirical evidence that supports the idea that multiple reference areas are required to understand the natural variability in a system. The collection of detailed site characterization information and, if possible, the completion of preexposure EEM studies will provide valuable data that should be used when conducting EEM studies during the operational period. Baseline information needs to be considered when evaluating the ecological significance and applying CES to statistical results during future EEM programs.


This case study was completed with the assistance of a dedicated field staff, the Slush Lake Camp, and the Key Lake Operation. Benthic invertebrate identification was done by Prof. Jack Zloty (Emeritus).