ANALYTICAL & STATISTICAL METHODS TM

Florida Department of Environmental Protection TMDL PROGRAM TECHNICAL SUPPORT SERVICES (YEAR 3) Contract No. WM861 Task Assignment No. 004.03/06-001...
Author: Eleanor Rich
3 downloads 0 Views 571KB Size
Florida Department of Environmental Protection

TMDL PROGRAM TECHNICAL SUPPORT SERVICES (YEAR 3) Contract No. WM861

Task Assignment No. 004.03/06-001

FLORIDA BMP DATABASE

ANALYTICAL & STATISTICAL METHODS TM MARCH 2006 Prepared for the

CENTER FOR ENVIRONMENTAL STUDIES Florida Atlantic University Prepared by:

URS Corporation 7650 West Courtney Campbell Causeway Tampa, Florida

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

State of Florida Total Maximum Daily Load Program BMP Analytical and Statistical Methods TM For the Florida Department of Environmental Protection

The BMP Analytical and Statistical Methods Technical Memo presents a discussion of analytical methods and statistical analyses, and recommends specific approaches for use in assessing the performance of BMPs in conjunction with the Florida BMP Database. This document has been produced as a Task Assignment under the Florida Department of Environmental Protection contract with the Florida Atlantic University Center for Environmental Studies (CES) under FAU Research Subagreement #CRE14, which is authorized by FDEP Contract No. WM861. The reproduction of this document was the responsibility of CES. The Technical Elements of this document were the primary responsibility of URS Corporation as a subcontractor to CES in collaboration with Camp Dresser & McKee, Inc. (CDM).

v v v

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

1.

March 2006 Technical Memo BMP Analytical and Statistical Methods

Background

The objective of Task Assignment 004.03/06-001 was to finalize the development of the Florida BMP Database, incorporate modifications to the BMP performance assessments and conduct an analysis of BMP performance. Previous database development efforts provided for the solicitation, acquisition, organization and data entry of the available BMP monitoring data that had been collected in Florida. This was followed by the development of BMP characteristic data tables and event-based monitoring data tables to serve as the basis for the evaluation of BMP performance characteristics in support of FDEP’s ongoing TMDL modeling activities. At that time, a standardized statistical method was developed for the assessment of BMP performance. With this updated TM, that approach has been updated to include a matched-pairs analysis to help account for the effects of correlation between inflow and outflow concentrations.

2.

Analytical & Statistical Methods

The analysis of monitoring data for the purpose of estimating the effectiveness of BMPs in reducing pollutant loads is a far more complex task than it would at first appear. Historically, a number of different methods and approaches have been used, all based on different and varying assumptions and yielding significantly different results. This has been presented by others (Strecker 1998, EPA 1999 and Strecker 2001) during the development of the NSW BMP Database. Strecker (2001) made several relevant observations regarding estimating efficiency based on loads using a storm-by-storm basis: ·

assumes that all storms are equal (generally untrue)

·

wet ponds and wetlands often have discharges which have no relationship to the inflow for the same event

·

storm-to-storm comparisons are probably not valid

2.1 Common Analytical Methods Over the history of BMP studies, a number of different analytical methods have been used to estimate the efficiency of a BMP. Four of the most commonly used methods include: ·

Efficiency Ratio (ER)

·

Summation of Loads (SL)

·

Regression of Loads (RL)

·

Mean Concentration (MC)

·

Efficiency of Individual Storm Loads (ISL)

None of these common methods assess the monitoring data statistically and do not examine the differences in inflow and outflow water quality for statistical significance. The Effluent Page 1

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

Probability (EP) method selected for use in the NSW BMP Database is an expansion of the efficiency ratio (ER) method and describes the statistical distributions of the upstream and downstream water quality to determine if there are statistically significant differences.

2.1.1

Efficiency Ratio (ER)

The efficiency ratio (ER) is defined in terms of the average event mean concentration (EMC) of a parameter from the inflow and outflow monitoring stations. It is computed as:

ER =

( Average Inlet EMC - Average Outlet EMC ) Average Inlet EMC

The ER method provides equal weighting to all storms, but the use of log transforms tends to minimize differences between EMCs and mass balance calculations. It is most applicable when loads are proportional to storm volume. Since inflow EMCs generally do not correlate with storm volume, the use of average EMC values for the inflow should be representative. This may not necessarily be true for outflow EMCs, and the accuracy of this method is expected to vary depending on the type of BMP.

2.1.2

Summation of Loads (SL)

The summation of loads (SL) is defined in terms of the ratio of all outgoing loads to all incoming loads. Load (L) is computed for each event as:

L = EMC * Volume The SL ratio is computed based on the sum of the loads as:

SL =

(å L( ) - å L( å L( ) inlet

outlet

)

)

inlet

The SL method is based on mass balance and is generally dominated by a few large storms in the monitoring data. For an accurate evaluation, it is important that sample collection be over a sufficiently long period so that the effects of temporary storage or export of pollutants are minimized. This is of particular concern for BMP systems that may have long residence times.

2.1.3

Regression of Loads (RL)

The regression of loads (RL) is computed using least-squares linear regression of the inlet and outlet loads and constraining the y-intercept to zero. This yields a relationship where the outlet loads are defined as a multiple (β) of the inlet loads, where b is the slope of the regression line. The efficiency of the BMP is then computed as: Page 2

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

RL = 1 - b Like the SL method, the RL method is based on mass balance and is often dominated by a few large storms in the monitoring data. Again, for an accurate evaluation it is important that sample collection be over a sufficiently long period so that the effects of temporary storage or export of pollutants are minimized. The RL method assumes that the outlet loads are a linear function of the inlet loads (generally observed to have poor correlation) and that treatment efficiency is the same for all events (often untrue).

2.1.4

Mean Concentration (MC)

The mean concentration (MC) method is identical to the ER method, but does not require that the event sample data be flow weighted (EMC). This permits grab samples and other non-flow weighted samples to be utilized in defining the efficiency of the BMP. Since sample data that is not flow weighted is applicable to this method, the MC method may introduce significant bias due to different sampling protocols and programs that could be lumped together for assessment.

2.1.5

Efficiency of Individual Storm Loads (ISL)

The efficiency of individual storm loads (ISL) computes the removal efficiency for each monitored event based on the inlet and outlet loads. These individual storm efficiencies (SE) are computed as:

SE

=

(L( inlet ) - L( outlet ) ) L( inlet )

The ISL efficiency is computed as the average of the SE values:

ISL =

å SE n

The ISL method assigns equal weighting to all storms. Large storms don’t dominate the efficiency, which is an average performance regardless of storm size. Both an inflow and outflow value must be available for each event included in the analysis. This excludes any data points that do not have a corresponding measurement at the inflow/outflow for a particular storm. It should be noted that a storm-by-storm analysis makes no provision for events in which the outflow being measured has little or no relationship to the inflow. This often occurs for smaller events in BMPs that have a permanent pool.

Page 3

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

2.2 Recommended Analytical Methods The performance of BMPs should be evaluated in terms of both concentration reductions (EMCs) and mass load reductions. The approach taken in the development of the NSW BMP Database was to use statistical characterizations of the inflow and outflow concentrations to evaluate the effectiveness of the BMPs. It has been suggested that if enough sample data were available to represent the population of storms expected annually, the same approach can be taken utilizing loads (rather than concentrations) into and out of the BMP. This approach is known as the Effluent Probability (EP) method (EPA 2002). One shortcoming of the EP method is that by assuming the inflow and outflow data represent two independent distributions, any correlation between inflow and outflow EMCs is ignored and becomes part of the variance and uncertainty in the outflow distribution. Two approaches were considered to deal with this; the first was to attempt to adjust the variance of the outflow EMCs to account for any observed correlation, the second was to evaluate the EMC reduction on an individual event basis. The second approach was considered to be more direct, with fewer assumptions required to implement, and was adopted for this study.

2.2.1

EMC Assessment

The Effluent Probability (EP) method is an expansion of the efficiency ratio (ER) method and describes the statistical distributions of the upstream and downstream water quality to determine if there are statistically significant differences, as well as confidence limits about the expected average values. In addition, if enough flow and EMC data are available, then the EP method can also be used to enhance the SL method in a similar fashion. Based on previous research (as well as generally accepted stormwater practices), the use of log-transform EMCs is recommended. The EP method adopted for this study provides standard descriptive statistics, box & whisker charts, normal probability charts and timeseries plots of the transformed data for both the inflow and outflow data. This information is used to demonstrate the differences in the mean EMCs, including confidence intervals, as well as the performance of the BMP throughout the range of influent and effluent EMCs. Since the EP method does not account for any correlation effects that may be present in the data, a second analytical method recommended for the evaluation of BMP performance is the efficiency individual storm loads (using EMCs), with a slight modification to fit a transformed log-normal distribution to the event efficiency data series. This method was called the individual storm efficiency (ISE) method and included estimates of the average and 95% confidence intervals to illustrate the variation in load reductions and the uncertainty in a BMPs performance. One of the shortcomings of a statistical characterization is the need for data quantity. Stormwater runoff concentrations have been observed to possess notoriously high variations. The greater the variation of a population, the larger the sample needed to accurately characterize the statistics of that population. Sample sets of less than ten data points often have such large confidence intervals that any reduction in concentration provided by the BMP remains ‘hidden’ within the data variation. This results in a finding of Page 4

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

“no significant difference” between the inflow and outflow mean values, when in reality the BMP may provide significant removal.

2.2.2

Mass Load Assessment

Ideally, the performance of a BMP would be expressed in terms of mass load reductions. However, mass loads can be surprisingly difficult to accurately characterize. There are essentially two basic BMP types that need to be dealt with. For a substantial number of BMP types (such as detention systems, wetlands and media filters) the volume of flow leaving the BMP is essentially the same as that entering the BMP. Losses in volume are not a significant part of the performance of these BMPs in reducing pollutant loads. As a result, their mass load reductions are the same as their EMC reductions and do not require a separate analysis. Other types of BMPs (such as retention and infiltration systems) depend heavily on volume reduction as a significant part of the BMPs performance in reducing pollutant loads. In some cases there may be no significant EMC reduction, or even increases in concentrations of flows passing through the BMP. For these BMPs, both EMC reduction and the reduced volume of discharge can be important factors in the performance of the BMP in reducing mass loads. Unfortunately, monitoring of flow volumes in BMP studies can be very challenging, since accurately measuring flow rates is often difficult. In addition, trying to track all significant sources and volumes can be almost impossible in some cases. The measured data from some BMP studies occasionally revealed more outflow than inflow, presumably due to groundwater contributions! This tends to make a consistent approach to assessing mass reduction performance very difficult to implement. In addition to the challenges presented in collecting accurate flow data from all significant sources, it is also highly unlikely that any set of monitored flows and volumes would be representative of average annual conditions, since event size, frequency and inter-event time are all highly variable factors with significant influence on this data. As a result, it is recommended that a better approach for mass load assessment might be to develop a longterm water balance model of the BMP to estimate the expected annual volume reduction. In addition, if there is a statistically significant difference in the measured inflow and outflow EMCs, a total BMP mass efficiency could be determined using the modeled volumes and the EMC statistics.

3.

BMP Performance Assessment

In this study, the performance of the BMPs within the Florida BMP Database was limited to EMC values, due to the difficulties in evaluating mass loads (see Section 2.2.2). The assessment of the EMC performance included two statistical approaches, one of which characterizes the inflow and outflow characteristics as if they were two independent distributions. Although some BMP/parameter combinations are expected to display an effluent characteristic that is generally independent of inflow concentrations, others are expected to exhibit effluent concentrations that are highly dependent upon inflow concentrations. Therefore a second statistical assessment which could account for

Page 5

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

correlation was applied by using paired inflow and outflow data points to represent the observed EMC performance for each measured event.

3.1 Selection of Parameters for Assessment The Florida BMP Database supports a wide range of analytes and the current Database includes data values for nearly 80 different parameters (see the BMP & Event Characteristics TM). It should be noted that the data for many of these analytes is very scarce, and they are not commonly evaluated in the context of BMP performance. Table 1 lists the parameters proposed for performance evaluation of the BMPs in the Florida BMP Database. Table 1: Parameters Selected for BMP Assessment STORET Code 530 600 605 610 625 630 665 671 1027 1042 1051 1092

STORET Parameter Name Residue, Total Nonfiltrable (mg/l) Nitrogen, Total (mg/l as N) Nitrogen, Organic, Total (mg/l as N) Nitrogen, Ammonia, Total (mg/l as N) Nitrogen, Kjeldahl, Total, (mg/l as N) Nitrite Plus Nitrate, Total 1 Det. (mg/l as N) Phosphorus, Total (mg/l as P) Phosphorus, Dissolved Orthophosphate (mg/l as P) Cadmium, Total (µg/l as Cd) Copper, Total (µg/l as Cu) Lead, Total (µg/l as Pb) Zinc, Total (µg/l as Zn)

Parameter ID TSS TN Org-N NH3+NH4-N TKN NOx or TON TP Ortho-P Cd, Total Cu, Total Pb, Total Zn, Total

3.2 Performance Calculations The performance of a particular BMP and parameter was expressed in terms of the average expected performance, along with an upper and lower 95% confidence limit to demonstrate the level of uncertainty in the statistical assessments. Given the large variance usually found in stormwater quality, these confidence limits can define a fairly wide range over which the average expected performance can vary. As a result, these confidence intervals are very important to consider when evaluating the performance of a BMP.

3.2.1

BDL Values

Analytical tests used by laboratories to establish the concentration of a parameter in a sample are limited in their ability to identify very low concentrations. The method detection limit (MDL) is the minimum concentration that can be measured and reported with 99% Page 6

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

confidence that it is greater than zero (but not necessarily quantify it – more on that below). The MDL is determined from the preparation and analysis of a sample in a given matrix containing the parameter and can vary with different samples and testing methods. When an analytical result is below the MDL, it is generally identified as a “below detection limit” (BDL) value, which distinguishes the result as uncertain, but below the MDL. Some laboratories report the actual analytical result (which does not meet the 99% confidence criteria), while others will only report that the test was below the MDL. The BMP studies and reports that formed the basis of the data entered into the BMP Database showed a variety of approaches in handling and reporting BDL values. These included: · Substituting the result with zero values · Substituting the result with the MDL value · Substituting the result with ½ the MDL value · Exclusion of BDL results None of these typical approaches is ideal and the greater the fraction of the data set represented by BDL results, the more impact they have on the resulting parametric statistics. The approach adopted for this study is known as a “robust method”, which utilizes the data above the MDL to estimate values that fall below the MDL assuming a log-normal distribution for the data series. Because the estimated values cannot be assigned to specific BDL samples, this approach can only be used for summary statistics and cannot be used for the paired-data (ISE) analysis. The robust method was applied by ranking the data and computing a least-squares regression through the log-transformed values which were above the MDL and their associated standard normal deviates. The least-squares regression was then used to extrapolate replacement values for the BDL results based on their ranked position and these replacement values were then substituted for the BDL values when computing distributional statistics. Because the assignment of replacement values would sometime cause the ranking order of the entire dataset to change, the robust method iterated until this no longer occurred (up to five iterations). It should be noted that the MDL value is the minimum concentration at which the analytical test can reliably detect the presence of the parameter, but not necessarily quantify it with any accuracy. The practical quantitation limit (PQL) is the lowest level of measurement that can be reliably achieved during routine laboratory operating conditions within specified limits of precision and accuracy. The FDEP has adopted a general guideline that if a laboratory fails to report a PQL, the PQL is estimated at four times the MDL. Due to a general lack of reported MDL values within the database (and no PQL data), the effects of the PQL were not addressed in this study.

3.2.2

Outlier Values

The statistical assessments conducted in this study generally apply the assumption of normally distributed data, albeit using transformed data. Such assessments can be strongly influenced by the presence of outlier values in the data sets. The presence of outlier values is not necessarily indicative of a bad data set, though errors in the data will frequently Page 7

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

appear as outliers. An outlier may be a valid data point, but is due to factors that are rare occurrences and not expected to be found in a data set of so few values. If the number of outliers identified in the data set is significant (greater than 2% to 5% of the data set population), then the assumption of a single normal distribution may not be valid and the resulting statistical analyses should be considered suspect. Outlier values were identified by assessing each data point against the computed distribution parameters for the log-transformed data set. If a log-transformed value fell above or below the expected range for that distribution it was considered an outlier. The range of expected values was defined by upper and lower fence values which are based on the hinges of the distribution (first and third quartiles) and the spread between them (interquartile range). The first (25% or Q1) and third (75% or Q3) quartile values of the distribution were computed based on a normal distribution of the log-transformed data. The difference between these two values is called the inter-quartile range (IQR). A “k” value of 1.5 was applied to the IQR to determine the fence values as follows:

Upper Fence = Q3 + k * (Q3 - Q1) Lower Fence = Q1 - k * (Q3 - Q1) A data point was considered an outlier if the transformed value was greater than the upper fence or below the lower fence. If a data set contained outliers, the parametric statistics were recomputed with the outlier values removed and the process repeated until no additional outliers were identified.

3.2.3

EMC Analysis – Effluent Probability (EP)

The Effluent Probability (EP) method was recommended in Section 2 due to its previous acceptance by ASCE, its general robust application, and the ability to quantify the level of uncertainty in the expected results. This method compares the statistical distributions (in this case, log-normal) of two different data series, the inflow and outflow EMCs for a particular BMP. The data for each monitoring station were log-transformed and ranked for fitting of a lognormal distribution by computing the mean or average (U) and standard deviation (W) of the transformed values. Note that the robust method was used to develop substitute values for any BDL results. These parametric statistics were then used to estimate the following parameters, based on a normal distribution of the log transformed values: · the estimated 25%, 50% and 75% quartile values (Q1, Q2 and Q3) · the inter-quartile range (IQR) value (75% - 25% quartile values) · the upper fence value (75% quartile plus 1.5 times the IQR) · the lower fence value (25% quartile minus 1.5 times the IQR) The data set was evaluated for outlier values and adjusted until all outliers were identified and removed from the data set as described in Section 3.2.2. The value used to represent the average value for subsequent performance analysis was the arithmetic estimate of the mean value (M) for the outlier adjusted data set. Based on a log-normal distribution, where

Page 8

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

U is the average of the log transformed values and W is the standard deviation of the log transformed values, M was computed as:

é W2ù M = exp êU + ú 2 û ë The modified Cox method, presented by Olsson, was used to estimate the 95% confidence interval (CI) about the estimated arithmetic mean values, based on the use of a log-normal distribution. The Cox method uses the following to estimate the upper and lower 95% confidence limit, where n is the number of data values in the series and t is the Students-t value at the 95% level (for a given n):

é W2 W2 W4 ù CL = exp êU + ±t + ú 2 n 2(n - 1) úû êë The ANOVA (Analysis of Variance) procedure using the log-transformed data was performed to test the mean values of the two monitoring stations for significant differences. The ANOVA analysis produces a P value (probability) that the data sets representing the two stations do not have a statistically different mean value. In this study a certainty level of 95% was adopted; thus a P value of 5% or less would indicate that there appeared to be a statistically significant difference in the mean values of the two stations. The average performance of the BMP was computed as the difference between the M values of the two distributions, divided by the M value of the inflow distribution. The uncertainty in the computed average performance was estimated using the upper (UCL) and lower (LCL) 95% confidence limits of the two distributions. These calculations were made as follows:

Average Performance =

Upper CL =

Lower CL =

3.2.4

M inf low - M outflow M inf low

UCLinf low - LCLoutflow UCLinf low LCLinf low - UCLoutflow LCLinf low

EMC Analysis – Individual Storm Efficiency (ISE)

The EP approach assumes that the two distributions represented by the inflow and outflow EMC data are independent of each other. Correlations between inflow and outflow concentrations result in increased deviation and uncertainty in the outflow distribution. While this effect could be addressed by removing any correlative effects between the two data series, a more direct approach was developed using the paired data sets to compute the Page 9

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

EMC reduction for each data pair (individual storm efficiency or ISE) and the resulting efficiency data set to characterize the performance of the BMP. The ISE data set has an upper bound of 1 (can approach 100% removal efficiency) and a lower bound of negative infinity (a virtually unlimited increase in EMC values is possible). The ISE data would also be expected to be skewed to the right. A transform was applied to the data so that it could be represented by a normal distribution as follows: · A value of 1, or 100% was subtracted from each efficiency value. This shifts the distribution to the left and creates an upper bound of 0. · The shifted values were multiplied by (-1) to invert the distribution about the Y-axis. The transformed values now have a lower bound of 0 and an upper bound of infinity, with a left skew, the same as log-normally distributed data. · The shifted and inverted values were then log-transformed to fit a normal distribution. The transformed efficiency data set was evaluated for outlier values and adjusted until all outliers were identified and removed from the data set as described in Section 3.2.2. The value used to represent the average efficiency was based on the arithmetic estimate of the mean value (M) for the outlier adjusted transformed efficiency data set. Based on a lognormal distribution, M was computed as:

é W2ù M = exp êU + ú 2 û ë The uncertainty in the computed average performance was estimated using the upper (UCL) and lower (LCL) 95% confidence limits of the transformed efficiency distribution using Cox’s method. These calculations were made as follows:

é W2 W2 W4 ù Upper CL = exp êU + +t + ú 2 n 2(n - 1) úû êë é W2 W2 W4 ù Lower CL = exp êU + -t + ú 2 n 2(n - 1) úû êë The robust BDL method cannot be applied to event specific data, so efficiency computations for paired events where one of the values was BDL were based on a value of ½ the MDL. This can dramatically affect the computed distribution of the efficiency values, so two assessments were conducted using the ISE approach. The first assessment included individual storm events with a BDL result at one of the two monitoring stations (but not both); the other excluded all events in which either station had a BDL result. Any event in which both the inflow and outflow values were both BDL were dropped from the data series completely.

Page 10

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

3.2.5

March 2006 Technical Memo BMP Analytical and Statistical Methods

Spreadsheet Analysis Tool

The tremendous volume of data in the Florida BMP Database can quickly become overwhelming when attempting to conduct the assessment of BMP performance as described above. In addition, databases are not easily adapted to perform many calculations, particularly between independent records. A specialized spreadsheet was developed early in this project to assist with data entry to the database and this tool was expanded to include the retrieval and assessment of BMP performance data for this study. The application of both the EP and ISE methods to the EMC data in the database was implemented into an Excel spreadsheet, “FL BMP Database.xls”. The spreadsheet uses a series of VBA macros to perform the statistical calculations based on user supplied input. A screen capture of the spreadsheet EMC analysis tool showing the user selectable options is included in Figure 1. Initially there are ten steps to complete (see Figure 1), but subsequent assessments can be performed with modifications to as little as one of the selection items.

Page 11

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

Figure 1: EMC Analysis Tool

A brief summary of the operation of the FL BMP Database Analysis Tool follows: ·

The user must identify where the Database is stored (Step 1) and where the output files should be stored (Step 8). These are both initialization steps and the values can be saved in the spreadsheet. They do not need to be updated for subsequent analyses unless the location of the Database is changed (or a different Database is desired), or the location of the output files needs to be changed.

·

The user must then “refresh” the spreadsheet with data from the Database (Step 2). When this button is clicked, the Database specified in Step 1 is queried to create a Page 12

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

list of the BMP Test Sites and Monitoring Stations. This step needs to be done each time the FL BMP Database.xls file is opened since this data are not stored in the spreadsheet file. However, it does not need to be repeated between analyses unless the FL BMP Database.xls file was closed. ·

Once the data lists have been updated to the spreadsheet, the user can select the BMP Test Site from the pull down list in Step 3. At this point the pull down list immediately below the Test Site will show the available images for the BMP. If desired, the user can click the “Show Image” button to view the selected image, which generally shows the BMP layout and the location of the various Monitoring Stations. Clicking the dotted button to the right of the image list displays the full comment associated with the image.

·

Following the selection of a BMP Test Site, the Inflow and Outflow Monitoring Station lists (Steps 4 and 5) will be automatically filtered to show only the Monitoring Stations for the selected Test Site. If a pull down list is blank, then either the spreadsheet needs to be refreshed (Step 2), or no data exists in the Database for the current Test Site. Both Monitoring Station lists are identical and it is incumbent upon the user to select the appropriate inflow and outflow stations.

·

Following the identification of the two Monitoring Stations, the desired parameter for analysis can be selected (Step 6) using a similarly filtered pull-down list.

·

After the selection of monitoring stations and parameter, the user can apply additional filters so that only specific types of water quality records will be examined (Step 7). Available filters include flow type, sample media type, sample type and exclusion of BDL values. The current inventory for this data is shown next to each of the filter selections to help guide the user. If the Auto Update box is selected, these values update each time a selection from one of the pull-down lists is changed. If the box is not selected, the “Update Sample Counts” button will need to be clicked before the inventory counts will update (which may be preferable if the database is being accessed over a local area network rather than locally on the user’s computer). o

Flow Types: The user can select Runoff or Baseflow to limit the selected water quality data to samples which have these types of flow data associated with them. If the user wants to select water quality for samples even if no flow data exist, the Not Specified box should be selected. Unless mass calculations will be conducted, the Not Specified box should usually be selected.

o

Sample Media Types: The user can filter the data request based on various media types by selecting the appropriate boxes. For a typical EMC analysis, usually only surface runoff/flow records are desirable.

o

Sample Types: The user can apply these filters to select data limited to specific types of samples, such as composite, grab, etc.

·

Each time the Water Quality Parameter is reselected (Step 6), the database is queried to determine the range of available dates for water quality data collected at the selected inflow and outflow monitoring sites. The user can modify these dates to select a subset of the data by date for statistical analysis.

·

The name of the output file is created automatically by the analysis tool, but can be modified by the user if desired (Step 9). Page 13

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

·

March 2006 Technical Memo BMP Analytical and Statistical Methods

Finally, clicking on the “Acquire & Analyze” button (Step 10) will activate the analysis macros and create the output file, which is left open for the user to examine once the macro successfully terminates.

The actions performed by the macro are outlined as follows: · The Database is queried for the data using the selections and filters as set by the user. A copy of the resulting data is pulled into the output spreadsheet file. · Substitute values for BDL results are computed for the inflow and outflow data sets using the robust method described in Section 3.2.1. · Outlier values in both the inflow and outflow data sets are identified and the data sets adjusted as described in Section 3.2.2. · An EMC analysis is performed using the EP method as described in Section 3.2.3. An example of the results is shown in Table 2. · An EMC analysis is performed using the ISE method as described in Section 3.2.4. An example of the results is shown in Table 3.

3.3 Interpretation of Analytical Output Once the FL BMP Database Analysis Tool has completed the statistical computations, it produces a spreadsheet output file with a summary statistics tab, three data tabs and seven analytical charts: · Summary statistics tab · Performance comparison chart · Box & whisker chart · Probability chart · Time series chart · Efficiency series chart · Efficiency probability chart · Correlation series chart · Inflow data tab · Outflow data tab · Paired data tab The ability to successfully use statistical characterizations to evaluate BMP performance is highly dependent on the amount of data as compared to the variance in that data. Stormwater EMCs can generally be classified as having large variations, and will thus require fairly large data sets for characterization. The inspection of the graphic charts included in the output file can often help reveal insights into the performance of a BMP with statistically inadequate datasets, or to confirm the results shown in the summary statistics tab.

3.3.1

Summary Statistics Tab

The summary statistics tab lists the Test Site, Inflow and Outflow Monitoring Stations, the assessment parameter, and selected query filters. It summarizes the distributional statistics of the log-transformed inflow and outflow data. The summary data shown in Tables 2 and 3 Page 14

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

(above) were taken from the Summary stats tab. Finally, several plotting support data tables are included for the Box & whisker chart and the Probability chart. Interpretation of the output file should begin with a review of the data on the Summary stats tab. The example output in Table 2 shows that the ANOVA test was passed, but more importantly the P Value was very small (< 0.000), indicating a distinct difference between the two means. Table 2: Example EMC Assessment (EP Method) Summary Statistics Site: Inflow MS: Outflow MS: Parameter:

FDEP-64 (Florida Aquarium Test Site) FDEP-64-01 (Inflow) FDEP-64-02 (Outflow) CU-T (COPPER, TOTAL (UG/L AS CU))

Statistics for Log-transformed Data:

LN (Inflow)

LN (Outflow)

Sum

155.877

52.072

207.949 551.977

ANOVA

Statistic Data Count

Inflow 54

Outflow 41

Sum^2

467.012

84.965

# BDL

1

7

Sum Sq.

17.055

18.832

# Outliers

0

1

Overall Mean

2.189

All

Mean

2.887

1.270

Source

Std Dev

0.567

0.686

Between groups

SS 60.904

df 1

MS 60.904

Minimum

1.386

-0.369

Within groups

35.887

93

0.386

Maximum

4.022

2.451

Total

96.791

Q10

2.160

0.391

F

157.831

Q25

2.504

0.807

P

0.000

Q75

3.269

1.733

Q90

3.614

2.149

IQR

0.765

0.926

PASS: MEANS ARE DIFFERENT

Removal Efficiency (Distribution Probability) Inflow

Outflow

95% t

2.006

2.021

Arithmetic Est. of Mean:

Std Error

0.077

0.107

Modified Cox 95% UCL:

24.894

5.737

95% CI

0.155

0.217

Modified Cox 95% LCL:

17.821

3.539

Lower CL

2.732

1.053

Average Difference:

78.6%

Upper CL

3.041

1.487

Max. Diff. (Inflow UCL - Outflow LCL):

85.8%

Max. Diff. (Inflow LCL - Outflow UCL):

67.8%

Lower Fence

1.356

-0.581

Upper Fence

4.417

3.121

21.063

4.506

Review of the example results shown in Table 2 (EP Method) finds the difference between the average EMC values of the two distributions indicates a reduction of 78.6%, with a 95% confidence interval of [67.8%, 85.8%]. It should be noted that due to the use of a logtransformed distribution, the confidence interval is not symmetrical about the mean.

Page 15

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

Table 3: Example EMC Assessment (ISE Method) Paired Data Summary Statistics

Data Count

Stat Value (Incl BDLs) 41

Stat Value (Excl BDLs) 34

# Outliers

1

1

Average

74.6%

71.3%

Std. Dev.

80.7%

81.3%

CV

24.1%

35.1%

Cox Upper 95% CL

80.0%

77.1%

Cox Lower 95% CL

67.7%

63.9%

Inflow/Outflow Correlation

-.025

-.072

Review of the example results shown in Table 3 (ISE Method) finds the average reduction of the EMC based on an individual storm assessment and excluding BDL events was 71.3%,with a 95% confidence interval of [63.9%, 77.1%].

3.3.2

Performance Comparison Chart

The performance comparison chart, shown in Figure 2, provides a visual summary of the expected EMC reductions based on both methods, the effluent probability (EP) method and the individual storm efficiency (ISE) method. In this particular example, both methods yield very similar results, ranging from approximately 70 to 80% average reduction estimate.

Page 16

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

Figure 2: Performance Comparison Chart

3.3.3

Box & Whisker Chart

The box & whisker chart, shown in Figure 3, provides a graphic representation of the distribution of the two EMC data sets and Figure 4 presents the interpretive elements and nomenclature of a box and whisker chart. Examining the results shown in Figure 3 reveals the distinctive differences in the two distributions. The two distributions appear to be very similar in shape, with the outflow distribution shifted downwards. The presence of an outlier value (>100µg/l) in the outflow distribution should be noted by the user.

Page 17

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

Figure 3: EMC Box & Whisker Chart

Page 18

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

Figure 4: Box & Whisker Nomenclature

10 Upper Fence

Event Mean Concentration (mg/l)

3rd Quartile

Upper 95% Confidence Limit Geometric Mean Lower 95% Confidence Limit 1 1st Quartile

Lower Fence Outlier

0.1

Page 19

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

3.3.4

March 2006 Technical Memo BMP Analytical and Statistical Methods

Probability Chart

The probability chart is shown in Figure 5. This chart plots the EMC values for both distributions against their computed deviation from their respective means and is another visual presentation of the EP method. The strongly linear plot supports the assumption of a log-normal distribution. This is further supported by the clustering of both data sets within 2 standard deviations of the mean. It should be noted that this chart excludes outlier values as no value greater than 60µg/l is shown. Figure 5: EMC Probability Chart

Page 20

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

3.3.5

March 2006 Technical Memo BMP Analytical and Statistical Methods

Time Series Chart

The time series chart is shown in Figure 6. This chart plots the EMC values for both monitoring stations against their respective dates. Although this is not a “statistical” chart, it can yield insights not otherwise provided by a statistical evaluation. For example, Figure 6 shows a gap in the monitoring data of more than six months. It should be noted that this chart also excludes outlier values. Figure 6: EMC Time Series Chart

Page 21

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

3.3.6

March 2006 Technical Memo BMP Analytical and Statistical Methods

Efficiency Chart

The efficiency chart is shown in Figure 7. This chart presents the computed individual storm efficiencies (ISE) over the period of record. Two ISE data series are shown, one includes BDL results, and the other excludes them. The example shown exhibits a strong clustering around the 80% reduction level. This chart excludes outlier values; though in this case outliers are based on the computed efficiency series (see the Paired Data tab of the output spreadsheet). The observed outflow EMC value 100+µg/l yielded a computed removal efficiency of -618% which was identified as an outlier value in the efficiency data series. Figure 7: Efficiency Chart

Page 22

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

3.3.7

March 2006 Technical Memo BMP Analytical and Statistical Methods

Efficiency Probability Chart

The efficiency probability chart is shown in Figure 8. This chart presents the ranked probabilities for both ISE series along with the expected values based on the transformed log-normal distribution described in Section 3.2.4. Again, the outlier value of -618% was excluded from this chart. Figure 8: Efficiency Probability Chart

Page 23

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

3.3.8

March 2006 Technical Memo BMP Analytical and Statistical Methods

Correlation Chart

The correlation chart is shown in Figure 9. This chart presents the inflow and outflow data for the individual storm events, along with a regression line and the regression coefficient (R2). The R2 value is a measure of how much the inflow EMC value affects the outflow EMC value and varies from 0 to 1, with a perfect correlation between the two yielding an R2 value of 1. The example in Figure 8 has an R2 value of 0.0006, which indicates that the variation in the inflow accounts for virtually none of the observed variation in the outflow. Figure 9: Correlation Chart

Page 24

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

However, it should be noted that one value appears to be an outlier, with an outflow EMC of over 100 µg/l. Unlike the previous charts and calculations, the EMC Analysis Tool does not currently adjust the correlation data sets to account for outliers! Fortunately, the Paired Data tab in the output spreadsheet file contains all the data and the outlier can be deleted from the chart, resulting in the revised correlation shown in Figure 10. The revised R2 value of 0.1043 indicates more than 10% of the variation in the outflow EMC is accounted for by the inflow EMC. It can also be seen that the values are positively correlated, thus the revised R value is 0.323 (computed using Excel’s CORREL function) as opposed to the value of 0.025 shown at the bottom of Table 3. Figure 10: Revised Correlation Chart

Page 25

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

3.3.9

March 2006 Technical Memo BMP Analytical and Statistical Methods

Inflow/Outflow Data Tabs

The inflow and outflow data used in the analyses are copied into their associated data tabs in the output spreadsheet. These tabs allow the user to confirm, or modify the analyses and charts, or to perform additional assessments of the data. At the top of each tab are a few brief tables providing information about the selected monitoring stations and parameter, summary statistics and distributional estimates. An example is shown in Table 4. Table 4: Inflow/Outflow Data Tab – Summary Tables Inflow Data Site ID: FDEP-64 Mon Sta ID: FDEP-64-01 Parameter: CU-T Summary Statistics LN Value (Value) Total Count 54 Data Count 54 BDL Count 1 # Outliers 0 Average 20.829 2.887 Median 18.550 2.920 Std. Dev. 11.507 0.567 CV 0.552 0.197

Arith. Est.

21.063 17.932 12.977 0.616

Distribution Estimates P Value 0.100 0.250 0.500 0.750 0.900 IQR

N Dist 6.082 13.068 20.829 28.590 35.575 15.522

LN Dist 8.668 12.231 17.932 26.291 37.099 14.060

Confidence Limits Trans 2.160 2.504 2.887 3.269 3.614 0.765

95% t CI Std Error Conf. Inter. Cox LCL Cox UCL

LN 2.007 0.155 0.077 0.155 2.880 3.215

EXP

17.821 24.894

Following these summary tables are the data records extracted from the database and used in the analysis. Interim calculation values are also included such as the substitute value for BDL results (see Section 3.2.1), the log transformed values, computed plotting probability and standard normal deviate. Outlier EMC values are relocated below the remaining data records. An example is shown in Table 5.

Page 26

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

Table 5: Inflow/Outflow Data Tab – Data Tables Sorted Data

Sample 11 12 15 21

Date-time 07/13/01 00:00 07/21/01 00:00 08/05/01 00:00 07/01/02 00:00

Database Value 1.000 1.000 1.000 1.150

Detection Limit 2.000 2.000 2.000 2.300

Is BDL? Yes Yes Yes Yes

Robust Value 0.691 0.904 1.077 1.231

LN (Robust Value) -0.369 -0.101 0.074 0.208

Plot Prob. 0.023 0.047 0.070 0.093

Std. Norm. Dev. -1.991 -1.680 -1.478 -1.322

Flow Type R R R R

Sample Media 1 1 1 1

Sample Type 1 1 1 1

R R R R

1 1 1 1

1 1 1 1

Flow Type R

Sample Media 1

Sample Type 1

(remaining rows not shown for brevity) 20 31 30 35

06/29/02 00:00 12/12/02 00:00 12/09/02 00:00 04/25/03 00:00

2.300 9.500 11.300 11.600

1.000 1.000 1.000 1.000

No No No No

2.300 9.500 11.300 11.600

0.833 2.251 2.425 2.451

0.209 0.907 0.930 0.953

-0.809 1.322 1.478 1.680

LN(Robust Value) 4.625

Plot Prob.

Std. Norm. Dev.

Outliers

Sample 40

Date-time 07/11/03 00:00

Database Value 102.000

Detection Limit 1.000

Is BDL? No

Robust Value 102.000

The Flow Type, Sample Media and Sample Type are shown using their code values from the BMP Database. Under Flow Type, there are three possible entries: R for surface runoff, B for baseflow, or blank (which indicates there is no flow data associated with that sample). Sample Media and Sample Type are coded based on the following: Code 0 1 2 3 4 5 6 7 8

Sample Media Groundwater Surface Runoff/Flow Soil Dry Atmospheric Fallout Wet Atmospheric Fallout Pond/Lake Water Accumulated Bottom Sediment Biological Other

3.3.10

Sample Type Flow weighted composite EMC Time weighted composite EMC Un-weighted composite EMC Grab Sample Sediment Sample Biological Sample Continuous meter Manual meter

Paired Data Tabs

The paired inflow and outflow data used in the ISE analyses are copied into the Paired Data tab in the output spreadsheet. This tab allows the user to confirm, or modify the analyses and charts, or to perform additional assessments of the data. At the top of this tab are a few brief tables providing information about the monitoring stations, parameter and summary statistics. An example is shown in Table 6.

Page 27

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

Table 6: Paired Data Tab – Summary Tables Paired Data Inflow MSID: Outflow MSID: Parameter:

FDEP-64-01 FDEP-64-02 CU-T

Paired Data Summary Statistics

Data Count # Outliers Average Std. Dev. CV Cox UCL Cox LCL I/O Correl

Stat Value (Incl BDLs) 41 1 74.6% 80.7% 24.1% 80.0% 67.7% -0.025

Stat Value (Excl BDLs) 34 1 71.3% 81.3% 35.1% 77.1% 63.9% -0.072

Following these summary tables are the data records extracted from the database and used in the analysis. Interim calculation values are also included such as the substitute value for BDL results (½ the MDL, since robust methods cannot be used), the computed storm efficiencies and the transformed efficiency values (see Section 3.2.4). An example of this data table is shown in Table 7. Table 7: Inflow/Outflow Data Tab – Data Tables

Date-time 12/18/00 00:00 01/08/01 00:00 03/04/01 00:00

Inflow Data Database Value MDL 40.20 1.0 12.90 1.0 8.90 1.0

Is BDL? No No No

Value Used 40.20 12.90 8.90

Date-time 12/18/00 00:00 01/08/01 00:00 03/04/01 00:00

Outflow Data Database Value MDL 7.40 1.0 5.50 1.0 9.20 1.0

Is BDL? No No No

Value Used 7.40 5.50 9.20

No Yes No

102.00 1.50 7.35

Removal Efficiency Trans Eff. Eff 81.6% 0.184 57.4% 0.426 -3.4% 1.034

(remaining rows not shown for brevity) 07/11/03 00:00 08/27/03 00:00 09/19/03 00:00

14.20 18.20 38.70

1.0 1.0 1.0

No No No

14.20 18.20 38.70

07/11/03 00:00 08/27/03 00:00 09/19/03 00:00

102.00 1.50 7.35

1.0 3.0 1.0

-618.3% 91.8% 81.0%

7.183 0.082 0.190

Page 28

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

Following the data tables are the sorted and transformed efficiency tables. These tables include the computed efficiency (transformed), the computed plotting probability and standard normal deviate along with the expected efficiency based on a log-normal distribution of the transformed efficiency values (see Section 3.2.4). There are two of these tables, one which includes the BDL values while the other excludes them. Any outlier efficiency values (again based on a log-normal distribution of the transformed efficiency values) are moved below these data tables. An example is shown in Table 8. Table 8: Inflow/Outflow Data Tab – Efficiency Probability Tables

Sample 10 15 11 23

Sorted Transformed Data (Including BDLs) LN (Trans. Plot Trans. Eff.) Prob. Date-time Eff. Eff. 07/13/01 00:00 94.0% 0.060 -2.821 0.024 08/07/01 00:00 93.4% 0.066 -2.715 0.049 07/21/01 00:00 93.2% 0.068 -2.681 0.073 08/17/02 00:00 92.6% 0.074 -2.601 0.098

Std. Norm. Dev. -1.971 -1.657 -1.453 -1.296

Expected Eff. 94.6% 93.4% 92.4% 91.5%

1.296 1.453 1.657 1.971

51.4% 46.0% 38.0% 23.4%

(remaining rows not shown for brevity) 29 17 18 3

Sample 39

12/09/02 00:00 06/24/02 00:00 06/28/02 00:00 03/04/01 00:00

Date-time 07/11/03 00:00

36.5% 29.4% 18.5% -3.4%

0.635 0.706 0.815 1.034

-0.454 -0.348 -0.205 0.033

Outliers (Including BDLs) LN (Trans. Trans. Eff.) Eff. Eff. -618.3% 7.183 1.972

0.902 0.927 0.951 0.976

Plot Prob.

Std. Norm. Dev.

Expected Eff.

Page 29

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

4.

March 2006 Technical Memo BMP Analytical and Statistical Methods

Recommendations

Ideally, the performance of a BMP would be expressed in terms of mass load reductions. However, mass loads can be surprisingly difficult to accurately characterize and it is extremely highly unlikely that any set of monitored flows and volumes would be representative of average annual conditions, since event size, frequency and inter-event time are all highly variable factors with significant influence on this data. As a result, it is recommended that a better approach for mass load assessment might be to develop a longterm water balance model of the BMP to estimate the expected annual volume reduction. In addition, if there is a statistically significant difference in the measured inflow and outflow EMCs, a total BMP mass efficiency could be determined using the modeled volumes and the EMC statistics. The performance of BMPs in regards to EMC reductions should be evaluated using both the EP method and the ISE method. ·

The assessment of BMP performance should be limited to the water quality parameters for which sufficient data exists in the Database, and which commonly occur in stormwater. This includes common nitrogen and phosphorous species, suspended solids, copper, lead and zinc.

·

EMC reductions should be evaluated using the EP method to estimate the average difference between the inflow and outflow distributions, along with the computed 95% confidence limits as a measure of uncertainty.

·

EMC reductions should also be evaluated using the ISE method to estimate the average expected EMC reduction, along with the computed 95% confidence limits as a measure of uncertainty.

·

Consideration should be given to the potential for some BMPs to exhibit a consistent effluent quality rather than a consistent EMC reduction.

·

Consideration should be given to the potential for some BMPs to exhibit a minimum effluent quality which may influence the observed EMC reductions.

·

Mass load reductions should be estimated using long-term water balance modeling combined with the statistical assessment of EMCs.

·

The data developed from the EMC reductions for each BMP should be aggregated by BMP category to determine the expected reductions and uncertainty by type of BMP.

F:\Projects\FDEP Projects\TMDL Contract\Year 3\004.04 BMP Database\3-Analytical Methods\Draft\BMP Performance Assessment Analytical & Statistical Methods TM Ver 2.doc

Page 30

Florida Department of Environmental Protection FLORIDA BMP DATABASE Task Assignment 004.03/06-001

March 2006 Technical Memo BMP Analytical and Statistical Methods

References Burton, G.A. Jr., Ph.D., Pitt, R.E., Ph.D., P.E., 2001, Stormwater Effects Handbook - A Toolbox for Watershed Managers, Scientists, and Engineers. Helsel, D.R. and Hirsch, R.M., September 2002, Techniques of Water-Resources Investigations of the United States Geological Survey, Book 4, Hydrologic Analysis and Interpretation, Chapter A3: Statistical Methods in Water Resources, U.S. Geological Survey Olsson, Ulf, 2005, Confidence Intervals for the Mean of a Log-Normal Distribution, Journal of Statistics Education, Volume 13, Number 1. Strecker, E.W., 1998. Considerations and Approaches for Monitoring the Effectiveness of Urban BMPs. Proceedings of the National Conference on Retrofit Opportunities for Water Resources Protection in Urban Environments. 65-82 Strecker, E., Quigley, M. and Urbonas, B., 2001. Determining Urban Stormwater BMP Effectiveness. Journal of Water Resources Planning and Management, ASCE. 127(3), 175185 United States Environmental Protection Agency, July 2, 1999. Determining Urban Stormwater Best Management Practice (BMP) Removal Efficiencies: Task 3.1 – Development of Performance Measures Technical Memorandum. Washington, District of Columbia. United States Environmental Protection Agency, June 25, 2000. Determining Urban Stormwater Best Management Practice (BMP) Removal Efficiencies, Task 3.4 - Final Data Exploration and Evaluation Report, Washington, District of Columbia. United States Environmental Protection Agency. July 2000. Determining Urban Stormwater Best Management Practice (BMP) Removal Efficiencies: Task 1.1 – National Stormwater BMP Database Data Elements. Washington, District of Columbia. United States Environmental Protection Agency, March 2001. National Stormwater Best Management Practices Database User’s Guide, Release Version 1.2, Washington, District of Columbia. United States Environmental Protection Agency. April 2002. Urban Stormwater BMP Performance Monitoring - A Guidance Manual for Meeting the National Stormwater BMP Database Requirements, EPA-821-B-02-001, Washington, District of Columbia.

Page 31