Paper No.: 14-3350

Comparison of Sichel and Negative Binomial Models in Hotspot Identification By Lingtao Wu* Ph.D. Candidate, Zachry Department of Civil Engineering Texas A&M University, 3136 TAMU College Station, Texas 77843-3136 Phone: 979-587-3518, fax: 979-845-6481 Email: [email protected] & Research Assistant, Research Institute of Highway, MOT, China 8th Xitucheng RD., Haidian District Beijing, China 100088 Yajie Zou, Ph.D. Zachry Department of Civil Engineering Texas A&M University, 3136 TAMU College Station, Texas 77843-3136 Phone: 936-245-5628, fax: 979-845-6481 Email: [email protected] Dominique Lord, Ph.D. Associate Professor and Zachry Development Professor I Zachry Department of Civil Engineering Texas A&M University, 3136 TAMU College Station, Texas 77843-3136 Phone: 979-458-3949, fax: 979-845-6481 Email: [email protected] Word count: 5100 (Text) + 1500 (5 Tables and 1 Figure) = 6600 words

November, 2013 *Corresponding author

Wu et al.

2

ABSTRACT The identification of crash hotspots is the critical component of the highway safety management process. Errors in hotspot identification (HSID) may result in the inefficient use of resources for safety improvements. One HSID method that is based on the empirical Bayesian (EB) method has been widely used as an effective approach for identifying crash-prone sites. For the EB method, the negative binomial (NB) model is usually needed for obtaining the EB estimates. Recently, some studies have shown that the Sichel (SI) model can be easily used within the EB modeling framework and potentially yield better EB estimates. The objective of this study is to compare the performance of the two crash prediction models (SI and NB models) in identifying hotspots using the EB method. To accomplish the objective of this study, empirical crash data collected at highway segments in Texas were used to generate simulated crash counts. Three commonly used HSID methods (simple ranking, confidence interval and EB) were applied using simulated data. False positives, false negatives and false identifications were calculated and compared across the methods. The simulation results in this study suggest that the SI-based EB method can consistently provide a better HSID result than the NB-based EB method. Moreover, EB methods yield lowest error percentage among the three HSID methods. This study confirms that the EB technique is an effective method for identifying hazardous sites. Based on the findings in this study, transportation safety researchers are recommended to consider the SI model as an alternative crash prediction model when using the EB approach. Keywords: Safety; Hot spot identification; Negative Binomial; Sichel; Empirical Bayes analysis

Wu et al.

3

INTRODUCTION Network screening to identify sites with potential for safety treatments and the before-after evaluation of treatments are two important tasks of road safety management (1). The identification of sites with promise, also known as crash hotspots, hazardous locations, high-risk sites, is the first step that is part of the overall safety management process (2). Transportation management agencies use various hotspot identification (HSID) criteria and methods. Traditionally, observed accident frequency and accident rate are often used as identification criteria. According to Hauer (3; 4), sites identified by the various ranking criteria are unfortunately not congruent. In other words, some methods and criteria will result in inaccurate hotspots. There are two types of errors when identifying hotspots: (1) false positive (FP, identifying a safe site as hazardous) and (2) false negative (FN, identifying an unsafe site as safe) (5). These two errors can be the results of inaccurate methods and/or random fluctuation of accidents over time. Either of these two errors in identifying hotspots can result in inefficient investments and additional loss of lives (5; 6). Because of the importance in finding adequate HSID precision, discussions about HSID methods have been extensively conducted for decades. For example, Norden et al. (7) suggested using statistical control techniques, analogous of industrial quality control, to the study highway crashes and identify hazardous sites. Morin (8) applied a similar method for a safety improvement project. Tamburri et al. (9) introduced a safety index that assigned weights for different crash severity levels (Equivalent Property Damage Only or EPDO). This was later incorporated a commonly used HSID method based on the idea that sites with severe crashes deserve more attention (3). The methods mentioned above are all based on historical accidents, while, crash prediction models have been widely used in HSID in recent years. As it is now known, many statistical models have been proposed to predict safety performance by transportation safety analysts (10; 11). They include the negative binomial (NB), the Poisson-lognormal (12; 13), the Conway-Maxwell-Poisson (14; 15), the gamma (16), and the negative binomial-Lindley models (17). Miranda-Moreno et al. (12) pointed out that various models and ranking criteria can lead to different lists of hazardous locations. The empirical Bayes (EB) method for estimating traffic safety was introduced by Hauer et al. (18). It combines estimated crash counts and the historical record. The EB method addresses two problems of safety estimation: it increases the estimation precision and corrects for the regression-to-mean (RTM) bias. [Note: it has recently been shown that the EB could still be affected by the site-selection effects (19). For this study, it is assumed that the EB is properly estimated.] Higle and Witkowski (20) applied the EB method to identify sites with unusually high accident rates. Hauer (21) documented how the EB method can be used as part of an HSID program based on multivariate regression models.

Wu et al.

4

Due to the two notable advantages, estimation precision and correction for the RTM bias, the EB method significantly outperforms other techniques in identifying hotspots. Cheng and Washington (5) evaluated three commonly used HSID methods: confidence interval, simple ranking and the EB. The results illustrated that the EB technique significantly outperforms the ranking and confidence interval techniques. In a follow up study, Cheng and Washington (22) proposed four quantitative tests for evaluating performance of HSID methods: accident frequency ranking, accident rate ranking, accident reduction potential (also referred to as “Site with Promise”), and the EB. The results also revealed the EB method as the superior method. Elvik (23) assessed five HSID methods using epidemiological criteria (sensitivity and specificity), and the result confirmed the EB technique as the most reliable HSID method. More recently, Montella (2) tested performance of seven HSID methods: crash frequency, crash rate, proportion method, EB, empirical Bayes estimate of severe-crash frequency, and potential for improvement. The HSID methods were compared against four quantitative evaluation criteria. The tests showed again that the EB method performed better than all other HSID methods. As discussed before, the EB method combines the expected number of crashes obtained from prediction models and the number of historical crashes occurring at a given site. Among the prediction models, the NB model is probably the most frequently used statistical model due to the over-dispersion commonly observed in crash data (24). Consequently, the NB model has been extensively used as the crash prediction model for obtaining the EB estimates (2; 5; 22). Recently, Sichel (SI) distribution has been introduced by Zou et al. (25) for modeling dispersed crash counts. Compared to the NB model, the SI model has three different parameters and a dispersion term that can be used to measure the level of dispersion as well as obtaining the EB estimates. The SI model is more flexible and can handle the observed dispersion more efficiently than the NB model, while maintaining the same logical properties (11; 26). Zou et al. (11) has also showed that identified hotspots from various models (i.e., the NB and SI model) could be different when using the EB method. The objective of this research is therefore to examine the performance of the SI and NB models in identifying hotspots. To conduct this examination, properties of empirical crash data are used to generate simulated crash counts at hypothetical sites via the NB and SI models. Three commonly used HSID methods are applied to the simulated crash counts with two levels of confidence, and the models are compared based on the FP, FN as well as False Identifications (FI). BACKGROUND This section describes the characteristics of the NB and SI models, the EB method, and some other HSID methods.

Wu et al.

5

Negative Binomial Model The NB distribution is the most frequently used model in predicting safety performance of transportation elements (24). The NB model has the following model structure, the number of crashes y during a given time period is assumed to be Poisson distributed, which is given by:

p( y  ) 

 y exp( )

(1)

y!

where,

 = mean response of the observed crash counts during given period.  is assumed to be gamma distributed with E      and VAR     2 2 . Equation (2) shows the probability density function (PDF) of  (27): 1

f (  ,  ) 

1 ( 2  )1/

2





1



 /  2

e  (1 /  2 )

2

 (2)

The NB distribution can be viewed as a mixture of Poisson distributions where the Poisson rate  is gamma distributed. The PDF of the NB (Equation (4)) can be obtained by summing out  in Equation (3) (readers are referred to Hilbe (28) for complete derivation of the NB model):

p( y | ,  )  



0

p( y  ,  ) 

1

1

 

 /  2 

e  1  e  2 1/ 2 y! ( ) (1/  2 ) 

y

2

d

(3)

1 ( y  )

 y 1 1 ( ) ( ) 1 ( )( y  1) 1   1   

(4)



where,

y = response variable;

 = mean response of the observation;    2 is the dispersion parameter.

Sichel Model Recently, Zou et al. (11; 25) used the SI distribution, also known as Poisson-generalized inverse Gaussian distribution, to model traffic crashes. The SI distribution has been shown to be particularly useful when modeling highly dispersed crash data. The SI distribution can be viewed

Wu et al.

6

as a mixture of Poisson distributions where the Poisson rate  is generalized inverse Gaussian (GIG) distributed. The PDF of the GIG is shown in Equation (5) (27):   v 1    1  c    c   exp    f ( |  ,  , v )           2K  1    2   c    v     v

(5)

The number of crashes y is given by 

p ( y |  ,  , v )   p ( y |  ) f ( |  ,  , v ) d  0

The SI distribution can be derived by solving the above convolution integral. The PDF of the SI distribution with distribution parameters  ,  and v is defined as:

p ( y  ,  , v) 

( / c) y K y v ( ) Kv (1/  ) y !( ) y v

(6)

where, y response variable;

 = mean response of the observation; 

= scale parameter;

v shape parameter;

 2   2  2 (c )1 ; c

Kv 1 (1/  ) ; and, Kv (1/  )

K v (t ) 

1  v 1 1 x exp(  t ( x  x 1 ))dx is the modified Bessel function of the third kind.  0 2 2

When the scale parameter    and shape parameter v  0 , the SI distribution becomes the NB distribution. Note that the gamma distribution is a special case of the GIG distribution. And for the SI model, the dispersion term can be defined as:

h( , v)  2 (v  1) / c  1/ c2  1

(7)

Wu et al.

7

Empirical Bayesian Method The EB method has been widely applied to correct for the RTM bias. With the EB approach, estimation of the long-term safety of an entity is obtained from two sources, as described above. According to Hauer (29), let K be the observed number of crashes which is Poisson distributed, and  be the expected crash count, the EB estimator of  is:

  wE ( )  (1  w) K

(8)

 denotes the EB estimate of the expected number of crashes. E ( ) can be estimated by the crash prediction model (i.e., the NB or SI model). The weight factor w is given as (29):

w

1 VAR( ) 1 E ( )

(9)

The weight factor w is a function of mean and variance of  and is always a number between 0 and 1. VAR( ) is the variance associated with the model. So far, Equations (8) and (9) do not rest on any assumption about the distribution of  . If  is gamma distributed, then the resulting K follows a NB distribution. If we let  take a GIG distribution, then K follows a SI distribution. For the NB model, the weight factor can be estimating estimated as a function of E ( ) and dispersion parameter  , wi 

1 . Similarly, for the SI model, 1   E ( )

the weight factor can be estimated as a function of E ( ) and dispersion parameter h( , v) ,

wi 

1 , which is shown in Zou et al. (11). 1  h( , v) E ( )

Note that the EB analysis in this study is not the same as the one documented in the Highway Safety Manual, which uses the difference between the EB and safety performance function (SPF) or potential for safety improvement (PSI) (30). Some previous studies (2) have showed that the EB method performs better than the PSI for identifying hotspots. Hotspot Identification Methods The HSID method, as the name implies, is used to identify transportation sites, where crashes occur more frequently than other locations having similar characteristics. These relative hazardous sites can be intersections, road segments, etc. Due to the limited resources, (i.e. funding that can be allocated to implement highway safety improvement projects), HSID has

Wu et al.

8

been widely used in various transportation safety management programs. By identifying high risk locations, safety improvement strategies can be conducted to improve the safety performance of these locations. There are three commonly used HSID techniques: (1) Simple Ranking (SR), (2) Confidence Intervals (CI), and (3) the EB methods (5). The SR is a straightforward method. It is based on the historical crash frequency. By descending order of crash frequencies of locations, the unsafe sites are then identified. The CI method (5; 31) assumes the crash frequencies follow normal distribution. It identifies a location i by the critical value C ,

C    K S

(10)

where,

 = average crash frequency of comparison locations; S = standard deviation of the group of comparison locations;

 = cutoff level, typically 0.90, 0.95. The cutoff level  corresponds to the percent of all sites identified as unsafe sites. So, when  is equal to 0.90 means that the top 10% sites with the most crash counts will be identified as hazardous locations. If the observed crash count at location i exceeds the critical value C , this location will be identified as an unsafe site. Otherwise, the location will be identified as safe. The third method is the one based on the EB approach, as discussed above. In this study, the  ' s , the Poisson mean, is assumed to be either gamma distributed or GIG distributed. METHODOLOGY The main objective of this analysis is to assess the performance of two crash prediction models, the NB and SI, in identifying hotspots. Since simulation experiment is the main method used in this study, the simulation procedures are described in this section. Before describing the simulation procedures, it is necessary to discuss the motive for using simulation experiments rather than relying on empirical data. As mentioned in the previous section and indicated by previous research (5; 32), due to the random fluctuation in the number of crashes over time, the truly hazardous sites are hardly known when real observed data are used. This makes it extremely difficult to evaluate different prediction models in identifying hotspots. In contrast, we have the advantage of knowing the true safety state for each site when working with simulated data (11). A priori sites that are hazardous can be established via simulation. This makes it possible to assess the effects of prediction models and HSID methods.

Wu et al.

9

The simulation procedure proposed by Cheng and Washington (5) was adopted in this study, but necessary changes needed to be made. It consists of several specific steps, which are described in the follows: (1) Generate mean crash frequencies from real data. Crash records from rural two-lane highway segments in Texas are used to determine various distributions of crash means (  ' s ). Gamma and GIG distributions are used to fit the observed crash data to capture the heterogeneity in site crash means. The gamma distributed or GIG distributed means are denoted as TPM’s for True Poisson Means, which represent the means of crashes across a collection of sites. 1,000 TPM’s are generated to take advantage of large sample statistical properties. (2) Generate random Poisson counts from TPM’s. 30 crash counts are randomly generated for each TPM. Thus, for a set of crash counts are simulated, it contains 1,000 sites, and each site has 30 Poisson distributed crash counts representing observed data for 30 different observation periods. (3) Evaluate the performance of prediction models. Since the true safety conditions of 1,000 sites are known through the TPM’s, and each site has 30 “observed” traffic counts, it is possible to conduct the three HSID methods to these data, and the performance of the NB and SI models for the EB method can be evaluated. Specifically, the three HSID methods (i.e., SR, CI and EB methods) are examined in separate simulation runs to rank sites. And these three methods are applied and analyzed column-wise. Moreover, as suggested by Cheng and Washington (5), we assume that rows (data across different observation periods for the same site) can be used to represent the comparison group in order to calculate E ( ) and VAR( ) . To reflect different types of facilities that have varying safety performance functions, real-world crash data are typically classified based on the percentage of zeros and heterogeneity. The crash data are classified in the similar way as in Cheng and Washington (5). There are three levels of zero percentage: high (denoted by H), where the percentage of zeros is more than 50%; median (denoted by M), where the percentage of zeros is between 20% and 50%; and low (denoted by L), where the percentage of zeros is less than 20%. There are two levels of heterogeneity: high heterogeneity (denoted by H), where the range in of crash counts is more than 50; low heterogeneity (denoted by L), where the range is less than 30. Thus, the crash data are classified into six categories. These six categories of crash data are assumed to reflect different types of roadway facilitates. Each category is named with two letters, and the first letter stands for the zero percentage and the second one stands for the heterogeneity level. For example, MH stands for the crash data with a median percentage of zeros and high heterogeneity. For each category, the 1,000 TPM’s are simulated for each site and then 30 periods of crash counts for each site were randomly generated. Tables 1 and 2 show snapshots of the generated data based on the NB and SI models, respectively. Table 1 shows 15 simulated periods

Wu et al.

10

for 30 sites. The column ‘TPM’ is the true safety state for these 30 sites. The other 15 columns represents 15 different periods of simulated crash counts. The columns in Table 2 have the same denotations. For the characters of the two prediction models and EB assumptions, the data in the two tables are Poisson distributed within the site, while the data of Table 1 and Table 2, in aggregated format, are NB and SI distributed, respectively. In Table 1 and Table 2, three sites (site 28, 29 and 30) can be identified as a priori hazardous, since the TPM’s reflect the true underlying safety state. The other 27 sites are considered “safe” or not problematic, so to speak. Using one of the three HSID methods in any “observed” period, the “hazardous” and “safe” sites can be identified. By comparing the identified result with the true state, FPs, FNs and FIs can be calculated. The calculation methods are described as follows (5):

PFN 

N FN  100 NTS

(11)

PFP 

N FP  100 NTH

(12)

PFI 

N FN  N FP  100 N

(13)

where,

PFN = percentage of false negative, %; PFP = percentage of false positive, %; PFI = percentage of false identifications, %;

NFN = number of false negatives; NTS = number of truly safe sites; NFP = number of false positives;

NTH = number of truly hazardous sites; and, N = total number of generated sites (1,000 in this study).

Wu et al.

11

Thus, by using a large number of simulation sites and periods, statistical estimates of FNs, FPs and FIs for each model can be generated. TABLE 1 Simulated Crash Counts for 30 Sites and 15 Observation Periods (NB) 

Observed period 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 1 1 0 0 0 0 0 1 0 2 0 0 1 1 0 0 2 1 0 0 2 1 2 0 0 0 0 1 0 0 1 3 0 3 1 1 0 0 0 1 0 1 1 0 0 0 2 2 1 1 4 1 0 0 0 0 0 0 0 4 0 1 3 1 2 0 1 5 1 1 0 1 0 0 0 2 1 0 1 2 0 1 1 1 6 1 0 0 1 0 2 0 0 1 1 3 2 0 1 0 0 7 1 2 0 0 2 0 1 1 3 1 1 0 0 0 0 0 8 1 2 1 0 0 0 1 0 0 0 0 0 0 2 1 2 9 1 0 0 0 0 0 0 1 0 0 1 1 0 0 3 0 10 1 2 1 1 0 1 1 1 0 1 0 1 0 2 1 0 11 1 2 1 1 1 0 2 1 1 2 1 0 0 3 1 1 12 1 0 1 0 2 0 0 0 1 2 1 2 1 3 0 3 13 1 0 1 0 2 2 2 1 0 3 1 3 1 2 1 0 14 1 2 1 2 1 1 2 2 1 0 0 0 3 0 1 1 15 1 1 1 1 0 1 2 1 2 0 1 3 2 2 1 2 16 2 0 2 0 1 0 3 3 1 3 6 1 1 1 2 1 17 2 4 4 1 1 1 2 0 1 1 2 2 2 4 1 0 18 3 2 0 2 2 3 2 5 5 2 3 1 2 2 4 3 19 3 6 2 3 1 1 9 2 3 3 4 1 6 3 4 4 20 10 7 13 14 12 1 13 6 11 5 12 13 14 12 9 7 21 12 11 12 12 11 12 13 14 10 12 15 10 6 8 12 12 22 12 10 9 16 10 11 12 11 12 9 18 13 13 10 12 13 23 15 10 12 9 20 11 19 21 18 19 17 16 9 18 11 17 24 15 21 18 14 13 14 21 16 13 12 13 18 12 21 11 11 25 16 11 17 10 8 16 12 14 10 9 16 17 14 15 22 17 26 17 21 19 19 20 14 14 15 15 25 19 14 18 19 15 20 27 20 19 20 33 17 19 16 22 23 14 25 20 21 13 24 23 28 27 22 28 23 29 30 23 18 34 33 28 23 25 24 27 27 29 28 32 23 34 23 20 24 27 31 22 38 27 29 25 30 20 30 50 46 43 58 53 46 58 51 53 46 58 52 43 55 51 45 Site = number of site; TPM = true underlying safety of site (Poisson mean); simulated crash counts = observed crash count in observation period; shaded cells represent ‘truly hazardous’ locations (sites 28, 29 and 30). Site

TPM

Wu et al.

12

TABLE 2 Simulated Crash Counts for 30 Sites and 15 Observation Periods (SI) 

Observed period 1 2 3 4 5 6 7 8 9 10 11 12 13 1 1 1 1 2 0 1 1 0 3 1 1 0 0 0 2 1 0 1 0 0 0 0 0 1 1 1 0 0 0 3 1 0 1 0 0 0 0 0 0 0 0 0 0 0 4 1 3 0 2 0 0 1 2 0 0 2 0 1 1 5 1 2 0 1 0 1 1 1 2 0 0 0 2 0 6 1 0 1 1 1 0 0 1 1 0 2 2 0 1 7 1 2 0 1 0 0 1 1 1 1 2 0 1 2 8 1 2 0 1 0 1 0 0 2 1 1 0 0 0 9 1 0 0 0 0 0 0 2 1 3 0 3 2 1 10 1 1 1 1 1 0 1 2 1 0 0 1 0 2 11 1 1 0 1 0 1 0 0 1 0 1 1 0 3 12 1 0 0 2 0 2 0 1 1 2 2 1 1 0 13 1 1 2 2 2 0 0 0 1 0 1 2 2 1 14 1 3 2 3 2 2 3 2 2 1 2 0 1 2 15 2 1 3 0 1 1 2 0 2 1 0 3 3 3 16 2 3 2 0 1 2 0 0 7 1 4 2 3 3 17 3 4 5 6 4 1 0 1 1 3 2 1 2 1 18 3 2 3 2 1 2 1 1 4 0 3 2 3 2 19 5 6 5 2 1 4 5 4 6 5 3 3 5 6 20 5 3 4 6 5 6 6 10 7 9 2 4 6 4 21 6 11 6 8 5 5 4 6 5 3 2 3 5 6 22 9 7 6 5 11 5 10 10 3 9 7 7 9 6 23 12 10 7 9 12 10 15 15 17 14 12 13 7 10 24 19 23 20 22 21 25 23 17 15 13 22 17 23 19 25 22 23 26 16 15 23 20 21 15 24 21 21 15 17 26 22 19 22 20 27 27 21 24 20 18 23 24 21 20 27 28 17 30 24 28 27 30 32 31 33 24 26 22 32 28 30 19 36 31 23 24 35 23 36 31 30 35 31 34 31 37 29 30 35 26 30 38 32 23 29 35 22 37 29 30 40 26 42 35 35 45 38 49 33 35 44 33 41 51 The columns as well as the shaded cells have the same meanings as those in Table 1. Site

TPM

14 0 0 1 0 0 0 2 2 0 1 0 1 1 0 1 0 4 2 4 2 11 6 15 20 15 17 32 32 34 46

15 0 0 0 1 3 1 0 3 0 0 1 0 2 0 2 0 3 5 4 3 2 14 6 16 17 16 33 35 26 53

Wu et al.

13

DATA DESCRIPTION

The real-world crash counts used in this study to generate the TPMs were collected from rural two-lane highway segments in Texas. These crash counts were collected from 2003 to 2008 with a six-year period. The dataset satisfies the recommendation made by Cheng and Washington (5) that no less than three years of crash history duration should be used in HSID analysis. Crash counts were selected from six counties, and each crash dataset reflects one category of crash data introduced in the previous section. Table 3 provides the summary statistics of the six categories of crash counts. The cumulative distribution of each dataset is shown in Figure 1. These six datasets are used to try to adequately describe the characteristics of different true crash count distributions. TABLE 3 Summary Statistics of Six Datasets Selected from Different Counties County Crash Type* Mean SD** Minimum Maximum Sample Size 1 MH 6.58 151.14 0 90 334 2 ML 3.64 21.12 0 26 242 3 HL 1.62 8.90 0 18 139 4 HH 2.07 29.49 0 52 111 5 LH 11.30 227.20 0 85 174 6 LL 3.42 15.10 0 18 48 * MH - median zero percentage and high heterogeneity; ML - median zero percentage and low heterogeneity; HL - high zero percentage and low heterogeneity; HH – high zero percentage and high heterogeneity; LH - low zero percentage and high heterogeneity; LL - low zero percentage and low heterogeneity.

** SD – Standard Deviation

Wu et al.

14

FIGURE 1 Cumulative distribution of real data.

Wu et al.

15

MODELLING RESULTS

To generate simulated data from the empirical dataset, gamma and GIG distributions were used to fit the six datasets. Table 4 provides the estimated parameters as well as the goodness-of-fit statistics. The coefficients for both models have plausible values. The goodness-of-fit statistics (i.e., deviance, Akaike information criterion (AIC) and Bayesian information criterion (BIC)) all indicate that the GIG distribution works better than the gamma distribution for the six datasets (for the dataset with a low percentage of zeros and low heterogeneity, the gamma distribution is slightly better). Based on the fitting results in Table 4, we can observe that the GIG distribution can better describe the empirical crash distributions, and this may help generating more accurate true Poisson means which essentially represent the true safety performance of each site. The fitted gamma and GIG distributions for each of the six crash datasets were used to generate 1,000 TPM’s, respectively. And 30 periods of crash counts were randomly generated from the 1,000 TPM’s of each site, under the assumption that the underlying crash process of each site can be well approximated by a Poisson process (33). Thus 180,000 crash counts, including 1,000 sites during 30 periods of six different types of data, were “observed” using simulation. Three HSID methods with cutoff level  of 0.9 and 0.95 were applied to each period, and error percentages of FNs, FPs and FIs were calculated. Table 5 provides the HSID results using the simulated crash counts for the six categories of dataset. It can be seen that for the dataset of median percentage of zeros and high heterogeneity, the overall results generated from the SI model are better than those produced from the NB model with cutoff  of both 0.9 and 0.95. All of the error percentages from NB model are higher than those of SI model when using the same HSID method. For example, when using SR method with cutoff level  of 0.9, the FN percentage of NB model is 1.6%, it is almost 1.5 times of the value from the SI model, 1.1%. The same results can be seen in the right six columns in Table 5 with the cutoff  of 0.95. The error percentages of the SI model are nearly two-thirds of those from the NB model when using the same HSID method. Generally, the EB method also outperforms the CI and SR methods. The error percentages of CI and SR are nearly 1.7 times of those from EB method under same scenarios. The same result was observed in Cheng and Washington’s study (5). In short, the SI-based EB method provides the best result in terms of hazardous sites identification. For the other five categories of crash counts, the results are similar: (1) when using the EB method, the SI model is more accurate than the NB model in identifying hotspots (except for one FN under CI method with the cutoff  of 0.95); and (2) for the six categories of crash dataset, the SI-based EB method consistently yields the lowest error percentage of FNs, FPs and FIs (except for several FPs under CI method).

Wu et al.

16

TABLE 4 Fitting Results of Six Crash Data Using Gamma and GIG Distributions Data

Estimates

log(  ) log(  ) MH

ML

HL

HH

LH



Deviance AIC BIC log(  ) log(  )



Deviance AIC BIC log(  ) log(  )



Deviance AIC BIC log(  ) log(  )



Deviance AIC BIC log(  ) log(  )



Deviance AIC BIC

Value 1.8983 0.3924 na

1.3181 0.2678 na

0.5416 0.3590 na

0.7757 0.4068 na

2.4340 0.2432 na

Gamma SE 0.0810 0.0317 na 1742.04 1746.04 1753.67 0.0840 0.0381 na 1064.23 1068.23 1075.20 0.1215 0.0494 na 363.23 367.23 373.09 0.1426 0.0548 na 323.87 327.87 333.28 0.0967 0.0451 na 1161.62 1165.62 1171.94

t-Value 23.4331 12.3898 na

Value 1.8988 1.2359 0.0877

15.6875 7.0325 na

1.3181 1.0493 0.2668

4.4591 7.2680 na

0.5424 0.9944 -0.4199

5.4410 7.4236 na

0.7685 1.1046 -0.4417

25.1754 5.3902 na

2.4352 1.2415 0.4395

GIG SE 0.1039 0.0573 0.0563 1674.44 1680.44 1691.88 0.0985 0.0786 0.0820 1038.68 1044.68 1055.15 0.2183 0.1254 0.1112 293.62 299.62 308.43 0.2742 0.1524 0.1100 261.25 267.25 275.38 0.1057 0.1429 0.0873 1153.52 1159.52 1169.00

t-Value 18.2782 21.5822 1.5577

13.3769 13.3487 3.2550

2.4848 7.9301 -3.7762

2.8022 7.2462 -4.0163

23.0289 8.6902 5.0337

Wu et al.

17

TABLE 4 Fitting Results of Six Crash Data Using Gamma and GIG Distributions (Continued) Gamma GIG Value SE t-Value Value SE t-Value log(  ) 1.2575 0.1654 7.6020 1.2570 0.1777 7.0755 log(  ) 0.1363 0.0877 1.5553 1.0100 0.2980 3.3895  na na na 0.5734 0.2134 2.6870 LL Deviance 214.11 212.60 AIC 218.11 218.60 BIC 221.85 224.22 Note: na = not applicable; SE = Standard Error; The data names have the same meaning as those in Table 3. Definitions of  ,  and  can be seen in Equations (2), (5) and (6). Data

Estimates

TABLE 5 Percent Errors

Percent errors: median zero percentage and high heterogeneity  0.9 Model NB SI NB Method CI SR EB CI SR EB CI SR FN 0.3 0.8 2.0 1.6 1.0 2.4 1.1 0.7 FP 46.9 18.7 12.0 17.2 8.6 3.8 12.8 6.0 FI 2.6 1.7 3.0 3.1 1.7 2.5 2.3 1.2 Percent errors: median zero percentage and low heterogeneity  0.9 Model NB SI NB Method CI SR EB CI SR EB CI SR FN 0.7 1.2 2.4 2.3 1.4 2.8 1.7 1.1 FP 58.3 32.3 25.0 25.3 12.5 11.1 24.4 10.1 FI 3.6 2.8 4.6 4.6 2.5 3.7 3.9 2.0 Percent errors: high zero percentage and low heterogeneity  0.9 Model NB SI NB Method CI SR EB CI SR EB CI SR FN 1.4 1.4 3.1 3.1 1.9 5.4 1.8 1.2 FP 46.5 43.6 34.1 34.7 16.9 1.1 27.7 10.5 FI 3.6 3.5 6.2 6.2 3.4 5.0 4.4 2.1

0.95 SI CI SR 0.2 0.6 27.9 14.2 1.6 1.2

EB 0.5 8.6 0.9

EB 0.3 6.5 0.7

0.95 EB 0.8 14.7 1.5

CI 0.7 40.3 2.6

SI SR 1.0 26.8 2.2

EB 0.6 11.6 1.2

SI SR 0.8 21.1 1.8

EB 0.5 9.6 1.0

0.95 EB 1.0 18.4 1.8

CI 1.1 12.8 1.7

Wu et al.

18

TABLE 5 Percent Errors (Continued) 

Percent errors: high zero percentage and high heterogeneity  0.95 0.9 Model NB SI NB SI Method CI SR EB CI SR EB CI SR EB CI SR EB FN 0.9 1.3 0.8 1.3 0.6 0.4 3.3 2.3 1.5 6.0 1.7 1.0 FP 55.6 34.6 15.4 3.7 15.7 6.9 17.9 32.3 13.8 0.1 19.0 9.2 FI 3.6 2.9 1.5 1.5 1.4 0.7 4.8 5.3 2.8 5.4 3.4 1.8 Percent errors: low zero percentage and high heterogeneity  0.95 0.9 Model NB SI NB SI Method CI SR EB CI SR EB CI SR EB CI SR EB FN 0.2 0.8 0.4 4.4 0.6 0.4 1.6 1.4 0.8 9.1 1.2 0.8 FP 46.7 17.7 8.6 0.0 15.0 7.0 12.6 15.7 7.5 0.0 14.4 7.0 FI 2.6 1.6 0.9 4.2 1.4 0.7 2.7 2.8 1.5 8.2 2.6 1.4 Percent errors: low zero percentage and low heterogeneity  0.95 0.9 Model NB SI NB SI Method CI SR EB CI SR EB CI SR EB CI SR EB FN 1.0 1.3 0.8 0.9 1.2 0.7 2.9 2.8 1.7 3.0 2.3 1.5 FP 54.3 35.9 15.8 44.2 29.1 13.2 28.6 30.4 15.5 20.8 31.0 13.9 FI 3.6 3.1 1.6 3.0 2.6 1.3 5.5 5.6 3.1 4.8 5.2 2.8 FN - False Negatives; FP - False Positives; FI - False Identifications; CI - Confidence Interval; SR - Simple Ranking; EB - Empirical Bayesian;  - cutoff level. The shaded cells show the lowest identification error rate. SUMMARY AND CONCLUSIONS

The HSID is the first stage for improving roadway safety (3), and it is the critical component of a highway safety management process. HSID errors will lead to inefficient use of resources for safety improvements. Except for the traditional historical crash-based HSID methods, statistical models have been extensively applied in identifying hotspots in recent years. Among these methods, the EB technique has been developed and widely used as an effective approach for identifying crash-prone sites. The NB model is the most frequently used statistical method for obtaining the EB estimates, due to the over dispersion commonly found in crash data. Recently, the SI distribution has been introduced to predict the safety performance of transportation entities and it can be easily used within the EB modeling framework and potentially yield better EB

Wu et al.

19

estimates. Furthermore, identified hotspots using the SI model are also different from that of the NB model. The objective of this study was to examine the performance of the NB and SI models in identifying hotspots. To accomplish the study objective, crash counts at 1,000 hypothetical sites and 30 time periods were randomly generated using real-word crash data of rural two-lane highway segments in Texas. Three commonly used HSID methods, CI, SR and EB, with cutoff levels of 0.9 and 0.95 were applied using simulated data. Error percentages of FNs, FPs and FIs were calculated and compared. The major conclusions can be summarized as follows: (1) the GIG distribution provides better fitting results than that of the gamma, which indicates the GIG distribution is a more reasonable model for describing the true Poisson mean. (2) In identifying hotspot, the SI-based EB method can consistently have smaller error rates than the NB-based EB method, which suggests that the SI model is preferred as the crash prediction model for the EB method. Based on the simulation results, the EB methods yielded lower error percentage of FNs, FPs and FIs, which supports that the EB method is better than the other two HSID methods (CI and SR). Since the SI-based EB methods provide more desirable identification results, transportation safety researchers are recommended to consider the SI model as an alternative crash prediction model when using the EB approach. One limitation of this study is that the total crash count was used for identifying hotspots. Crash severity and collision type are useful indicators for HSID and to take these factors into consideration, the SI-based EB and the NB-based EB methods were applied using a real crash data (the road type was urban interstate highway segments). Some new criteria (22) were adopted to compare the SI and NB models. Crash severity (fatal, incapacitating injury, and non-incapacitating injury) and collision type (rear-end) were also considered. Although not documented here, the preliminary results generally support the premise that the SI-based EB method performs better than the NB-based EB method. The analyses are still ongoing and the results will be documented in a future publication. Finally, it is also anticipated that the SI-based EB method will be compared with other recent HSID identification techniques (e.g., full Bayesian (34), etc.) using different types of roadways. ACKNOWLEDGMENT

The authors acknowledge the FHWA’s Highway Safety Information System (HSIS) for providing roadway and crash data. The comments and suggestions of reviewers are also gratefully acknowledged. REFERRENCES

[1] Persaud, B., B. Lan, C. Lyon, and R. Bhim. Comparison of empirical Bayes and full Bayes approaches for before-after road safety evaluations. Accident Analysis and Prevention, Vol.

Wu et al.

20

42, No. 1, 2010, pp. 38-43. [2] Montella, A. A comparative analysis of hotspot identification methods. Accident Analysis and Prevention, Vol. 42, No. 2, 2010, pp. 571-581. [3] Hauer, E. Identification of Sites with Promise. Transportation Research Record, Vol. 1542, No. 1, 1996, pp. 54-60. [4] Hauer, E., and B. N. Persaud. Problem of identifying hazardous locations using accident data. Transportation Research Record, Vol. 975, 1984, pp. 36-43. [5] Cheng, W., and S. P. Washington. Experimental evaluation of hotspot identification methods. Accident Analysis and Prevention, Vol. 37, No. 5, 2005, pp. 870-881. [6] Persaud, B., C. Lyon, and T. Nguyen. Empirical Bayes procedure for ranking sites for safety investigation by potential for safety improvement. Transportation Research Record: Journal of the Transportation Research Board, Vol. 1665, No. 1, 1999, pp. 7-12. [7] Norden, M., J. Orlansky, and H. Jacobs. Application of statistical quality-control techniques to analysis of highway-accident data. Highway Research Board Bulletin, No. 120, 1956, pp. 17-31. [8] Morin, D. A. Application of statistical concepts to accident data. Highway Research Record, Vol. 188, 1967, pp. 72-29. [9] Tamburri, T. N., R. N. Smith, J. Mills, and V. Perini. The safety index: A method of evaluating and rating safety benefits. Highway Research Record, Vol. 332, No. 307, 1970, pp. 28-43. [10] Lord, D., and F. Mannering. The statistical analysis of crash-frequency data: A review and assessment of methodological alternatives. Transportation Research Part A, Vol. 44, No. 5, 2010, pp. 291-305. [11] Zou, Y., D. Lord, Y. Zhang, and Y. Peng. Comparison of Sichel and Negative Binomial Models in Estimating Empirical Bayes Estimates Bayes Estimates. Transportation Research Record, 2013, forthcoming. [12] Miranda-Moreno, L. F., L. P. Fu, F. F. Saccomanno, and A. Labbe. Alternative risk models for ranking locations for safety improvement. Transportation Research Record, No. 1908, 2005, pp. 1-8. [13] Lord, D., and L. F. Miranda-Moreno. Effects of low sample mean values and small sample size on the estimation of the fixed dispersion parameter of Poisson-gamma models for modeling motor vehicle crashes: A Bayesian perspective. Safety Science, Vol. 46, No. 5, 2008, pp. 751-770. [14] Lord, D., S. R. Geedipally, and S. D. Guikema. Extension of the Application of Conway-Maxwell-Poisson Models: Analyzing Traffic Crash Data Exhibiting Underdispersion. Risk Analysis, Vol. 30, No. 8, 2010, pp. 1268-1276. [15] Lord, D., S. D. Guikema, and S. R. Geedipally. Application of the Conway-Maxwell-Poisson generalized linear model for analyzing motor vehicle crashes. Accident Analysis and Prevention, Vol. 40, No. 3, 2008, pp. 1123-1134.

Wu et al.

21

[16] Oh, J., S. P. Washington, and D. Nam. Accident prediction model for railway-highway interfaces. Accident Analysis and Prevention, Vol. 38, No. 2, 2006, pp. 346-356. [17] Geedipally, S. R., D. Lord, and S. S. Dhavala. The negative binomial-Lindley generalized linear model: Characteristics and application using crash data. Accident Analysis and Prevention, Vol. 45, 2012, pp. 258-265. [18] Hauer, E., D. W. Harwood, F. M. Council, and M. S. Griffith. Estimating safety by the empirical Bayes method - A tutorial. Transportation Research Record, No. 1784, 2002, pp. 126-131. [19] Lord, D., and P.-F. Kuo. Examining the effects of site selection criteria for evaluating the effectiveness of traffic safety countermeasures. Accident Analysis & Prevention, Vol. 47, No. 0, 2012, pp. 52-63. [20] Higle, J. L., and J. M. Witkowski. Bayesian identification of hazardous locations (with discussion and closure). Transportation Research Record, No. 1185, 1988, pp. 24-36. [21] Hauer, E. Empirical bayes approach to the estimation of “unsafety”: The multivariate regression method. Accident Analysis & Prevention, Vol. 24, No. 5, 1992, pp. 457-477. [22] Cheng, W., and S. Washington. New criteria for evaluating methods of identifying hot spots. Transportation Research Record, No. 2083, 2008, pp. 76-85. [23] Elvik, R. Comparative Analysis of Techniques for Identifying Locations of Hazardous Roads. Transportation Research Record, No. 2083, 2008, pp. 72-75. [24] Lord, D., and F. Lord. The statistical analysis of crash-frequency data: A review and assessment of methodological alternatives. Transportation research. Part A, Policy and practice, Vol. 44, No. 5, 2010, pp. 291-305. [25] Zou, Y., D. Lord, and Y. Zhang. Analyzing highly dispersed crash data using the Sichel generalized additive models for location, scale and shape. Paper submitted for publication, 2012. [26] Stein, G., W. Zucchini, and J. Juritz. Parameter Estimation for the Sichel Distribution and its Multivariate Extension. Journal of the American Statistical Association, Vol. 82, No. 399, pp. 938-944. [27] Rigby, B., and M. Stasinopoulos. A flexible regression approach using GAMLSS in R. http://www.gamlss.org/wp-content/uploads/2013/01/Lancaster-booklet.pdf. Accessed July 28, 2013. [28] Hilbe, J. Negative binomial regression. Cambridge : Cambridge University Press, Cambridge, 2007. [29] Hauer, E. Observational before--after studies in road safety : estimating the effect of highway and traffic engineering measures on road safety. Pergamon, Tarrytown, N.Y., U.S.A., 1997. [30] AASHTO. Highway Safety Manual. 2010. [31] Laughland, J., L. Haefner, J. Hall, and D. Clough. Methods for evaluating highway safety improvements. Washington : Transportation Research Board, National Research Council,

Wu et al.

22

Washington, 1975. [32] Miranda-Moreno, L. F., A. Labbe, and L. Fu. Bayesian multiple testing procedures for hotspot identification. Accident Analysis and Prevention, Vol. 39, No. 6, 2007, pp. 1192-1201. [33] Lord, D., S. P. Washington, and J. N. Ivan. Poisson, Poisson-gamma and zero-inflated regression models of motor vehicle crashes: balancing statistical fit and theory. Accident Analysis and Prevention, Vol. 37, No. 1, 2005, pp. 35-46. [34] Huang, H., H. C. Chin, and M. M. Haque. Empirical Evaluation of Alternative Approaches in Identifying Crash Hot Spots. Transportation Research Record, Vol. 2103, 2009, pp. 32-41.