International Journal of Basic & Applied Sciences IJBAS-IJENS Vol:10 No:01

48

Probability Distribution Relationships Yousry H. Abdelkader and Zainab A. Al-Marzouq Dept. of Math., Faculty of Science , Alexandria University, Egypt Dept. of Math., Girls College, King Faisal University, KSA. E-mail address: [email protected] E-mail address: [email protected] Abstract — In this paper, we are interesting to show the most famous distributions and their relations to the other distributions in collected diagrams. Four diagrams are sketched as networks. The first one is concerned to the continuous distributions and their relations. The second one presents the discrete distributions. The third diagram is depicted the famous limiting distributions. Finally, the Balakrishnan skew-normal density and its relationship with the other distributions are shown in the fourth diagram.

Term-Probability Distributions, Transformations, Limiting Distributions. Index

I. INT RODUCT ION In spite of the variety of the probability d istributions, many of them are related to each other by different relat ions hips. Deriving the probability d istribution fro m other probability distributions are useful in different situations, for example, parameter estimations, simu lation, and finding the probability of a certain distribution depends on a table of another distribution. The relationships among the probability distributions could be one of the two classifications: the transformations and limiting distributions. In the transformations, there are three most popular techniques for finding a probability distribution fro m another one. These three techniques are: 1- The cumulative distribution function technique, 2- The transformation technique, and 3- The moment generating function technique. The main idea of these techniques works as follows: For given functions g i (X 1 , X 2 , ..., X n ) , for i  1, 2, , k where the joint distribution of random variables (r.v.'s ) X 1 , X 2 , ..., X n is given, we define the functions

Y i  g i (X 1 , X 2 , ..., X n ), i  1, 2, ..., k The joint distribution of Y 1 ,Y 2 , ...,Y n can be determined by one of the suitable method sated above. In particular, for k  1 , we seek the distribution of

Y  g (X ) For some function

g (X ) and a given r.v. X .

The equation (1) may be linear or non-linear equation. In the case of linearity, it could be taken the form n

Y   ai X i i 1

Many distributions, for this linear transformation, give the same distributions for different values for

ai such as: normal,

gamma, chi-square and Cauchy for continuous distributions and Poisson, binomial, negative binomial for d iscrete distributions as indicated in the Fig. by double rectangles. On the other hand, when

ai  1 , the equation (3) gives another

distribution, for examp le, the sum of the exponential r.v.'s gives the Erlang distribution and the sum of geomet ric r.v.'s gives negative- binomial d istribution as well as the sum of Bernoulli r.v.'s gives the binomial distribution. Moreover, the difference between two r.v.'s give another distribution, for example, the d ifference between the exponential r.v.'s gives Laplace distribution and the difference between Poisson r.v.'s gives Skellam distribution, see Fig. 1 and 2. In the case of non-linearity of equation (1) , the derived distribution may give the same distribution, for example, the product of log-normal and the Beta distributions give the same distribution with different parameters; see, for examp le, Cro w and Shimizu (1988), Kotlarski (1962), and Krysicki (1999). On the other hand, equation (1) may be give d ifferent distribution as indicated in the Fig.. The other classificat ion is the asymptotic or approximating d istributions. The asymptotic theory or limiting distribution provides in some cases exact but in most cases approximate d istributions. These approximations of one distribution by another one exist. Fo r example, for large n and small p the binomial distribution can be appro ximated by the Poisson distribution. Other approximations can be given by the central limit theorem. For examp le, for large n and constant p , the (1) central limit theorem gives a normal approximation of the binomial d istribution. In the first case, the binomial distribution is discrete and the appro ximating Poisson distribution is also discrete. While, in the second case, the binomial distribution is discrete and the appro ximating normal d istribution is continuous. In most cases, the normal or (2) standard normal p lays a very predo minant ro le in other distributions.

1001-91310-3434 IJBAS-IJENS © February 2010 IJENS

IJEN S

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol:10 No:01 The most important use of the relationships between the probability distributions is the simulation technique. Many of the methods in computational statistics require the ability to generate random variables fro m known probability distributions. The most popular method is the inverse transformation technique wh ich deals with the cu mu lative distribution function, F (x ) , of the distribution to be simulated. By setting

F (x )  U Where F (x ) and U are defined over the interval (0,1) and U is a r.v. follows the uniform distribution. Then, x is uniquely determine by the relation

x  F 1 (U )

(4)

Unfortunately, the inverse transformation technique can not be used for many distributions because a simple closed form solution of (4) is not possible or it is so complicated as to be impractical. When this is the case, another distribution with a simple closed form can be used and derived from another or other distributions. For example, to generate an Erlang deviate we only need the sum m exponential deviates each with expected value 1 expressed as m

x  yi   i 1

Where

m . Therefore, the Erlang

variate

x is

1m  lnU i ,  i 1

y i is an exponential deviate with parameter  ,

generated by the inverse transform technique and

U i is a

random nu mber fro m the uniform distribution. Therefore, a complicated situation as in simulat ion models can be replaced by a comparatively simp le closed form distribution or asymptotic model if the basic conditions of the actual situation are compatible with the assumptions of the model. Leemis (1986), Taha (2003) and Rider (2004) have tried to figure the relationships amog the probability d istributions in limited attempts. The first and second authors have presented a diagram to show the relat ionships among probability distributions. The diagrams have twenty eight and nineteen distributions including: continuous, discrete and limiting distributions, respectively. The Rider's diagram div ided into four categories: discrete, continuous, semi-bounded, and unbounded distributions. The diagram has only twenty distributions. This paper presents four diagrams. The first one shows the relationships among the continuous distributions. The second diagram p resents the discrete distributions as well as the analogue continuous distributions. The third diagram is concerned to the limiting distributions in both cases: continuous and discrete. The Balakrishnan skew-normal density and its relationships with other distributions are shown in the fourth diagram. It should be mentioned that the first diagram and the fourth one are connected. Because the fourth diagram depends on some continuous distributions such as: standard normal, ch i-square, the standard Cauchy, and the student's t-distribution.

49

Throughout the paper, the words "diagram" and " fig." shall be used synonymously. II. THE M AIN FEAT URES OF T HE FIGURES Many distributions have their genesis in a prime distribution, for examp le, Bernoulli and uniform distributions form the bases to all distributions in discrete and continuous case, respectively. The main features of Fig. 1 exp lain the continuous distribution relationships using the transformat ion techniques. These transformations may be linear or non-linear. The uniform distribution forms the base to all other distributions. The Appendix contains the well known d istributions which are used in this paper and are obtained fro m the following web site: http://www.mathworld.wo lfram.co m. It is written in the Appendix in concise and compact way. We do not present proofs in the present collection of results. For surveys of this materials and additional results we refer to Johnson et al. (1994) and (1995). The main features of Fig. 2 can be expressed as follows. If X 1 , X 2 , ... is a sequence of independent Bernoulli r.v.'s, the number of successes in the first n trials has a binomial distribution and the number of failures before the first success has a geometric distribution. The number o f failure before the kth success (the sum of k independent geometric r.v.'s) has a Pascal or negative bino mial distribution. The samp ling without replacement, the number of successes in the first n trials has a hyper-geometric distribution. Moreover, the exponential distribution is limit of geo metric d istribution, and the Erlang distribution is limit of negative binomial distribution. The other relationships of discrete distribution as well as the analogue continuous distribution can be seen clearly in Fig. 2. Fig. 3 is depicted the asymptotic d istributions together with the conditions of limiting. Limit ing distributions, in Fig. 3 and 4, are indicated with a dashed arrow. The standard normal and the binomial d istribution play a very predominant role in other distributions. Sharafi and Behboodian (2008) have introduced the Balakrishnan skew-normal density ( SNB n ( ) ) and studied its properties. They defined the

SNB n ( ) with integer

n  1 by

f n (x ;  )  c n ( )  (x ) n ( x ), x ,   , n  where the coefficient

c n ( ) 

1







Where U

 (x )  n (x ) dx



c n ( ) is given by

1

E   n ( U ) 

N (0,1) .

1001-91310-3434 IJBAS-IJENS © February 2010 IJENS



IJEN S

,

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol:10 No:01

50

n  1 and c n ( )  2 which g ives the skew-normal density (SN ( )) , see Azzalin i

Fig. 4 explains the main features of their results. The dotted arrows indicate asymptotic distribution.

(1985). The probability distribution of the Balakrishnan skewnormal density and its relationships of the other distributions are also discussed. The most important properties of SNB n ( ) are:

III. CONCLUDING REMARKS It is a reasonable assertion that all probability distributions are related to one another. In this paper, four diagrams summarize the most popular relationships among the probability distributions. The relationships among the probability d istributions are one of the two classificat ions: transformation and limit ing. Each d iagram exp lains itself. The advantages of using these diagrams are: the student at the senior undergraduate level or beginning graduate level in statistics or engineering can use the diagrams to supplement course material. Besides, the researchers can use the diagrams for fasting search for the relationships among the distributions. These diagrams are just start points. Similar d iagrams can be constructed to summarize many statistical theorems such as: the characterizat ions of distributions based on; order statistics, Records and Moments, see Gather et al. (1998).

An special case arises when

(i ) SNB n ( ) is strongly unimodal,

(ii ) c1 ( )  2, 1

1 1  c 2 ( )    sin 1   ,  4 2  1

1 3  c 3 ( )    sin 1   ,  8 4  Where



is denoted the correlation coefficient. For

n  4 , there is no closed form for c n ( ) . But some approximate values can be found in Steck (1962). The bivariate case of

SNB n ( ) and the location and scale

parameters are also presented. Appendix Continues Distribution

Domain

Beta Distribution

[0,1]

 0,  0

f (x ) 

Beta Prime Distribution

[0, )

 0,  0

f (x ) 

Cauchy Distribution

(, )

Parameters



PDF

,b 0

x  1 (1  x )  1 B ( ,  )

x  1 (1  x )   B ( ,  ) 1 b f (x )   (x   )2  b 2 1 n

Chi Distribution

[0, )

2 2 n 1  12 x 2 f (x )  x e   n2 

n 0

x

n 1 2

e

 12 x

Chi-Squared Distribution

[0, )

n 0

f (x ) 

Degenerate Distribution

x 0 

x0 

1 f (x )   (x  x 0 )   0

Erlang Distribution

[0, )

Exponential Distribution

[0, )

n  1, 2,3,

 0

 0

,

  n2  2 2 n

, x  x0 , x  x0

1x

x n 1e  f (x )  (n  1)! n f (x ) 

1



e



1001-91310-3434 IJBAS-IJENS © February 2010 IJENS

x



IJEN S

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol:10 No:01 n1

F- Distribution

[0, )

n1  0, n 2  0

Gamma Distribution

[0, )

 0 , 0

Gibrat's Distribution

(0, )

None

 0 ,

Gumbel Distribution

(, )

Half-Normal Distribution

[0, )

h 0

Inverse Chi-Square Distribution

(0, )

 0

Inverse-Gamma Distribution

(0, )

Kumaraswamy Distribution

[0,1]

Laplace Distribution

(, )

Levy Distribution

[0, )

Logistic Distribution

1x

f (x ) 

1

    x   

  x     

   e 

 

e h

 0 , 0

f (x ) 

  x  1e   

a  0,b  0

f (x )  ab x a 1 (1  x a )b 1

b  0, 

f (x ) 

b  0, 

Maxwell Distribution

[0, )

a0 n  0,   0

2

x 2 /

  2 2 2 1  21x f (x )   x e  2  x

1  x  b e 2b c e c 2 x f (x )  2 x 3/ 2

f (x ) 

 x      b 

e

 x      b 1  e  b    

 ln x   2   f (x )  exp     2 2 x  2   2  x 2 2a 2 2 x e f (x )   a3 f (x ) 

x

n 1 2

e

 x      2 

( x ) k  2k k !(k  n2 ) k 0 2 

n



n1 , n 2 , 1 , 2  0

2

1

22 [0, )

n1 n 2 2

x  1e  f (x )  ( )   2 1 f (x )  e  (ln x ) / 2 x 2

2h

  0, 

Noncentral F-Distribution



, n22 (n1  n 2 x )

n1 2

f (x ) 

[0, )

[0, )





Lognormal Distribution

Noncentral Chi-Squared Distribution

B

n2 1 2

e

c 0

(, )

f (x ) 

n2

n1 2 n 2 2 x

51



f (x )   k 0 l 0

n1

k  n1 2

2k  l e

n2

12 2

l n2 2



x

k  n1 1 2

1k 2l

B k  n21 , l  n22 (n 2  n1x )

1001-91310-3434 IJBAS-IJENS © February 2010 IJENS

( k l )



 n1 n 2 2



IJEN S

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol:10 No:01

52

n

f (x )  Noncentral Student's t-Distribution

(, )

Normal Distribution

(, )

  0, 

Pareto Distribution

[ x 0 , )

x 0  0, k  0

Pearson Type III Distribution

[0, )

 ,   0, 

Rayleigh Distribution

n  0,   0

2

[0, )

  0 ,  0

[0, )

Standard Normal Distribution

(, )

None

Student's t-Distribution

(, )

n 0

2n e 

(n  x 2 ) 2   n2  n

/2



 2 x F n  1; 3 ;  2 x 22 1 1 2 2 2( n  x )   2 (n  x )  n21    1 f (x )  e  2 k x 0k f (x )  k 1 x

 F 1 1

2 2

 1

x   f (x )    ( )     f (x ) 

xe

f (x ) 

x



2 2  1; 12 ; 2(n xx 2 )   n  x 2   n2  1  

n 2

( x   )2

1



 0

Rice Distribution

n 2n! 2

e

 x       

x2 2 2

2



 2

e

x 2  2 2 2

 x  lo  2   

2

1  x2 f (x )  e 2   n21  f (x )  n 1 n    n2  (1  x 2 / n ) 2

Triangular Distribution

[a, b ]

a c b

Uniform Distribution

[a, b ]

a, b

Wald Distribution

(0, )

  0,   0

Weibull Distribution

,   0

[0, )

Discrete Distribution

Domain

Bernoulli Distribution

{0,1}

Beta Binomial Distribution

{0,1,

, n}

 2(x  a )  (b  a )(c  a ) a  x  c  f (x )    2(b  x ) c  x b   (b  a )(b  c ) 1 f (x )  b a 1/ 2   ( x   ) 2     f (x )   exp  2 2 x  3  2 x    x 

   f (x )   x  1e    

Parameters

0  p  1, q  1 p,  ,   0, n  1, 2,



P.M.F.

q P (x )   p

x 0 x 1

 n  B (x   , n  x   ) P (x )    B ( ,  ) x 

1001-91310-3434 IJBAS-IJENS © February 2010 IJENS

IJEN S

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol:10 No:01

0  p  1, Binomial Distribution

{0,1,

, n}

q  1 p, n  1, 2,

Discrete Uniform Distribution

{0,1,

, n}

n  1, 2,

Geometric Distribution

Hypergeometric Distribution

{0,1, 2, }

{0,1,

, n}

Log-Series Distribution

{1, 2,3, }

n  P (x )    p x q n  x x  P (x ) 

0  p  1,

,

K  0,1,

,N ,

n  0,1,

,N ,

k , q  1 p N

0  p 1

0  p  1,

[3]

[4] [5]

P (x )  

x x ln(1   )

{0,1, 2, }

q  1 p, k  1, 2,

 x  k  1 k x P (x )   p q  k 1 

Poisson Distribution

{0,1, 2, }

 0

P (x ) 

None

 P (x )   1 2

Skellam Distribution

[2]

 Np  Nq     x  n  x  P (x )   N    n

Pascal Distribution

Rademacher Distribution

[1]

1 n

P (x )  p q x

q  1 p, N  0,1, 2,

p

53

{1,1}

{ , 1,0,1, }

1  0, 2  0

REFERENCES A. Azzalini, A class of distributions with includes the normal ones, Scandinavian journal of statistics, 12 (1985), 171-178. E. L. Crow, K. Shimizu, Lognormal distributions: Theory and Applications, Marcel Dekker, New York, 1988. U.Gather, U., Kampus, and N. Schweitzer, Characterizations of distributions via identically distributed functions of order statistics, In: Balakrishnan, N., and Rao, C. ed., Handbook of Statistics, 16, Elsevier Science, 1998, 257-290.

 x e  x! 1 2

P (x )  e

 ( 1  2 )

x  1 x 1

 1     2 

x /2



I x 2 12



[11] H. A. Taha, Operations Research: An introduction, 3rd ed, Macmillan Publishing Co., 2003.

N. Johnson, S. Kotz and N. Balakrishnan, Continuous univariate distributions, Vol. 1, 2nd ed. Wiley, New York, 1994. N. Johnson, S. Kotz and N. Balakrishnan, Continuous univariate distributions, Vol. 2, 2nd ed., Wiley, New York, 1995.

[6]

I. Kotlarski, On group of n independent random variables whose product follows the beta distribution, Collog. Math. IX Fasc., 2 (1962), 325-332. [7] W. Krysicki, On some new properties of the beta distribution, Statistics & Probability Letters, 42(1999), 131-137. [8] L. Leemis, Relationships among common univariate distributions, T he American Statistician, 40(1986), 143-146. [9] M. Sharafi, J. Behboodian, The Balakrishnan skew-normal density, Statistical Papers, 49(2008), 769-778. [10] J.W. Rider, Probability distribution relationships, http://www.jwrider.com, 2004.

1001-91310-3434 IJBAS-IJENS © February 2010 IJENS

IJEN S

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol:10 No:01

54

Fig. 1. Continuous Distribution Relationships

1001-91310-3434 IJBAS-IJENS © February 2010 IJENS

IJEN S

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol:10 No:01

55

Fig. 2. Bernoulli’s trials and its related distributions

Fig. 3. Limiting distributions 1001-91310-3434 IJBAS-IJENS © February 2010 IJENS

IJEN S

International Journal of Basic & Applied Sciences IJBAS-IJENS Vol:10 No:01

Fig. 4. T he Balakrishnan Skew-normal density

56

SNB n ( )

1001-91310-3434 IJBAS-IJENS © February 2010 IJENS

IJEN S