The art of Counting Combinatorics

1. say numbers: to say numbers in order, usually starting at one 2. add up: to add things up to see how many there are or to find the value of an amount of money 3. include: to include somebody or something in a calculation 4. consider or be considered: to consider somebody or somethi ng, or be considered, in a particular way or as a particular thing 5. be of importance: to be of importance or value 6. have a value: to have a specific value 7. music dance keep time: to keep time by counting beats

•

• Portrait of Georg Gisz

[14th century. The noun is via old French conte ; the verb directly from Old French conter “to reckon (estimaciones),” from Latin computare , literally “to reckon together.”]Microsoft® Encarta® Reference Library 2003. © 1993-2002 Microsoft Corporation. All rights reserved. com (con) y putare (pensar)

1

2

1532

What is counting?

What is counting?

• Ethymologies – [14th century. The noun is via old French conte ; the verb directly from Old French conter “to reckon (estimaciones),” from Latin computare , literally “to reckon together.”]Microsoft® Encarta® Reference Library 2003. © 1993-2002 Microsoft Corporation. All rights reserved. – com (con) y putare (pensar)

– One famous quote by Terence reads: "Homo sum, humani nihil a me alienum puto", or "I am human, nothing that is human is alien to me." This appeared in his play Heauton Timorumenos.

• Synonims – calculation, number crunching, reckoning, computation – total, sum total, sum, amount, tally

• Tally: – Paleolithic tally bones with numerical notches. http://www.hunmagyar.org/turan/magyar/tor/carp.htm

– Mesopotamia: http://www.edp.ust.hk/math/history/2/2_2.htm

3

What is counting?

4

Strategies for counting • Intension vs. Extension

• An encyclopedic account of how counting was develloped:

– Extension: • Direct link or enumeration of each of the elements of a set

– Intension:

HISTORIA UNIVERSAL DE LAS CIFRAS

• Description of the elements of a set by means of the properties

de IFRAH, GEORGES ESPASA-CALPE, S.A. ISBN: 8423997308 G. Ifrah: The Universal History of Numbers. John Wiley & Sons, New York 2000 [ISBN 0-471-39340-1]

– Combinatorics: • Counting without enumeration. 5

6

1

Strategies for counting

Bijection

• Intension vs. Extension

• One to one correspondence between sets of equal size.

1 ↔ Elem1 2 ↔ Elem2 M N ↔ ElemN

– Extension:

•Application to infinite sets •Hilbert’s hotel

– Intension:

Elem1 Elem2 Description↔ M ElemN

This tapestry, created in the late 1400s, is in the collection of the Cluny Museum in Paris, France.

A painting by the American Edward Hicks (1780–1849), showing the animals boarding Noah's Ark two by two.

7

8

Bijection

Bijection

• Counting with the pigeon hole+description.

• Compression of files.

– There must be at least two people in London with the same number of hairs on their head.

– No Lossless compression of files.

• A typical head has arround 150.000 hairs

– Birthday problem with N>365 – Collisions in a Hash table. • Number of keys exceed the number of indices in the array.

Count all possible files of 1M and assign them a number

9

10

Bijection

Bijection

• One to one correspondence between sets. • Examples:

– Multiples of 6 with 3 cifers that beging with 4 • Notice the structure of the problem:

– Multiples of 7 between 39 and 292

i ↔ 7(5 + i)

• Examples: 402 = 6*67 =6*(66 +1)

498 = 6*83 =6*(66 +17)

1 ↔ 42 2 ↔ 49

i ↔ 6(66 + i)

M 36 ↔ 287

1 ↔ 402 2 ↔ 408 M 36 ↔ 498

11 Examples taken from VÉLEZ , HERNÁNDEZ, Cálculo de Probabilidades

12 Examples taken from VÉLEZ , HERNÁNDEZ, Cálculo de Probabilidades

2

Bijection

Bijection

• Example Extension/Intension:

• Numerable sets->Cardinality of N

– Compute the number of games on a tennis championship with 84 players • First round->42 games (42players), Second->21g (21p), third->10g (11p),fourth->5g (6p),fifth->3g (3p), sixth->1g (2p), seventh->1g (1p) • Total: 42+21+10+5+3+1+1=83

– Algorithm: Play matches until one player remains>Bijection between eliminated players and games. • Notice that for n players->n-1 games.

– We admit infinite sets as long as are numerable. – R are not admited • Cantors diagonalization: n1 = (0, 2 n2 = (0, 1 n3 = (0, 9 n4 = (0, 5 n5 = (0, 1 n6 = (0, 3 n7 = (0, 1

2 3 6 1 1 6 1

3 5 1 6 1 6 7

9 6 4 5 4 2 7

n0 = (0, 3

4

5

6

5 4 4 5 5 5 7 ... 6

3 6 7 3 8 8 9

5 6 2 1 1 2 8

4 8 7 9 4 3 4

5 6 8 2 1 8 9

1...) 6...) 1...) 8...) 3...) 9...) 6...)

9

9 …………….....)

13

14

Examples taken from VÉLEZ , HERNÁNDEZ, Cálculo de Probabilidades

Bijection

Bijection

• Numerable sets->Biblioteca de Babel

• Numerable sets->Biblioteca de Babel

Borges' Biblioteca de Bábel (exc.) El universo (que otros llaman la Biblioteca) se compone de un número indefinido, y tal vez infinito, de galerías hexagonales, con vastos pozos de ventilación en el medio, cercados por barandas bajísimas. Desde cualquier hexágono, se ven los pisos inferiores y superiores: interminablement e. La distribución de las galerías es invariable. Veinte anaqueles, a cinco largos anaqueles por lado, cubren todos los lados menos dos; su altura, que es la de los pisos, excede apenas la de un bibliotecario normal. Una de las caras lires da a un angosto zaguán, que desemboca en otra galería, idéntica a la primera y a todas. A izquierda y a derecha del zaguán hay dos gabinetes minúsculos. Uno permite dormir de pie; otro, satisfacer las necesidades finales. Por ahí pasa la escalera espiral, que se abisma y se eleva hacia lo remoto. En el zaguán hay un espejo, que fielmente duplica las apariencias. Los hombres suelen inferir de ese espejo que la Biblioteca no es infinita (¿si lo fuera realmente, ¿a qué esa duplicaci ón ilusoria?); yo prefiero soñar que las superficies bruñidas figuran y prometen el infinito ... La luz procede de unas frutas esf éricas que llevan el nombre de lámparas. Hay dos en cada hexágono: transversales. La luz que emiten es insuficiente, incesante.

book1 → 0000000001 book 2 → 0000000002 M bookN → 9999999999 M book∞ → ∞ ? Probability of an infinite set? 15

Product rule

Product rule

• Given the sets A1, L Ak each with n1 ,L nk elements, the cartesian product A1 × A2 L× Ak has n1n2 Ln k elements

Feller An Introduction to Probability … .. Examples taken from VÉLEZ , HERNÁNDEZ, Cálculo de Probabilidades

16

17

• Examples: – Matrix of linear algebra. Number of coefficients when you have m equations and n unknowns

18

3

Product rule

Product rule

• Examples:

• Posible Aplications of n-grams:

– Bigrams or n-grams

– Prob of words of two letters with one vocal 24 ⋅ 24 = 576 24 ⋅ L 24 = 24n

http://www.inference.phy.cam.ac.uk/mackay/itila/book.html

Pr =

Countof ways foraresult Countofallpossibleresults

Pr =

Numberoftimesaresultappears Countofthenumberoftrials

19

20

Product rule

Usual Patterns • Orderings/Permutations

• Example: – How many different words of 5 different letters can be formed with the letters A,E,I,L,M,N,P with a vocal in first position

( A, E, I )

6

5

4

•

Note: Remeber D ’Alembert ’s mistake !

• Definition of Permutation – on elements uses a matrix, where the first row is and the second row is the new arrangement. For example, the permutation which switches elements 1 and 2 and fixes 3 would be written as

↓ X{ X{ X{ X{ X { 3

– Application of the product rule – Distinguishable objects may be arranged in various different orders

3

3 * 6 * 5 * 4 * 3 = 1080 21

22

Examples taken from VÉLEZ , HERNÁNDEZ, Cálculo de Probabilidades

Usual Patterns

Usual Patterns

• Counting permutations:

• Counting permutations/Comments: – How to order n different objects in r possitions?

– How to order n different objects?.

{o1 , o2 ,L , on } {o1 , o2 ,L, on } − {oi} ↓

{o1 , o2 ,L , on } {o1 , o2 ,L, on } − {oi}

n ( n − 1)( n − 2) L 1 = n !

↓

O ↓ n− 1 n− 2

n ( n − 1)( n − 2 )L( n − r +1 ) =

2

(n)r

X{ X{ XL { X {

X{ XX X { { LX {{ n

O ↓

n

1 23

n− 1 n− 2

X {

n − ( r − 2) n −( r − 1)

n!

(n − r ) !

Is there a relation with the indistiguishable case? 24

4

Usual Patterns

Usual Patterns

• Counting permutations:

• Counting permutations without repetition:

– Example: Possible noncoincident birthdays of r persons.

– Another way of setting the problem.

365364363 − (r − 2)365 − (r − 1) { { { L365 1424314243 n−1

n

n−2

n −( r −2 )

r

n−( r −1)

Possible configurations

365!

(365 − r )!

– Note: This example will be used for the coincidence problem, and for deriving the Poison pdf

365 25

26

Usual Patterns

Usual Patterns

• Counting permutations with repetition:

• Counting permutations with repetition:

– How to order n different objects with repetition?.

{o1 , o2 ,L ,on } {o1 , o2 ,L ,on } ↓

– Another way of setting the problem.

nnn L3 n =nr 1 424 r

r

O

Possible configurations

O

↓

365r

X{ XX X{ X { {L { n

n

n

n

365

n http://www.kimbellart.org/database/images/AP1987_06.gif

27

28

Usual Patterns • Internet’s Diameter:

Usual Patterns • Internet’s Diameter:

•Mean number of links/node= k •Number of nodes=N N

k = ∑ ki

•Mean number of links/node= k •Number of nodes=N kn

N

k = ∑ ki

N

i =1

i =1

kn

O

N Ok3 O

kd = N d=

log N ≈ 19 log k

Ok3

k2

Ok1

k2

kd = N

Ok1 29

d=

log N ≈ 19 log k

But 1016 >> 109 , what does it mean? Maximum path from one node to the other.

30

5

Usual Patterns – What does

d=

Usual Patterns

log N ≈ 19 mean? log k

• Six Degrees – Six degrees of separation is the hypothesis that anyone on Earth can be connected to any other person on the planet through a chain of acquaintances with no more than five intermediaries.

– Number of atoms in the universe is N = 107 0 d =

– The logarithm gives

log10 N = 70 log10 10

The Collective Dynamics of ‘Small World’ Networks

• Notice: the logarithm gives the number of decimal places needed to write the number.

D. J. Watts and S. H. Strogatz Nature, vol. 393, no. 6684, pp 440 2. (1998)

– It is a description of the number •Mean number of links/node= k •Number of nodes=N 31

32

Usual Patterns

Usual Patterns

• Six Degrees/Example • Comparison:

– How do you get from Muhammad Ali to Bob Dylan?

– Number of orderings n different objects selected r times with and without repetition ?

• 1: Muhammad Ali's manager for most of his career was Angelo Dundee. • 2: Angelo Dundee also managed Heavyweight Champion Jimmy Ellis. • 3: In 1964, Jimmy Ellis was defeated by Rubin "Hurricane" Carter. • 4: The song Hurricane was written about Rubin Carter and appears on Bob Dylan's album Desire.

•

nr vs.

n!

(n − r )!

http://www.coudal.com/dylan/

33

34

Usual Patterns

Usual Patterns

• Counting permutations/ indistinguishability: – – – –

{X , o , o , X L , X ,o j 1

j 2

↑ ↑

j nj

↑

,X

}

• Counting permutations

{ { { { { {

Some indistinguishable objects. n objects each with { n1 , n2 ,L , nn } elements Notation: {o kj }Object k of class j Notice that for a given permutation:

→ { X , oj , oj , X L , X , oj , X}

Fixed

• The {n j }permutations are equivalent, therefore have to be discounted

35

Some indistinguishable objects : X , o1j , o2j , X L , X , o3j , X X , o1j , o3j , X L , X , o2j , X j j j X , o2 , o1 , X L , X , o3 , X j j j → {X , o , o , X L , X , o , X } X , o2j , o3j , X L , X , o1j , X All the n j ! are different sets are equivalent to X , o3j , o1j , X L , X , o2j , X one. j j j n! X , o3 , o2 , X L , X , o1 , X DifferentPermutations →

} } } } } }

nj!

36

6

Usual Patterns

Usual Patterns

• Counting permutations:

• Counting permutations: – Example: With the simbols {a,a,a,b,b,c,c,d} how many words can be constructed? – Note that the same word can be constructed in {a,a, b,c,a,b,c,d} different ways.

– General Formula n! n = n1 + n2 + L + nk n1!n2 !L nk !

{a,a, b,c,a,b,c,d}

– Different ways to order a collection of n objects with k subsets of indistinguishable objects.

– The number of words is:

8! = 1680 3!2!2!1!

37

38 Examples taken from VÉLEZ , HERNÁNDEZ, Cálculo de Probabilidades

Usual Patterns

Usual Patterns

• Models of the equation

• multinomial example (Rao, 1973)

– Thermodynamics: {n } particles in a given state. How many different realizations can we have? – Language Models – Derivation of the Entropy (Thermodynamics/Information Theory)

There are n=197 animals classified into one of 4 categories: (y1 ,y2 ,y3 ,y4 )=(125,18,20,34) n! Number of different orderings is y1! y 2! y3 ! y 4!

j

{a,a, b,c,a,b,c,d} {a,a, b,c,a,b,c,d}

http://en.wikipedia.org/wiki/Image:Jan_Bruegel_d._%C3%84._003.jpg

39

Rao, C. R. (1973). Linear statistical inference and its applications. Wiley.

Usual Patterns

Usual Patterns

• multinomial example (Rao, 1973)

• Language Models

There are n=197 animals classified into one of 4 categories: (y1 ,y2 ,y3 ,y4 )=(125,18,20,34) n! Number of different orderings is y1! y 2! y3 ! y 4!

Does a text correspond to an author? Apriori probability of using a word: A text has the word frequencies:

Objective of the paper: test if the observations relate to the probability model π .

Nature

(125,18,20,34)

40

y

y

{ p1 , p2, L, pn } {n1 , n2 ,L, nn }

Number of different books with the same word frequenc¡es n! y

y

n! π 1 1− π 2 1 − π 3 π 4 y1! y2 !y3 ! y4 ! 4 4 4 4

n1 !n2 !Ln n ! Note: Does not take into account the probability of a word.

Rao, C. R. (1973). Linear statistical inference and its applications. Wiley.

41

42

7

Usual Patterns

Meanings of model • Engineering vs. Mathematics/logic

• Language Models Does a text correspond to an author? Apriori probability of using a word: A text has the word frequencies:

{ p1 , p2, L, pn } {n1 , n2 ,L, nn }

Description of nature

Nature

Probability that the text was written by the author n! n n n p1 1 p 2 2 L p n n n1!n2 !L nn! http://humanum.arts.cuhk.edu.hk/humftp/Fine_Arts/Gallery/matisse/matisse.jpg

43

44

Usual Patterns

Usual Patterns

• Subsets :

• Subsets : – How many possibilities when: Sell n sits in plane and have r cancelations

– Given a set of n elements, how many subsets of cardinality r can be made? • Examples: How many possibilities when:

• Notice 2 points:

– Throw a die n times and get r faces – Sell n sits in plane and have r cancelations

– Temporal order of booking is not important – Sets are ‘bookings’, not ‘cancelations’

» Overbooking. D’alembert’s mistake.

» D’alembert’s mistake. Cancelation

45

{b1 ,

L

b2 ,

L

, b n −1 ,

Usual Patterns

Usual Patterns

– n sits in plane and have r cancelations

– n sits in plane and have r cancelations

• Define a set of indices: • Possible sets of bookings:

{i1, i2 ,L , in− r }

{

bi1 , bi2 , L , bin −r

b46n }

• If each booking is different

}

• But in a set order is not important

– Why (n-r)? (orders?)

• If each booking is different n ( n −1 )( n − 2 ) L( n − r +1) =

– Is there a difference between the sets of cancelations? n! ( n − r )!

n ( n −1) ( n − 2 )L ( n − r +1 ) =

• But in a set order is not important

n!

Cancelled bookings

Bag of bookings

– Is there a difference between the sets of cancelations:

{b , i1

47

bi 2 ,

L,

r ( r −1 )( r −2 ) L1 = r !

( n − r )!

b i n− r ,

b*1 ,

L

, b * r −1,

b* r

} 48

8

Usual Patterns

Usual Patterns

– n sits in plane and have r cancelations

– n sits in plane and have r cancelations

n!

• Another way of looking at the problem

n ( n −1) ( n − 2 )L ( n − r +1 ) =

r ( r −1 )( r −2 ) L1 = r !

( n − r )!

{b , i1

bi 2 ,

L,

– Group the left neighbours of a cancelation, which gives r!

Cancelled bookings

Bag of bookings

b*1 ,

b i n− r ,

Discounting the orders of cancelations

L

different orderings.

, b * r −1,

Total =

b* r

}

n! r !( n − r )! 49

Usual Patterns

n! 1 n ( − r )! r !

50

Usual Patterns

• Subsets : General expression – How many possibilities when: Sell n sits in plane and have r cancelations

• Properties of C(n,k): – Binomial Theorem

n n! C (n , k ) = = k (n − k )!k ! 1 11 121 1331 14641 1 5 10 10 5 1

• Read n choose k

51

Usual Patterns

52

Usual Patterns

• Properties of C(n,k):

• Properties of C(n,k):

– Pascal’s triangle

– Recursive Computation C(n,k) C(n,k)

n

1 11 121 1331 14641 1 5 10 10 5 1

1 2 3 4 5 6

n

1 11 121 13 31 14641 1 5 10 10 5 1 53

1 2 3 4 5 6 54

9

Usual Patterns

Usual Patterns

• Properties of C(n,k):

• Properties of C(n,k):

– Binomial Theorem/Important consequence

– Total number of subsets of a set. • General Expression

• Total number of subsets of a set.

(a + b)3 = 1a 3 + 3a2b + 3ab2 + b 3 =

3 3 a 0

{aab, aba, baa}

3 3 3 + a 2b + ab 2 + b3 1 2 3

n n n N = (1 + 1) = ∑ = 2 n k =1 k

3 3 3 3 Number Subsets = + + + 0 1 2 3 55

Usual Patterns

56

Usual Patterns

• Properties of C(n,k):

• Simple application:

– Total number of subsets of a set. • Another derivation

– Number of links in a graph

n N =1 222 42L 4 32 = 2

Present

n

Absent

{elem1,

elem2,

L

L

,elemn−1, elemn} http://mathworld.wolfram.com/CompleteGraph.html

57

Stirling’s Formula

58

Stirling Formula

• Justification of Stirling’s Formula

• Short derivation

– Helps to deal and compute enourmous numbers n!

(n − r )!

t=1:10;plot(t,log(t),'r - o');hold on;bar(t,log(t));hold off;plot(t,log(t),'r - x');axis tight

1 2 3 4 5 6 7 8 9 10

59

1 2 6 24 120 720 5040 40320 362880 3628800

60

Weisstein, Eric W. "Stirling's Approximation." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/StirlingsApproximation.html

10

Stirling Formula

Stirling Formula • Caveat

• Short derivation

n 1 2

n!

Stirling 1

Abs(Stirling -n!)

n!

Stirling /n!

0.922137

0.077863

1.08444

2

1.919

0.0809956

1.04221

3

6

5.83621

0.16379

1.02806

4

24

23.5062

0.493825

1.02101

5

120

118.019

1.98083

1.01678

720

6

– Division of enormous numbers

710.078

9.92182

1.01397

7

5040

4980.4

59.6042

1.01197

8

40320

39902.4

417.605

1.01047

9

362880

359537

3343.13

1.0093

10

3.63E+06

3.60E+06

30104.4

1.00837

t=1:10;plot(t,log(t),'r - o');hold on;bar(t,log(t));hold off;plot(t,log(t),'r - x');axis tight

1

(n − r )! r!

Important: •Ratio not difference !

61

62

Weisstein, Eric W. "Stirling's Approximation." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/StirlingsApproximation.html

Exercices

Exercices

• Ten persons select a paper each from an urn with ten papers. One paper has the prize. How many permutations are possible if the winner has made selection ‘i’.

• Secretary problem (dowry, perfect husband, etc.) – You have 20 candidates and examine them sequentially. – How many possibilities do you have of selecting the best, if your strategy is to examine the first 10, and then select the best of the last 10.

63

64

11