INFORMATION ~'~ND CONTROL 9, 6 0 2 - 6 1 9

(1966)

The Definition of Random Sequences PER h~ARTI,N'-L(JF

Institute of Mathematical Statistics, Unh'ersity of Stockholm, Stockholm, Sweden Kolmogorov has defined tile conditional complexity of an object y when the object x is already given to us as the minimal length of a binary program which by means of x computes y on a certain asymptotically optimal machine. On the basis of this definition he has proposed to consider those elements of a given large finite population to be random whose complexity is maximal. Almost all elements of the population have a complexity which is close to the maximal value. In this paper it is shown that the random elements as defined by Kolmogorov possess all conceivable statistical properties of randomness. They can equivalently be considered a s the elements which withstand a certain universal stochasticity test. The definition is extended to infinite binary sequences and it is shown that the non random sequences form a maximal constructive null set. Finally, the Kollektivs introduced by yon Mises obtain a definition which seems to satisfy all intuitive requirements. I. THE COMPLEXITY MEASURE OF KOLMOGOROV C o n s i d e r t h e set of all w o r d s o v e r s o m e finite a l p h a b e t . T h e l e n g t h n of such a s t r i n g x = ~i~2 - . . ~, will b e d e n o t e d b y l ( x ) . L e t A b e a n algor i t h m t r a n s f o r m i n g finite b i n a r y sequences i n t o w o r d s o v e r s o m e finite a l p h a b e t . W e s u p p o s e t h a t t h e a l g o r i t h m c o n c e p t h a s b e e n m a d e precise in o n e of t h e v a r i o u s e q u i v a l e n t w a y s t h a t h a v e b e e n p r o p o s e d , e.g. b y m e a n s of t h e t h e o r y of p a r t i a l r e c u r s i v e functions. F o l l o w i n g K o l n m g o r o v we define t h e c o m p l e x i t y of t h e e l e m e n t x w i t h r e s p e c t to t h e a l g o r i t h m A as t h e l e n g t h of t h e s h o r t e s t p r o g r a m w h i c h c o m p u t e s it,

KA(x) = minl(p). I f t h e r e is no such p r o g r a m , i.e. A ( p ) ~ x for all b i n a r y s t r i n g s p, w e p u t K.~(x) -- -t- oo. T h i s c o m p l e x i t y m e a s u r e d e p e n d s in a n e s s e n t i a l w a y on t h e b a s i c a l g o r i t h m A . W e a l m o s t g e t rid of t h i s d e p e n d e n c e b y 602

DEFINITION OF RANDOM SEQUENCES

603

means of the following theorem, proved by Kohnogorov and Solomonoff (196~). There exists an algorithm A such that for any algorithm B KA(x) =< K , ( x ) q- c, where c is a conslanl (dependent on A and B but nol on x). Such an algorithm is called asymptotically optimal by ]Kolmogorov and universal by Solomonoff. The complexity of x with respect to a fixed algorithm of this type we shall call simply the complexity of x and denote by K ( x ) . In an analogous way we can introduce the concept of conditional complexity. To do this, let p, x --->A (p, x) = y bc an algorithm of two variables, where p is a finite binary sequence, called the program, x a string over some alphabet, and y a word ovei" a possibly different alphabet. The quantity KA(y Ix) = minl(p) will be called the conditiolml complexity of y given x with respect to A. There exists an algorithm A such that, for an arbitrary algorithm B, Ka(ylx)

h(m), where h is a suitable nondecreasing general recursire function, an explicit definition of which we could evidently write down with soine effort. A comparison with the universal test completes the proof. Note that, by the law of large numbers, all real numbers 0 (not only computable ones) occur as limit frequencies, lira-s" = O,

0 =< 0 =< 1.

We finally state the analogue of the last theorem of the previous section, the idea of the proof being the same. The limit fl'equency cannot vanish,

lhn s, = 0, unless ~ = 0 for all n.

This theorem is important since, in the ease of an experiment with an arbitrary finite number of outcomes, it allows us to reduce the sample space by excluding those outcomes whose limil~ frequencies equal zero. More suggestively, an event with vanishing limit frequency is actually impossible. This contrasts sharply with the conception of yon Mises, who explicitly stated t h a t the opposite might occur. It seems as if he strained his seldom failing intuition on this point in order not to conflie~ with his somewhat arbitrary definition of randomness. RECEIVEn: April 1, 1966 REFERENCES I~OLMOGOROV,A. N., (1965), Tri podhoda k opredeleniju ponjatija "koli~estvo informacii." Problemy peredaSi informacii 1, 3-11. SOLOX~ONOFF,R. J., (196t), A formal theory of inductive inference. Part I. Inform. Control 7, 1-22.