Computability and computational complexity Lecture 2: Turing machines Ion Petre Computer Science, Åbo Akademi University Fall 2015

http://users.abo.fi/ipetre/computability/

October 26, 2015

http://users.abo.fi/ipetre/computability/

1

Content

n 

Algorithms – finite vs. infinite Turing machine (TM) basics TM as algorithms TM with multiple tapes

n 

Convention: we denote the empty word by 1, the blank space by #

n  n  n 

October 26, 2015

http://users.abo.fi/ipetre/computability/

2

ALGORITHMS: FINITE VS. INFINITE October 26, 2015

http://users.abo.fi/ipetre/computability/

3

A model for algorithms n 

To discuss about algorithmic solutions (or lack of them) for a given problem we need a formalization for “algorithm” q 

n 

What is an algorithm? A “natural” list of requirements: i.  ii.  iii.  iv. 

n 

Turing machines will be our formal model for algorithms in this course

it consists of a finite number of simple operations the instructions are applied deterministically the instructions are universal – they apply to each input instance the instructions are terminating: for each legal input, the result is obtained in a finite number of applications of operations

Without loss of generality we can assume that all algorithms have nonnegative integers as input and output q 

q 

any string can be seen as a nonnegative integer – the set S* of strings over the finite alphabet S is isomorphic with N, for any S an algorithm A calculates a function fA:N→N

October 26, 2015

http://users.abo.fi/ipetre/computability/

4

Formalizing the notion of algorithm n 

Assume that we have an arbitrary formalization for “algorithm”, that satisfies conditions i-iv q 

q 

n 

this means to have a precise definition for an algorithm it also means they can be effectively enumerated: A1, A2, A3, …, where effective means that for a given i, algorithm Ai can be found

Consider now the following algorithm D: q 

its input is a nonnegative integer

q 

for each input n, define D(n)=fAn(n)+1

q 

in other words: for an input n, select algorithm An, calculate its output on n and give its successor as an output of D

October 26, 2015

http://users.abo.fi/ipetre/computability/

Recall our “natural” requirements of an algorithm: i.  it consists of a finite number of simple operations ii.  the instructions are applied deterministically iii.  the instructions are universal – they apply to each input instance iv.  the instructions are terminating: for each legal input, the result is obtained in a finite number of applications of operations

5

The diagonalization argument: defining the output of D(n)

D(n)

A1

1

fA1(1)+1

2 3

A2

A3

…

An

…

fA2(2)+1 fA3(3)+1

…

…

n

fAn(n)+1

…

…

Note: algorithm D is not in our list! if D=Ar, for some r, then we should have fD(r)=fAr(r), but by definition, we have fD(r)=fAr(r)+1 October 26, 2015

http://users.abo.fi/ipetre/computability/

6

Discussion n 

Note: the technique above is called a “diagonalization argument” q 

Step ii. is natural (because of practical considerations) but nondeterministic algorithms are also useful, at least for theoretical considerations n 

q 

n 

more on this later in connection to NP

The escape is in considering also algorithms that do not satisfy step iv.

Conclusion: in any formalization of the notion of algorithm, we must allow partial, rather than total functions q  q  q 

allow algorithms that do not terminate for all legal inputs whenever they terminate, they give the correct answer indeed, this is what we have in Turing machines

October 26, 2015

http://users.abo.fi/ipetre/computability/

Recall our “natural” requirements of an algorithm: i.  it consists of a finite number of simple operations ii.  the instructions are applied deterministically iii.  the instructions are universal – they apply to each input instance iv.  the instructions are terminating: for each legal input, the result is obtained in a finite number of applications of operations

7

TURING MACHINES

October 26, 2015

http://users.abo.fi/ipetre/computability/

8

Turing machines: basic definition n 

Definition: A Turing machine is a quadruple M=(K,Σ, δ,s) q  q  q 

q 

K is a finite set of states s∈K is the initial state Σ is a finite set of symbols, called the alphabet of M; K∩Σ=∅; there are two special characters in Σ: the blank symbol ‘#’ and the first symbol ‘>’ δ : KxΣ → (K ∪ {h,’yes’, ’no’}) x Σ x {←,→,-} is the transition function, where: n  n  n  n  n 

q 

h is a halting state, ‘yes’ is an accepting state, ‘no’ is a rejecting state {←,→,-} are cursor directions assume that {h,’yes’,’no’, ←,→,-} ∩ (K ∪ Σ) = ∅ if δ(q,a)=(q’,a’,d), then we usually write (q,a)→(q’,a’,d) δ can be thought of as the “program” of the machine

(q,>)→(p,>,→) n 

Intuitively: the first symbol is never erased and the cursor move on it is always to the right

October 26, 2015

http://users.abo.fi/ipetre/computability/

9

Computation of a Turing machine n 

Start in the initial state s

n 

Input string is >, followed by a finite sequence x of symbols from Σ-{#}

n 

q 

we say that x is the input of the machine

q 

the cursor is pointing on >

From this initial configuration, the machine takes a step according to the transition function q 

enters a new state, rewrites the current symbol, moves the cursor

n 

It can never fall-off the left-end of the tape

n 

It can go the right arbitrarily much q 

n 

it will read a blank space and handle it according to δ

The machine can only stop if it reaches h, ‘yes’, or ‘no’ q 

we say in this case that the machine has halted

q 

if the state is “yes” we say that the machine accepts its input; output is “yes”

q 

if the state is “no” we say that the machine rejects its input; output is “no”

q 

if the state is h, then its output is the current string of M n 

q 

if >y###… is the current string of M, where last(y) ≠ #, then M(x)=y;

if the machine does not halt, then we write M(x)=↑

October 26, 2015

http://users.abo.fi/ipetre/computability/

10

Discussion n 

Turing machines allow non-terminating computations

n 

TM defined with strings as inputs q 

can easily consider a number as input through its n-aric representation (decimal, binary, etc)

October 26, 2015

http://users.abo.fi/ipetre/computability/

11

Example: right transfer machine n 

Input: aba

6. 

s, >aba q, >aba qa, >#ba qb, >#aa qa, >#ab# h, >#aba

n 

Output: #aba

p∈K

σ∈Σ

δ(p,σ)

1. 

s

>

(q,>,→)

2. 

q

a

(qa,#,→)

3. 

qa

b

(qb,a,→)

4. 

qa

#

(h,a,-)

5. 

Useful trick to remember: using the states as a finite memory

October 26, 2015

http://users.abo.fi/ipetre/computability/

12

Example: binary successor function n 

Input: 010

(s,0,→)

1. 

(s,1,→)

2. 

s, >010 s, >010 s, >010 s, >010 s, >010 # q, >010 # h, >011#

p∈K

σ∈Σ

δ(p,σ)

s

>

(s,>,→)

s

0

s

1

3. 

n 

Input: 11

1. 

s, >11 s, >11 s, >11 s, >11# q, >11# q, >10# q, >00# t, >00# t’, 10# t’, 10# h, 100 Output: 100

2.  3. 

s

#

(q,#,←)

q

0

(h,1,-)

q

1

(q,0,←)

q

>

(t,>,→)

t

0

(t’,1,→)

t’

0

(t’,0,→)

t’

#

(h,0,-)

11. 

t

#

(h,#,-)

n 

October 26, 2015

4.  5.  6.  7. 

4.  5.  6.  7.  8.  9. 

n 

Output: 011

http://users.abo.fi/ipetre/computability/

10. 

13

Configuration n 

Definition: A configuration is (q,w,u) where: q  q 

q 

n 

Definition: Configuration (q,w,u) yields configuration (q’,w’,u’) in one step, denoted (q,w,u)→(q’,w’,u’) if: q  q  q  q 

n 

q∈K, w is the string to the left of the cursor, including the symbol scanned by the cursor, u is the string to the right of the cursor

δ(q,a)=(q’,a’,→) and w=xa, u=by, w’=xa’b, u’=y δ(q,a)=(q’,a’,→) and w=xa, u=1, w’=xa’#, u’=1 δ(q,a)=(q’,a’,←) and w=xba, w’=xb, u’=a’u δ(q,a)=(q’,a’,-) and w=xa, w’=xa’, u’=u

Define “yields” as a transitive closure of “yields in one step”: q 

configuration (q,w,u) yields (q’,w’,u’) if there is n≥0 and configurations (qi,wi,ui), 1≤i ≤n such that n  n 

(q,w,u)=(q0,w0,u0), (q’,w’,u’)=(qn,wn,un) (qi,wi,ui)→(qi+1,wi+1,ui+1), for all 1≤i ≤n-1

October 26, 2015

http://users.abo.fi/ipetre/computability/

14

Example: palindrome checker n 

n 

A string is a palindrome if it consists of the same sequence of letters left-to-right and right-to-left A Turing machine for deciding whether its input is a palindrome or not: q 

q  q 

q 

q 

remember the first symbol, transform it into #, travel to the right end and compare it to the last symbol if they do not coincide, halt with answer ‘no’ if they coincide, change the last symbol into # and travel to the left searching for the first non-blank letter repeat the procedure above until a string of length 0 or 1 is left, in which case halt with answer ‘yes’ Skip the detailed formulation of the machine

October 26, 2015

http://users.abo.fi/ipetre/computability/

15

Turing machines as algorithms n 

Let L⊆(Σ-{#})*

n 

A Turing machine decides L iff for every x∈ (Σ-{#})*, q  q  q 

n 

if x∈L, then M(x)=‘yes’ if x∉L, then M(x)=‘no’ In this case, we say that L is a recursive language

A Turing machine accepts L iff for every x∈ (Σ-{#})*, x∈L iff M(x)=‘yes’ q 

if x∉L, M might not terminate on x n 

q  q 

n 

we could ask that M(x)=↑ for all x∉L

In this case we say that L is a recursively enumerable language Note: any recursive language is recursively enumerable

A Turing machine computes a function f : (Σ-{#})* → Σ* iff for every x∈ (Σ-{#})*, M halts on x and M(x)=f(x) q 

In this case we call f a recursive function

October 26, 2015

http://users.abo.fi/ipetre/computability/

16

Solving problems with Turing machines n 

The instance of the problem (e.g., a graph) is given a string representation

n 

An algorithm for a decision problem is a TM that decides the corresponding language q 

n 

accept if the input represents a ‘yes’ instance of the problem, reject otherwise

An algorithm for an optimization problem is a TM that computes the corresponding function from strings to strings q 

the output is similarly represented as a string

October 26, 2015

http://users.abo.fi/ipetre/computability/

17

Discussion: representations n 

Any “finite” mathematical object can be represented as a finite string q  q  q  q 

n 

Note: representation of numbers q 

q 

n 

elements of a finite set: integers in binary tuples of elements: integers separated by commas sets: use commas and parenthesis graphs: adjacency matrix, with rows separated by a special character

unary representation of a number requires exponential space with respect to the binary representation of the same number we always assume in this course binary representations of numbers

We always assume a “reasonably” succinct representation of input q 

important in judging the complexity of a problem

October 26, 2015

http://users.abo.fi/ipetre/computability/

18

MULTI-TAPE TURING MACHINES October 26, 2015

http://users.abo.fi/ipetre/computability/

19

Turing machines with multiple tapes/strings n 

Definition: A k-tape Turing machine is a quadruple M=(K,Σ, δ,s) q 

K is a finite set of states

q 

s∈K is the initial state

q 

q 

Σ is a finite set of symbols, called the alphabet of M; K∩Σ=∅; there are two special characters in Σ: the blank symbol ‘#’ and the first symbol ‘>’ δ : KxΣk→ (K ∪ {h,’yes’, ’no’}) x (Σ x {←,→,-})k is the transition function n 

n  n 

Intuitively: the machine has now k tapes, with a string and an own cursor on each, the cursors can move independently of each other Similarly as for 1-tape TMs, on each tape the first symbol is > the first symbol on each tape is never erased and the cursor move on it is always to the right

October 26, 2015

http://users.abo.fi/ipetre/computability/

20

Example: more efficient palindrome check n 

Two-tape TM q  q  q 

copy the input to the second tape move the cursor of the second tape on the right-most symbol move the two cursors in opposite directions and check the equality

October 26, 2015

http://users.abo.fi/ipetre/computability/

21

Configurations, transitions of k-tape TM n 

Configuration: a (2k+1)-tuple (q,w1,u1,…,wk,uk), where q  q 

n 

Transition: (q,w1,u1,…,wk,uk)→(q’,w’1,u’1,…,w’k,u’k) defined as follows: q  q 

q  q  q  q 

n 

q is the current state wiui is the current content of the i-th tape with the last symbol of wi holding the cursor denote ai the last symbol of wi, for all 1≤i≤k assume δ(q,a1,…, ak)=(p,a’1,D1,…,a’k,Dk) Di = → and wi=xi ai, ui=bi yi, w’i =xia’i bi, u’=yi Di=→ and wi=xiai, ui=1, w’ i=xia’i*, u’i=1 Di=← and wi=xibiai, w’ i=xibi, u’ i=a’iui Di=- and wi=xiai, w’ i=xia’ i, u’ i=ui

The “yields in n steps”, denoted →n, and “yields”, denoted →*, defined as before

October 26, 2015

http://users.abo.fi/ipetre/computability/

22

Computations with k-tape TM n 

Start of the computation: (s,>,x,>,1,…,>,1) q  q 

the first tape holds the input with the symbol > in front of it; all other tapes hold just the first symbol >

n 

If (s,>,x,>,1,…,>,1) →*(‘yes’,w1,u1,…,wk,uk), then we say that M(x)=‘yes’

n 

If (s,>,x,>,1,…,>,1) →*(‘no’,w1,u1,…,wk,uk), then we say that M(x)=‘no’

n 

If (s,>,x,>,1,…,>,1) →*(h,w1,u1,…,wk,uk), then we say that M(x)=y, where y is obtained from wkuk by removing the first symbol >, as well as the largest (infinite) suffix formed just by #

n 

With these definitions, acceptance, decision, and computation are defined as before

October 26, 2015

http://users.abo.fi/ipetre/computability/

23

Time n 

If (s,>,x,>,1,…,>,1) →n (H,w1,u1,…,wk,uk), for some H∈{h,’yes’,’no’}, then the time required by M on input x is n

n 

If M(x)=↑, then the time required by M on x is ∞

n 

Time bounds for a machine M q 

n 

Let f:N→N. We say that M operates within time f(n) if for any input string x, the time required by M is at most f(|x|)

If a language L is decided by a multi-tape TM operating in time f(n), then we say that L∈TIME(f(n)) q  q 

TIME(f(n)) is called a complexity class It is a set of languages

October 26, 2015

http://users.abo.fi/ipetre/computability/

24

Example n 

Recall the 1-tape TM for checking whether a string of length n is a palindrome q  q 

q  q  q  q 

q  q 

n 

it needs ⎡n/2⎤ stages first stage: 2n+1 steps (transitions) to compare the first and the last symbol and come back to the beginning of the string second stage: same thing for a string of length n-2 third stage: same for a string of length n-4, etc. Calculate (2n+1)+(2n-3)+(2n-7)+…=(n+1)(n+2)/2 Conclusion: the language of all palindromes is in TIME((n+1)(n+2)/2) Note: the computation may halt much faster, for example for input 01n-1 Note: the 2-tape TM for checking palindromes takes time at most f’(n)=3n+3 à a quadratic speed-up

Question: How much speed-up can be gained through multi-tapes?

October 26, 2015

http://users.abo.fi/ipetre/computability/

25

Multi-tape vs. single-tape TMs n 

Theorem. For any k-tape Turing machine M operating within time f(n) we can construct a (single-tape) TM M’ operating within time O(f2(n)) such that for any input x, M(x)=M’(x)

n 

Proof: M=(K,Σ,δ,s). We build M’=(K’ ,Σ’,δ’,s) as follows q 

q  q 

maintain in the unique string of M’ the concatenation of the k strings of M, delimited by some markers remember the position of the k cursors of M simulate the functioning of M

October 26, 2015

http://users.abo.fi/ipetre/computability/

26

Multi-tape vs single-tape TM: proof (continued) q 

q 

Let Σ’= Σ∪Σ∪{>’,>’, ’, >’, < are new symbols M’ represents a configuration (q,w1,u1,…,wk,uk) of M through a configuration (q,>, w’1u1