Computability and computational complexity Lecture 3: Linear speed-up; Space complexity; Nondeterministic complexity Ion Petre Computer Science, Åbo Akademi University

Fall 2015

http://users.abo.fi/ipetre/computability/

November 1, 2015

http://users.abo.fi/ipetre/computability/

1

Content n 

Linear speed-up

n 

Space complexity

n 

Nondeterministic TM

n 

Nondeterministic time and space complexity

November 1, 2015

http://users.abo.fi/ipetre/computability/

2

LINEAR SPEED-UP

November 1, 2015

http://users.abo.fi/ipetre/computability/

3

Linear speed-up n 

Next result: the rate of growth of the time/space requirements is important, but the multiplicative constants are not

n 

Theorem. Let L∈TIME(f(n)). Then for any ε>0, L∈TIME(f’(n)), where f’(n)=εf(n)+n+2

n 

Proof sketch. q 

q  q 

q  q 

Let M=(K,Σ,δ,s) be a k-tape machine deciding L in time f(n). Construct a k’-tape machine M’ =(K’,Σ’,δ’,s’) deciding L in time f’(n) We need at least 2 tapes. If k=1, then k’=2, otherwise k’=k Main trick: increase the “word length”, i.e., each symbol of M’ encodes several symbols of M and each move of M’ several moves of M Let m be a constant (that we define later) Setup: read the input left to right and compress it on the second tape: first m letters encoded by a single character of M’, the following m letters encoded by the second character, etc. Consider padding with #s if the input length is not a multiple of m

November 1, 2015

http://users.abo.fi/ipetre/computability/

4

Linear speed-up n 

Proof (continued) q  q 

A stage: M’ will simulate m steps of M in at most 6 steps of its own In each stage M’ reads the characters to the left and to the right of the cursor (one move to the left, two to the right, one to the left) and remembers the content into its state n 

n 

q  q 

it can now predict the next m moves of M because within m moves M cannot affect more than m symbols either to the left or to the right of its cursors implement all m moves in two steps of M’ (changes in the current tuple and possibly in one of its neighbors)

Once M accepts/rejects, M’ does the same Total time spent on input x in M’ is at most |x|+2+6⎡f(|x|)/m⎤ n 

enough to take m=⎡6/ε⎤

November 1, 2015

http://users.abo.fi/ipetre/computability/

5

Consequences of the linear speedup theorem n 

TMs operating in less than n steps are anyway not interesting q 

they should read their whole input, so f(n)≥n

n 

If f(n) is linear, say f(n)=an+b, then, according to the theorem, the constant a can be made arbitrarily close to 1

n 

If f(n) is super-linear, say f(n)=14n2+31n, then the constant 14 can be made arbitrarily small

n 

Conclusion: If L is a polynomially decidable language, then L is in TIME(nk), for some k>0

n 

Definition: The set P of all languages decided by Turing machines in polynomial time is P=∪k≥1TIME(nk)

November 1, 2015

http://users.abo.fi/ipetre/computability/

6

SPACE COMPLEXITY

November 1, 2015

http://users.abo.fi/ipetre/computability/

7

Space complexity n 

Goal: define a measure of complexity in terms of space

n 

Note: tapes cannot become shorter during computation q 

n 

also, an “interesting” machine should read all its input

We can define the space complexity either through the sum of the space used by the end of the computation on all tapes, or through the max space on any one of the tapes q 

the two measures only differ through a constant

n 

Space complexity should capture the amount of the extra “work space” and not count the space needed to get the input and the space used to write the output

n 

Idea: how about treating the input tape as a read-only tape and the output tape as a write-only tape and ignoring their length from the estimate of the space complexity

November 1, 2015

http://users.abo.fi/ipetre/computability/

8

Turing machines with input and output n 

Definition. A k-tape Turing machine with input and output is a k-tape TM with the following restriction on the transition function: if δ(p,a1,…,ak)=(q,b1,D1,…,bk,Dk), then q  q  q 

b1=a1 (read-only input tape) if a1=#, then D1=← (end of input, do not create more space on the first tape) Dk≠← (write-only output tape)

n 

Question: Do we lose computing power by restricting to TMs with input/output?

n 

Proposition. For any k-tape Turing machine M operating within time bound f(n) there is a (k+2)-tape TM M’ with input and output operating within time bound O(f(n)) Proof sketch q  copy the input to the second tape q  simulate M on tapes 2 through k+1 q  when M halts, copy the output to tape k+2

November 1, 2015

http://users.abo.fi/ipetre/computability/

9

Space consumption n 

Let M be a k-tape TM and x an input with (s,>,x,>ε,…,>,ε)→(H,w1,u1, …,wk,uk), where H is a halting state. q 

n 

The space required by M on input x is ∑ki=1|wiui|

If M is a TM with input and output, then the space required by M on x is

∑k-1i=2|wiui| n 

For f:N→N, the TM M is said to operate within space bound f(n) if for any input x it requires space at most f(|x|)

November 1, 2015

http://users.abo.fi/ipetre/computability/

10

Space complexity n 

We say that a language L is in the space complexity class SPACE(f(n)) if there is a TM with input and output that decides L and operates within space bound f(n) q 

Notation: L=SPACE(log(n))

n 

There is a linear speed-up theorem also for space complexity:

n 

Theorem. Let L be a language in SPACE(f(n)). Then for any ε>0, L∈SPACE(2+εf(n)). q 

In other words, constants can be ignored in space complexity too

November 1, 2015

http://users.abo.fi/ipetre/computability/

11

NONDETERMINISTIC TURING MACHINES November 1, 2015

http://users.abo.fi/ipetre/computability/

12

Nondeterministic Turing machines n  n 

An unrealistic model of computation Show that they can be simulated by deterministic machines q  q 

n 

exponential loss of efficiency can we really not do better?

Definition: A nondeterministic Turing machine is a quadruple M=(K,Σ, δ,s), where K, ∑, s are like before and q 

δ ⊆ (KxΣ) × [(K ∪ {h,’yes’, ’no’}) x Σ x {←,→,-}] is the transition relation n 

n 

in other words, for each state-symbol combination, there may be more than one legal move compare with the deterministic TM: δ : KxΣ → (K ∪ {h,’yes’, ’no’}) x Σ x {←,→,-} is the transition function

November 1, 2015

http://users.abo.fi/ipetre/computability/

13

Nondeterministic TM n 

Definition: A configuration is (q,w,u) where: q  q  q 

n 

Definition: Configuration (q,w,u) yields configuration (q’,w’,u’) in one step, denoted (q,w,u)→(q’,w’,u’) if: q  q  q  q  q 

n 

q∈K, w is the string to the left of the cursor, including the symbol scanned by the cursor, u is the string to the right of the cursor

w=xa, u=by, w’=xa’b, u’=y and [(q,a),(q’,a’,→)]∈δ w=xa, u=1, w’=xa’*, u’=1 and [(q,a),(q’,a’,→)} ∈δ w=xba, w’=xb, u’=a’u and [(q,a),(q’,a’,←)] ∈δ w=xa, w’=xa’, u’=u and [(q,a),(q’,a’,-)] ∈δ Note: compare with the correspondent notion for deterministic TM

Define “yields” as a transitive closure of “yields in one step”: q 

configuration (q,w,u) yields (q’,w’,u’), denoted (q,w,u)→*(q’,w’,u’) if there is n≥0 and configurations (qi,wi,ui), 1≤i ≤n such that n  n 

(q,w,u)=(q0,w0,u0), (q’,w’,u’)=(qn,wn,un) (qi,wi,ui)→(qi+1,wi+1,ui+1), for all 1≤i ≤n-1

November 1, 2015

http://users.abo.fi/ipetre/computability/

14

Nondeterministic computation

Time

“no”

“no”

“no”

November 1, 2015

“yes”

“yes”

“no”

http://users.abo.fi/ipetre/computability/

“no”

“yes”

15

Computing with a nondeterministic TM Deterministic TM n  n 

Nondeterministic TM

Let L⊂(Σ-{#})* A Turing machine decides L iff for every x∈ (Σ-{#})*, q  q 

n  n 

if x∈L, then M(x)=‘yes’ if x∉L, then M(x)=‘no’

Let L⊂(Σ-{#})* A nondeterministic Turing machine decides L iff for every x∈ (Σ-{#})*, q  q 

n 

A Turing machine accepts L iff for every x∈ (Σ-{#})*, x∈L iff M(x)=‘yes’ q 

if x∉L, M might not terminate on x n 

we could ask that M(x)=↑ for all x∉L

n 

if x∈L, then there is a ‘yes’-computation starting from x if x∉L, then all computations starting from x end in “no”

A nondeterministic Turing machine accepts L iff for every x∈ (Σ-{#})*, x∈L if and only if (s,>,x) →*(“yes”,w,u)

n 

A Turing machine computes a partial (string) function f : (Σ-{#})* → Σ* iff for every x∈ (Σ-{#})*, M(x)=f(x) q 

In this case we call f a recursive function

November 1, 2015

q 

q 

if x∈L, then there is a ‘yes’-computation starting from x if x∉L, some of the computations starting from x might not halt; those that halt end in “no”

http://users.abo.fi/ipetre/computability/

16

Discussion n 

An input is accepted if there is some sequence of nondeterministic choices that lead to “yes” q 

Many other computations on the same input might exist, some of them maybe even non-halting

n 

An input is rejected if there are no accepting computations

n 

Note the asymmetry in the definitions of accepting/rejecting Similar asymmetry in defining the accepting for a deterministic TM

n 

November 1, 2015

http://users.abo.fi/ipetre/computability/

17

Nondeterministic time complexity n 

Definition: A nondeterministic TM decides language L in time f(n) where f:N→N, if q  q 

n 

N decides L for any x∈∑*, if (s,>,x)→k(q,u,v), then k≤f(|x|)

Discussion q 

q 

any computation must halt in at most f(n) steps, where n is the length of the input there may be exponentially many computations to explore n  n 

n 

n 

to say “yes”, one single “witness” is enough to say “no”, all witnesses must be checked

Definition: The set of languages decided by nondeterministic TM within time f is denoted as NTIME(f(n)) In particular, NP=∪k≥1NTIME(nk) q 

Note: P⊆NP

November 1, 2015

http://users.abo.fi/ipetre/computability/

18

Example: TSP(D): is there a tour of budget at most B n 

Not known whether it is in P

n 

It is clearly in NP: can be decided by a nondeterministic TM in O(n2)

n 

Algorithm: q  q 

q 

q  q 

consider a 2-tape nondeterministic TM for a given input with n cities, write an arbitrary sequence of letters of length n on the second tape check (deterministically!) whether the string is a permutation of the cities and if so, whether it is a tour of cost at most B. Answer “yes” if this is the case, “no” otherwise Needs at most O(n2) steps

November 1, 2015

http://users.abo.fi/ipetre/computability/

19

Example: TSP(D): is there a tour of budget at most B - continued n 

The algorithm on the previous slide decides TSP(D) q 

q 

It accepts the input iff its associated TSP(D) problem has answer “yes”: if x∈TSP(D), then there is a tour of cost at most B and on the branch that generates that tour, the nondeterministic TM will halt in the accepting state n  n 

q 

many other tours and even non-sense strings might be checked as well not more than O(n2) steps on each branch

if x∉TSP(D), then all branches will end in the rejecting state

November 1, 2015

http://users.abo.fi/ipetre/computability/

20

From nondeterministic to deterministic Turing machines

November 1, 2015

http://users.abo.fi/ipetre/computability/

21

From non-deterministic to deterministic n 

Question: Can nondeterministic Turing machines be simulated by deterministic Turing machines? q  q 

n 

Answer: YES!! Catch: but with an exponential slow-down

The core of the P=NP? problem: can it be done with just a polynomial slow-down?

November 1, 2015

http://users.abo.fi/ipetre/computability/

22

From nondeterministic to deterministic TM n 

Theorem. Suppose that a language L is decided by a nondeterministic Turing machine N in time f(n). Then it is decided by a 3-tape deterministic Turing machine M in time O(cf(n)), where c>1 is some constant depending on N. In other words:

∪c>1TIME(cf(n))

NTIME(f(n)⊆

November 1, 2015

http://users.abo.fi/ipetre/computability/

23

Proof n 

Let N=(K,∑,δ,s) be a NTM

n 

Let d be the degree of nondeterminism of N – the maximum number of nondeterministic choices in any state-symbol pair in δ q 

in some configurations there is only one choice

n 

Any computation of N is a sequence of nondeterministic choices, i.e., a sequence of integers in the range 0…d-1

n 

Idea: simulate first all computation branches of length 1, then all those of length 2, etc.

November 1, 2015

http://users.abo.fi/ipetre/computability/

24

Proof – continued n 

Simulating all computation branches of N of length at most t with a deterministic TM M: q 

q 

q 

n 

When do we reject? q 

n 

consider all sequences of integers 0…d-1 in increasing order of their length and simulate N on each such sequence of nondeterministic choices write the current sequence on a second tape; the next one is the successor of the current sequence when seen as a number in base d if N answers “yes” on any such sequence of choices, then M halts with “yes”

whenever there is no continuing computation of the current length

Time complexity for M q  q 

number of sequences to examine: ∑t=1f(n) dt = O(df(n)+1) generating and simulating each sequence: clearly less than O(2f(n))

November 1, 2015

http://users.abo.fi/ipetre/computability/

25

Nondeterministic space complexity

November 1, 2015

http://users.abo.fi/ipetre/computability/

26

Nondeterministic space complexity n 

The definition of NTM with input/output is similar to the corresponding notion for TM q  q 

n 

first tape read-only last tape write-only

Definition. Given a k-tape nondeterministic TM with input and output N=(K,∑,δ,s), we say that N decides L within space f(n) if q  q 

N decides L and for any x∈(∑-{#})*, if (s,>,x,>,1,…,>,1)→*(q,w1,u1,…,wk,uk), then ∑k-1i=2|wiui|≤f(n) n  n  n 

this is similar as for deterministic TM it applies even to non-accepting computations N may not even halt on all computations

November 1, 2015

http://users.abo.fi/ipetre/computability/

27

Example: REACHABILITY n 

REACHABILITY q  q 

n  n 

The deterministic algorithm in Lecture 1 required O(n) space REACHABILITY is nondeterministically decidable within space O(log n) q 

q  q  q 

n 

use two more tapes – in one of them write the current node i in binary (initially node 1), in the other “guess” an integer 1