( x )= M ( x ) if M ( x ) terminatesundefinedotherwise. numberoftapecellsinalargestconfigurationof  Time ( x )= m if M ( x ) hasexactly m + 1 configurationsundefinedotherwise;Space  to N ,asfollows: Time and Space ,bothofwhichmapfrom Σ M ( x ) ,defin

(1)

Complexity Measures and Classes Deterministic Complexity Measures: Time and Space

Deterministic Time and Space Complexity Measures

Definition

LetMbe any DTM withL(M)⊆Σ^∗, and letx ∈Σ^∗ be any input. For the computationM(x), define thetime functionand thespace function, denoted respectively byTimeM andSpace_M, both of which map fromΣ^∗ toN, as follows:

Time_M(x) =







m ifM(x)has exactlym+1configurations undefined otherwise;

Space_M(x) =











number of tape cells in a largest

configuration ofM(x) ifM(x)terminates

undefined otherwise.

(2)

Blum’s Axioms

Letϕ₀, ϕ₁, ϕ₂, . . .be a fixed G ¨odelization (i.e., an effective enumeration) of all one-argument functions inIP, the class of all partial recursive (i.e., computable) functions.

LetIRbe the class of all total (i.e., everywhere defined) recursive functions.

LetΦ∈IPbe a function mapping fromN×Σ^∗ toN, and write Φi(x)as a shorthand forΦ(i,x).

We say thatΦis aBlum complexity measureif and only if the following two axioms are satisfied:

Axiom 1: For eachi ∈N,D_Φ_i =D_ϕ_i. Axiom 2: The set{(i,x,m)

Φ_i(x) =m}is decidable.

(3)

Deterministic Time and Space Complexity Measures

Definition (continued)

Define the functionstimeM :N→Nandspace_M :N→Nby:

time_M(n) =











x:|x|=nmax Time_M(x) if Time_M(x)is defined for eachx with|x|=n undefined otherwise;

space_M(n) =











x:|x|=nmax Space_M(x) if Space_M(x)is defined for eachx with|x|=n undefined otherwise.

(4)

Deterministic Time and Space Complexity Classes

Definition

Lettandsbe functions inIRmapping fromNtoN.

Define the followingdeterministic complexity classes with resource function t and s, respectively:

DTIME(t) =







A A=L(M)for some DTMMand, for eachn∈N, timeM(n)≤t(n)







;

DSPACE(s) =







A A=L(M)for some DTMMand, for eachn∈N, space_M(n)≤s(n)





 .

(5)

Deterministic Time and Space Complexity Classes

Remark:

Note that a deterministic Turing machineM decidesits language.

IfA=L(M)then bothtime_M(n)andspace_M(n)are defined for each n∈N.

In contrast, a nondeterministic Turing machineacceptsits language. Thus, the nondeterministic case is treated slightly differently.

The resource functionst ands in are called thenamesof DTIME(t)andDSPACE(s), respectively.

(6)

Deterministic Time and Space Complexity Classes

Remark:

IfMis a Turing machine with more than one tape, thenSpace_M(x), the size of “a largest configuration ofM(x),” is defined to be the maximum number of tape cells, where the maximum is taken both

over all tapes and

over all configurations in the computation.

If there is a separate read-only input tape, then only the space used on the working tapes is to be taken into account (reasonable due to sublinear space functions such as logarithmic space).

(7)

Complexity Measures and Classes Nondeterministic Complexity Measures: Time and Space

Nondeterministic Time and Space Measures

Definition

LetMbe any NTM withL(M)⊆Σ^∗, and letx ∈Σ^∗ be any input.

LetTime_M(x, α)andSpace_M(x, α)denote thetimeandspacefunctions for each pathαinM(x). For the computationM(x), define the

nondeterministictimeandspacefunction, denoted byNTime_M and NSpace_M, both of which map fromΣ^∗ toN, as follows:

NTime_M(x) =







M(x)accepts on pathmin αTime_M(x, α) ifx ∈L(M)

undefined otherwise;

NSpace_M(x) =







M(x)accepts on pathmin αSpace_M(x, α) ifx ∈L(M)

undefined otherwise.

(8)

Nondeterministic Time and Space Measures

Lettandsbe functions inIRmapping fromNtoN. We say thatM accepts a set A in time t if

for eachx ∈A, we haveNTimeM(x)≤t(|x|), and for eachx 6∈A,M does not acceptx.

We say thatM accepts a set A in space sif

for eachx ∈A, we haveNSpace_M(x)≤s(|x|), and for eachx 6∈A,M does not acceptx.

(9)

Nondeterministic Time and Space Complexity Classes

Define the followingnondeterministic complexity classes with resource function t and s, respectively:

NTIME(t) =







A A=L(M)for some NTMM that acceptsAin timet(n)







;

NSPACE(s) =







A A=L(M)for some NTMM that acceptsAin spaces(n)





 .

(10)

Complexity Measures and Classes Resource Functions

Resource Functions

Remark:

It is reasonable to consider collectionsF of “similar” resource functions and to define the complexity class corresponding toF by

DTIME(F) = [

f∈F

DTIME(f)

etc.

Such a collectionF contains all resource functions with asimilar rate of growth.

(11)

Complexity Measures and Classes Resource Functions

Resource Functions

Remark:

Consider the collections of functions mapping fromNtoNeach:

ILincontains all linear functions, IPolcontains all polynomials,

2^ILincontains all exponential functions whose exponent is linear inn, and

2^IPolcontains all exponential functions whose exponent is polynomial inn.

More generally, for any functiont :N→N, define the collection of all functions linear int(respectively, polynomial int) by:

ILin(t) = {f

f =`◦tand`∈ILin};

IPol(t) = {f

f =p◦t andp∈IPol}.

(12)

Complexity Measures and Classes Asymptotic Rate of Growth

Asymptotic Rate-of-Growth Notation

Definition

For functionsf andg mapping fromNtoN, define the following notation:

f(n)≤_aeg(n)to mean thatf(n)≤g(n)is true for all but finitely manyn∈N.

Analogously, the notations <ae, ≥ae, and >ae are defined.

The subscript “ae” of “≤_ae,” etc. stands for “almost everywhere.”

Similarly, the notationf(n)≤_iog(n)means thatf(n)≤g(n)is true for infinitely manyn∈N.

Analogously, the notations <io, ≥io, and >io are defined.

The subscript “io” of “≤io,” etc. stands for “infinitely often.”

(13)

Asymptotic Rate-of-Growth Notation

Definition

For functionsf andg mapping fromNtoN, define the following notation:

f ∈ O(g) ⇐⇒ there is a real constantc >0 such that f(n) +1 ≤_ae c·(g(n) +1).

f ∈o(g) ⇐⇒ for all real constantsc>0, f(n) +1 <_ae c·(g(n) +1).

(14)

Asymptotic Rate-of-Growth Notation

Definition

f g ⇐⇒ lim sup_n→∞ _g(n)+1^f⁽ⁿ⁾⁺¹ <∞.

Note thatf ∈ O(g) ⇐⇒ f g.

Intuitively,f g means that, by order of magnitude,f does not grow faster thang, with at most finitely many exceptions allowed.

f ≺g ⇐⇒ lim sup_n→∞ _g(n)+1^f⁽ⁿ⁾⁺¹ =0.

Note thatf ∈o(g) ⇐⇒ f ≺g.

Intuitively,f ≺g means that, by order of magnitude,g does grow strictly faster thanf, with at most finitely many exceptions allowed.

(15)

Asymptotic Rate-of-Growth Notation

Definition

f_iog ⇐⇒ lim infn→∞ f(n)+1 g(n)+1 <∞.

Intuitively,f_iog means that, by order of magnitude,f does not grow faster thang, at least not for infinitely many arguments.

f≺_iog ⇐⇒ lim infn→∞ f(n)+1 g(n)+1 =0.

Intuitively,f≺_iog means that, by order of magnitude,gdoes grow strictly faster thanf, at least for infinitely many arguments.

(16)

Asymptotic Rate-of-Growth Notation

Remark:

1 The additive constant 1 in the denominators of the limit

expressions above merely ensures that the denominator in _g(n)+1^f⁽ⁿ⁾⁺¹ is distinct from zero, so the quotient is well-defined.

2 For expressions such aslim sup_n→∞ _g(n)+1^f⁽ⁿ⁾⁺¹, the additive constant 1 in the enumerator prevents it from going to zero withoutg growing strictly faster thanf.

For example, the constant functions 0 and 2, which do not grow at all, should satisfy 020. Without the additive 1, however, we had 0≺2, which is not desirable.

(17)

Asymptotic Rate-of-Growth Notation

Definition Write

f gforg f, f gforg ≺f, f_iog forg_iof, and f_iog forg≺_iof.

(18)

Asymptotic Rate-of-Growth Notation

Example

1 logn≺n.

2 n2nand 2nn.

3 For

f(n) = n(nmod2) +n²((n+1) mod2) and g(n) = n²(nmod2) +n((n+1) mod2) we simultaneously havef≺_iog andg≺_iof.

4 If, in addition,h(n) =n², we simultaneously have f h and f≺_ioh and h_iof.

(19)

Complexity Measures and Classes Some Central Worst-Case Complexity Classes

Some Central Worst-Case Complexity Classes

Space classes

L = DSPACE(log) NL = NSPACE(log) LINSPACE = DSPACE(ILin) NLINSPACE = NSPACE(ILin) PSPACE = DSPACE(IPol) NPSPACE = NSPACE(IPol) EXPSPACE = DSPACE(2^IPol) NEXPSPACE = NSPACE(2^IPol)

(20)

Some Central Worst-Case Complexity Classes

Time classes REALTIME = DTIME(id)

LINTIME = DTIME(ILin) P = DTIME(IPol) NP = NTIME(IPol) E = DTIME(2ÎLin) NE = NTIME(2ÎLin) EXP = DTIME(2ÎPol) NEXP = NTIME(2ÎPol)

(21)

Polynomial versus Exponential Functions

t(n) n=10 n=20 n=30

n .00001 sec .00002 sec .00003 sec

n² .0001 sec .0004 sec .0009 sec

n³ .001 sec .008 sec .027 sec

n⁵ .1 sec 3.2 sec 24.3 sec

2ⁿ .001 sec 1.0 sec 17.9 min

3ⁿ .059 sec 58 min 6.5 years

Table:Comparing some functions (Garey & Johnson 1979)

(22)

Polynomial versus Exponential Functions

t(n) n=40 n=50 n=60

n .00004 sec .00005 sec .00006 sec

n² .0016 sec .0025 sec .0036 sec

n³ .064 sec .125 sec .256 sec

n⁵ 1.7 min 5.2 min 13.0 min

2ⁿ 12.7 days 35.7 years 366 centuries 3ⁿ 3855 centuries 2·10⁸centuries 1.3·10¹³centuries

Table:Comparing some functions (Garey & Johnson 1979)

(23)

What if the Computers Get Faster?

t_i(n) Computer 100 times 1000 times

today faster faster

t₁(n) =n N₁ 100·N₁ 1000·N₁ t₂(n) =n² N₂ 10·N₂ 31.6·N₂ t₃(n) =n³ N₃ 4.64·N₃ 10·N₃ t₄(n) =n⁵ N₄ 2.5·N₄ 3.98·N₄ t₅(n) =2ⁿ N₅ N₅+6.64 N₅+9.97 t₆(n) =3ⁿ N₆ N₆+4.19 N₆+6.29 Table:What if the computers get faster? (Garey & Johnson 1979)

(24)

Time versus Space for Deterministic Classes

Theorem

1 DTIME(t)⊆DSPACE(t).

2 DSPACE(s)⊆DTIME(2^ILin(s))if s≥log.

Proof:

1 The first statement is immediately clear, since in timetany Turing machine can move its heads by at mostt tape cells.

2 To prove the second statement, letMbe a DTM working in space s(n)and in timet(n). SupposeM has

qstates,

k working tapes and one input tape, and a working alphabet with`symbols.

(25)

Time versus Space for Deterministic Classes

For each inputx of lengthn,M’s time boundt(n)is bounded above by the number of distinct configurations ofM(x).

To see why, note that if there were one configuration occurring twice in the computation ofM(x), then sinceM works deterministically, it would loop forever and would never halt, a contradiction.

How many distinct configurations canM(x)have?There are q possible states,

npossible head positions on the input tape,

(s(n))^k possible head positions on thek working tapes, and

`^k·s(n)possible tape inscriptions.

(26)

Time versus Space for Deterministic Classes

Hence, there exist suitable positive constantsa,b, andcsuch that:

t(n) ≤ q·n·(s(n))^k ·`^k·s(n)

≤ q·2^logⁿ·2^a·s(n)

≤ q·2^b·s(n)

≤ 2^c·s(n),

where the third inequality uses the assumption thats≥log.

Thus,tis in 2^ILin(s). q

(27)

Best Wishes for Christmas!

(28)

Best Wishes for Christmas! Unless ... Well ...

Merkel

( x )= M ( x ) if M ( x ) terminatesundefinedotherwise. numberoftapecellsinalargestconfigurationof  Time ( x )= m if M ( x ) hasexactly m + 1 configurationsundefinedotherwise;Space  to N ,asfollows: Time and Space ,bothofwhichmapfrom Σ M ( x ) ,defin

Deterministic Time and Space Complexity Measures

Blum’s Axioms

Deterministic Time and Space Complexity Measures

Deterministic Time and Space Complexity Classes

Deterministic Time and Space Complexity Classes

Deterministic Time and Space Complexity Classes

Nondeterministic Time and Space Measures

Nondeterministic Time and Space Measures

Nondeterministic Time and Space Complexity Classes

Resource Functions

Resource Functions

Asymptotic Rate-of-Growth Notation

Asymptotic Rate-of-Growth Notation

Asymptotic Rate-of-Growth Notation

Asymptotic Rate-of-Growth Notation

Asymptotic Rate-of-Growth Notation

Asymptotic Rate-of-Growth Notation

Asymptotic Rate-of-Growth Notation

Some Central Worst-Case Complexity Classes

Some Central Worst-Case Complexity Classes

Polynomial versus Exponential Functions

Polynomial versus Exponential Functions

What if the Computers Get Faster?

Time versus Space for Deterministic Classes

Time versus Space for Deterministic Classes

Time versus Space for Deterministic Classes

Best Wishes for Christmas!

Best Wishes for Christmas! Unless ... Well ...

#«