iRRAM (GMP/MPFR)

Martin Ziegler

Asymptotic Running Times

Running times of some sorting algorithms

BubbleSort: O(n²) comparisons and copy instr.s QuickSort: typically O(n—log n) steps

but O(n²) in the worst-case

HeapSort: always at most O(n—log n) operations

Here: focus on worst-case considerations!

w.r.t. input size (e.g. bit length) =: n → ∞

32 years 4 months

19 days

≈2.5min 100 000

11 days

≈1 day 1.5 days

≈2min 10 000

17min 17min

≈3h

≈1.5min 1000

40 Mrd. Y 1sec

10sec 11min

≈1min 100

1msec 1msec

0.1sec 33sec

33sec 10

2ⁿnsec n³ µsec

n² msec n—log n sec

log₂n —10s n

Martin Ziegler

P P P P P P P

P := { := { L L ⊆ ⊆ { { 0 0 , , 1 1 }* }*

decidabledecidable in in polynomialpolynomial time time

} }

⊆ ⊆ NP NP NP NP NP NP NP NP := { := { L L

verifiableverifiable in in polynomialpolynomial time time

} }

⊆ ⊆ PSPACE PSPACE PSPACE PSPACE PSPACE PSPACE PSPACE PSPACE := { := { L L

decidabledecidable in in polynpolyn. . spacespace

} }

⊆ ⊆ EXP EXP EXP EXP EXP EXP EXP EXP := { := { L L

decidabledecidable in time 2in time 2^pp(n(n))

} }

(Discrete) Complexity Classes

''Definition:Definition:' ' AlgorithmAlgorithm

A A

decidesdecides setset

L L ⊆ ⊆ { { 0 0 , , 1 1 }* }*

ifif

•• on on inputsinputs

x x ∈ ∈ L L

printsprints 11 and and terminatesterminates,,

•• on on inputsinputs

x x ∉ ∉ L L

printsprints 00 and and terminatesterminates..

Def Def : : A A

runsruns in in polynompolynom. time. time ifif

∃ ∃ p p ∈ ∈

NN

[ [ N N ] ]

: : on on inputinput

x x ∈ ∈ { { 0 0 , , 1 1 } }

ⁿⁿ makesmakes at at mostmost

p p ( ( n n ) )

stepssteps

/ / usesuses at at mostmost

p p ( ( n n ) )

bitsbits of of memorymemory.. //spacespace

Example:

Example: L L ={ ={ 10 10 , , 11 11 , , 101 101 , , 111 111 , , 1011 1011 , , 1101 1101 , , … … } }

Martin Ziegler

Example Problems (I)

Def Def : : A A Boolean Boolean term term Φ Φ ( ( Y Y

₁₁

, , … … Y Y

_nn

) ) is is composed composed from from variables variables Y Y

₁₁

, , … … Y Y

_nn

, ,

constants

constants 0 0 and and 1 1 , and , and operations

operations ∨ ∨ , , ∧ ∧ , , ¬ ¬ . .

EVAL EVAL : : Given Given 〈 〈 Φ Φ ( ( Y Y

₁₁

, , … … Y Y

_nn

) ) 〉 〉 and and y y

₁₁

, , … … y y

_nn

∈ ∈ {0,1} {0,1} , , does does Φ Φ ( ( y y

₁₁

, , … … y y

_nn

) ) evaluate evaluate to to 1 1 ? ?

[ [ k k - - ] SAT ] SAT : : Given Given Φ Φ ( ( Y Y

₁₁

, , … … Y Y

_nn

) ) [in [in k k - - CNF CNF ], ],

does does it it hold hold ∃ ∃ y y

₁₁

, , … … y y

_nn

∈ ∈ {0,1} {0,1} : : Φ Φ ( ( y y

₁₁

, , … … y y

_nn

)=1 )=1 ? ?

∈ P P P P P P P P

suitably

suitably encoded encoded over over { { 0 0 , , 1 1 }* }*

Φ Φ in in 3 3 - - CNF CNF if if Φ Φ = = ∧ ∧ ^{( ( ( ( ¬ ¬ ) ) y y}

ⁱⁱ

^{∨ ∨ ( ( ¬ ¬ ) ) y y}

^jj

^{∨ ∨ ( ( ¬ ¬ ) ) y y}

^ℓℓ

^{) )}

Examples: • 0

• ( ¬ x ∨ y ) ∧ ( x ∨ ¬ y )

• ( ¬ x ∨ y) ∧ (x ∨ y) ∧ ¬ y

• ( ¬ x ∨ y) ∧ (x ∨ ¬ z )

∧ (z ∨ ¬ y) ∧ x ∧ ( ¬ y)

Martin Ziegler

EVAL EVAL : : Given Given 〈 〈 Φ Φ ( ( Y Y

₁₁

, , … … Y Y

_nn

) ) 〉 〉 and and y y

₁₁

, , … … y y

_nn

∈ ∈ {0,1} {0,1} , , does does Φ Φ ( ( y y

₁₁

, , … … y y

_nn

) ) evaluate evaluate to to 1 1 ? ?

[ [ k k - - ]SAT ]SAT : : Is Is given given Φ Φ ( ( Y Y ) ) [in [in k k - - CNF CNF ] ] satisfiable satisfiable , , i.e i.e . . ∃ ∃ y y

₁₁

, , … … y y

_nn

∈ ∈ {0,1} {0,1} : : Φ Φ ( ( y y

₁₁

, , … … y y

_nn

)=1 )=1 ? ?

P P P

P vs NP NP NP NP Millennium Problem

P P P P P P P

P := { := { L L ⊆ ⊆ { { 0 0 , , 1 1 }* }*

decidabledecidable in in polynomialpolynomial time time

} }

⊆ ⊆ NP NP NP NP NP NP NP NP := { := { L L

verifiableverifiable in in polynomialpolynomial time time

} }

⊆ ⊆ PSPACE PSPACE PSPACE PSPACE PSPACE PSPACE PSPACE PSPACE := { := { L L

decidabledecidable in in polynpolyn. . spacespace

} }

⊆ ⊆ EXP EXP EXP EXP EXP EXP EXP EXP := { := { L L

decidabledecidable in exponential time in exponential time

} }

Def Def : : L L ⊆ ⊆ {0,1}* {0,1}* is is verifiable verifiable in in polyn polyn . time . time if if

L L = = { { x x ∈ ∈ {0,1} {0,1}

ⁿⁿ

| | n n ∈ ∈ , , ∃ ∃ y y ∈ ∈ {0,1 {0,1 } }

^q(q(nn))

: : 〈 〈 x x , , y y 〉 〉 ∈ ∈ V V } }

for for some some V V ∈ ∈ P P _{and and} q q ∈ ∈ N _N [ [ N N ] ] . .

∈ P P P P P P P P

∈ EXP EXP EXP EXP EXP EXP EXP EXP

∈

∈ ∈

∈ ∈

∈

∈

∈ PSPACE PSPACE PSPACE PSPACE PSPACE PSPACE PSPACE PSPACE ∈ ∈ NP NP NP NP P P P P NP NP NP NP P P P P ? ?

Martin Ziegler

Example Problems (II)

Eulerian cycle in an undirected graph G traverses each edge precisely 1×; Hamiltonian cycle visits each vertex precisely 1x.

EC := { 〈 G 〉 | G admits a Eulerian cycle } HC := { 〈 G 〉 | G admits Hamilton. cycle }

G admitting a Eulerian cycle is connected and

Theorem: Conversely every connected graph with an even number of edges incident to each vertex admits a Eulerian cycle.

has an even number of edges incident to each vertex except for

isolated vertices

∈ P P P P P P P P

?

∈ NP NP NP NP NP NP NP NP

∈ NP NP NP NP NP NP NP NP

P P := { := { L L ⊆ ⊆ { { 0 0 , , 1 1 }* }* decidable decidable in in polynomial polynomial time time } }

⊆ ⊆ NP NP := { := { L L verifiable verifiable in in polynomial polynomial time time } }

Martin Ziegler

P P := { := { L L ⊆ ⊆ { { 0 0 , , 1 1 }* }* decidable decidable in in polynomial polynomial time time } }

⊆ ⊆ NP NP := { := { L L verifiable verifiable in in polynomial polynomial time time } }

More Problems in NP NP NP NP

4 4 - - SAT SAT : : Is Is given given Φ Φ ( ( Y Y ) ) in 4 in 4 - - CNF CNF satisfiable satisfiable ? ? 3 3 - - SAT SAT : : Is Is given given Φ Φ ( ( Y Y ) ) in 3 in 3 - - CNF CNF satisfiable satisfiable ? ? 2 2 - - SAT SAT : : Is Is given given Φ Φ ( ( Y Y ) ) in 2 in 2 - - CNF CNF satisfiable satisfiable ? ?

EC := { 〈 G 〉 | G admits a Eulerian cycle } HC := { 〈 G 〉 | G admits Hamilton. cycle }

∈ P P P P P P P P

?

CLIQUE := { 〈 G,k 〉 | ^graph G=(V,E)

contains k pairwise adjacent vertices }

IS := { 〈 G,k 〉 | ^graph G=(V,E) ^contains k pairwise non-adjacent vertices }

∈ ? ? P P P P P P P P

?

?

Martin Ziegler

CLIQUE := { 〈 G,k 〉 | ^graph G=(V,E)

contains k pairwise adjacent vertices }

IS := { 〈 G,k 〉 | ^graph G=(V,E) ^contains k pairwise non-adjacent vertices } Comparing Problems: Reduction

Don't

Don't know know whether whether

CLIQUE, IS ∈ P P P P _{or or not. not.}

Do Do know know : : either either both both or or none none ∈ P P P P

Def Def : : Polynom. Polynom. reduction reduction from from A A to to B B ⊆{ ⊆ { 0 0 , , 1 1 }* }*

is is a a f f :{ :{ 0 0 , , 1 1 }* }* → → { { 0 0 , , 1 1 }* }* computab computab . in . in poly.time poly.time s.t.

s.t. x x ∈ ∈ A A ⇔ ⇔ f f ( ( x x ) ) ∈ ∈ B B . .

G G =( =( V V , , E E ) ) → → ( ( V V , , V V × × V V \ \ E E ) )

Lemma:

Lemma: i) i) A A

_p

B, B B, B

_p

C C ⇒ ⇒ A A

_p

C C

ii) ii) A A

_p

B, B B, B _∈ P P P P ⇒ ⇒ A A _∈ P P P P

Write

Write A A

_p

B B . .

Martin Ziegler

Example Reduction: 4SAT vs. 3SAT

Def Def : : Polynom. Polynom. reduction reduction from from A A to to B B ⊆{ ⊆ { 0 0 , , 1 1 }* }*

is is a a f f :{ :{ 0 0 , , 1 1 }* }* → → { { 0 0 , , 1 1 }* }* computab computab . in . in poly.time poly.time s.t.

s.t. x x ∈ ∈ A A ⇔ ⇔ f f ( ( x x ) ) ∈ ∈ B B . .

4 4 - - SAT SAT : : Is Is given given Φ Φ ( ( Y Y ) ) in 4 in 4 - - CNF CNF satisfiable satisfiable ? ? 3 3 - - SAT SAT : : Is Is given given Φ Φ ( ( Y Y ) ) in 3 in 3 - - CNF CNF satisfiable satisfiable ? ? Given Φ = (a ∨ b ∨ c ∨ d) ∧ (p ∨ q ∨ r ∨ s) ∧ … with literals a,b,c,d, p,q,r,s,….

Introduce new variables u,v,… and consider Φ ' := ( a ∨ b ∨ u ) ∧ ( ¬ u ∨ c ∨ d )

∧ ( p ∨ q ∨ v) ∧ ( ¬ v ∨ ∨ r ∨ s ) ∧ …

variables,

possibly negated

f: 〈 Φ 〉 → 〈 Φ ' 〉 Write

Write A A

_p

B B . .

Martin Ziegler

Φ = C

₁

∧ C

₂

… ∧ C

_k

, C

_i

= x

_i1

∨ x

_i2

∨ x

_i3

, x

_is ^literals

V:= { (i,1),…(i,3): i≤k }, E:= { {(i,s),(j,t)} : i=j or x

_is

= x

_jt

}

(1,1)

(1,2)

(1,3)

(k,1)

(k,2)

(k,3)

Produce, given a 3-CNF term Φ, within polynomial time a graph G and integer k such that it holds:

iff G contains k pairwise non-adjacent vertices.

e.g.

( u ∨ .. ∨ .. ) ∧ ( .. ∨ ¬ u ∨ .. ) ∧ ( .. ∨ .. ∨ u ) ∧ ( u ∨ .. ∨ .. )

Richard Karp

Reduction 3SAT ^≼ _p IS

Φ is satisfiable

Martin Ziegler

P

NP c NP

Discrete Complexity Picture

Have Have shown shown : :

CLIQUE ≡_p IS ≼≼≼≼_p SAT ≡_p 3SAT ≼≼≼≼_p IS.

Theorem

Theorem (Cook'72/Levin'71): (Cook'72/Levin'71):

For For every every L L ∈ ∈ NP NP it it holds holds L L ^{≼ ≼≼ ≼}

_p

SAT SAT

Now Now know know ≈ ≈ 500 500 natural

natural problems problems all all polynom. polynom. - - time time

equivalent

equivalent

to to SAT SAT . .

Martin Ziegler

Real Function Complexity

Function f:[0,1] → ^{computable computable}

if some TM can, on input of n ∈ and of

(a

_m

) ⊆ ^with |x-a

_m

/2

^m+1

|<2

^-m

output b ∈ ^with |f(x)-b/2

ⁿ⁺¹

|<2

^-n

^.

in time

in time t t ( ( n n ) )

i R R A M ( G M P / M P F R )

Examples: a) + + , , × × , , exp exp

b) f f ( ( x x ) ) ≡ ≡ ^{∑ ∑}

_nn∈∈LL

^{4 4}

^--nn

^{iff L L} ⊆ _⊆ { { 0 0 , , 1 1 } }

^{* *}

decidable in time

in time t t ( ( n n ) )

Observation:

Observation: IfIf

ƒ ƒ

^{computablecomputable ⇒⇒ continuouscontinuous..}

µ µ ( ( n n )= )= t t ( ( n n +2) +2)

isis a a modulusmodulus of uniform of uniform continuitycontinuity of of

f f

::

∀ ∀ x x , , y y : | : | x x - - y y | | ≤ ≤ 2 2

^--µµ(n(n))

⇒ ⇒ | | f f ( ( x x ) ) - - f f ( ( y y )| )| ≤ ≤ 2 2

^-n-n..

_n

:= { k/2

ⁿ

: k ∈ }, = 

_n

_n

dyadic rationals

on [0;1] ^!

polytime polytime-

c) 1/ 1/ ln(e ln(e / / x x ) ) exptime, not polytime-computable c) sgn sgn , , Heaviside Heaviside not computable

IfIf

ƒ ƒ

^{computablecomputable in time in time}

t t ( ( n n ) )

, , thenthen

Martin Ziegler

a)a) MultivaluedMultivalued ''functionsfunctions'' b) b) DiscreteDiscrete ''adviceadvice''

c) c) ParameterizedParameterized complexity

complexity

Real Function Complexity

Observation:

Observation: IfIf

ƒ ƒ

^{computablecomputable ⇒⇒ continuouscontinuous..}

µ µ ( ( n n )= )= t t ( ( n n +2) +2)

isis a a modulusmodulus of uniform of uniform continuitycontinuity of of

f f

::

∀ ∀ x x , , y y : | : | x x - - y y | | ≤ ≤ 2 2

^--µµ(n(n))

⇒ ⇒ | | f f ( ( x x ) ) - - f f ( ( y y )| )| ≤ ≤ 2 2

^-n-n..

c) 1/ 1/ ln(e ln(e / / x x ) ) exptime, not polytime-computable

IfIf

ƒ ƒ

^{computablecomputable in time in time}

t t ( ( n n ) )

, , thenthen

Examples: a) + + , , × × , , exp exp

3rd effect3rd effect

computable

computable in time in time poly( poly( n n + + k k ) ) on on [ [ - - k k , , k k ] ]

on [0;1] ^!

polytime

Martin Ziegler

other other famous famous complexity complexity

class class

Complexity of Real Operators

ƒ ƒ :[0;1] :[0;1] → → [0;1] [0;1]

polytimepolytime computablecomputable ((⇒⇒ continuouscontinuous))

• • Max: Max: ƒ ƒ → → Max( Max( ƒ ƒ ): ): t t → → max max { { ƒ ƒ ( ( s s ): ): s s ≤ ≤ t t } }

Max( Max( ƒ ƒ ) )

computablecomputable in exponential time;in exponential time;

polytime

polytime--computablecomputable iffiff PP==NPNP

• • ∫ ∫ : : ƒ ƒ → → ∫ ∫ ƒ ƒ : : t t → → ∫ ∫

₀₀^tt

ƒ ƒ ( ( s s ) ) ds ds

∫ ƒ

∫ ƒ

^{computablecomputable} in exponential time;in exponential time;

"#

"#PP-complete-complete""

• • dsolve dsolve : C[0;1] : C[0;1] × × [ [ - - 1;1] 1;1] ∋ ∋ ƒ ƒ → → z z : : Ŝ Ŝ ( ( t t )= )= ƒ ƒ ( ( t t , , z z ), ), z z (0)=0. (0)=0.

in in generalgeneral no no computablecomputable solutionsolution

z z ( ( t t ) )

forfor

ƒ∈ ƒ∈ C C

¹¹ ""PSPACEPSPACE--completecomplete""

forfor

ƒ∈ ƒ∈ C C

^kk ""CHCH--hardhard""

even when restricting

to

ƒ∈ ƒ∈ C C

^∞∞

butbut forfor analytic analytic

ƒƒƒƒ ƒƒƒƒ

polytime polytime

[Friedman&Ko'80ies][Friedman&Ko'80ies]

other other famous famous complexity complexity

classes classes

Martin Ziegler

Replace

Replace continuouscontinuous hat hat functionfunction byby appropriately

appropriately scaledscaled

CC^∞∞''pulse' pulse' functionfunction

φφ((tt) = exp() = exp(--tt²/1²/1--tt²), |²), |tt|<1|<1

NP NP ∋ ∋ L L = = { { x x ∈ ∈ { { 0 0 , , 1 1 } }

ⁿⁿ

| | ∃ ∃ y y ∈ ∈ { { 0 0 , , 1 1 } }

^pp((n)n)

: : 〈 〈 x x , , y y 〉 〉 ∈ ∈ V V } }

n n - - th th large large interval interval : : size size 2 2

^-n-n

, , containing

containing 2 2

ⁿⁿ

subintervals subintervals : : one one for for each each x x ∈ ∈ { { 0 0 , , 1 1 } }

ⁿⁿ

, , in turn

in turn subdivided subdivided into into 2 2

^pp((nn))

subsubintervals subsubintervals for for y y 's 's n n =1 =1

n n =2 =2

xx=0=0 x=x=11 x=x=

0000 0101 1010 1111

yy==0000,01,01,…,… yy==0000,01,01,…,…

11

½½

¼¼

To To every every L L ∈ ∈ NP NP _{there there exists exists a a polytime polytime}

computable

computable C C

^∞∞

function function f f

_LL

:[0,1] :[0,1] → → ^{s.t.: s.t.:}

[0,1]

[0,1] ∋ ∋ t t → → max max f f

_LL

| |

_[0,t[0,t]]

again again polytime polytime iff iff L L ∈ ∈ P P

' Max ^{is NP NP NP NP -hard'}

e.g e.g . . p p (1)=2, (1)=2, p p (2)=3, (2)=3,

V V = { = { ^{〈 〈 0 0} , , 00 00 〉 〉 , , 〈 〈 0 0 , , 10 10 〉 〉 , , 〈 〈 0 0 , , 11 11 〉 〉 , ,

〈 〈 1 1 , , 01 01 〉 〉 , , 〈 〈 1 1 , , 10 10 〉 〉 , ,

〈00 〈 00 , , 010 010 〉 〉 , 〈 , 〈00 00 , , 100 100 〉 〉 , , … … } }

conti conti - -

nuous nuous

0 0.2 0.4 0.6 0.8 1

-1 -0.5 0 0.5 1