Theory of Computer Science D1. Nondeterministic Algorithms, P and NP Gabriele R¨oger

(1)

Theory of Computer Science

D1. Nondeterministic Algorithms, P and NP

Gabriele R¨oger

University of Basel

May 3, 2021

(2)

Theory of Computer Science

May 3, 2021 — D1. Nondeterministic Algorithms, P and NP

D1.1 Motivation

D1.2 How to Measure Runtime?

D1.3 Decision Problems

D1.4 Nondeterminism

D1.5 P and NP

(3)

Overview: Course

contents of this course:

A. backgroundX

. mathematical foundations and proof techniques B. automata theory and formal languagesX

. What is a computation?

C. Turing computability X

. What can be computed at all?

D. complexity theory

. What can be computed efficiently?

E. more computability theory

(4)

D1. Nondeterministic Algorithms, P and NP Motivation

D1.1 Motivation

(5)

A Scenario (1)

Example Scenario

I You are a programmer at a logistics company.

I Your boss gives you the task of developing a program to optimize the route of a delivery truck:

I The truck begins its route at the company depot.

I It has to visit 50 stops.

I You know the distances between all relevant locations (stops and depot).

I Your program should compute a tour visiting all stops and returning to the depot on ashortest route.

(6)

A Scenario (2)

Example Scenario (ctd.)

I You work on the problem for weeks, but you do not manage to complete the task.

I All of your attempted programs

I compute routes that are possibly suboptimal, or

I do not terminate in reasonable time(say: within a month).

I What do you say to your boss?

(7)

What You Don’t Want to Say

“I can’t find an efficient algorithm, I guess I’m just too dumb.”

(8)

What You Would Like to Say

“I can’t find an efficient algorithm, because no such algorithm is possible!”

Source: M. Garey & D. Johnson, Computers and Intractability, Freeman 1979, p. 2

(9)

What Complexity Theory Allows You to Say

“I can’t find an efficient algorithm, but neither can all these famous people.”

(10)

Why Complexity Theory?

Complexity Theory

Complexity theorytells us which problems can be solvedquickly(“simple problems”) and which onescannot (“hard problems”).

German: Komplexit¨atstheorie

I This is useful in practice because simple and hard problems requiredifferent techniquesto solve.

I If we can show that a problem is hard we do not need to waste our time with the (futile) search for a “simple” algorithm.

(11)

Test Your Intuition! (1)

I The following slide lists some graph problems.

I The input is always adirected graph G =hV,Ei.

I How difficult are the problems in your opinion?

I Sort the problems

fromeasiest (= requires least amount of time to solve) to hardest (= requires most time to solve)

I no justification necessary, just follow your intuition!

I anonymous andnot graded

(12)

Test Your Intuition! (2)

1 Find a simple path (= without cycle) fromu ∈V tov ∈V with minimal length.

2 Find a simple path (= without cycle) fromu ∈V tov ∈V with maximal length.

3 Determine whether G is strongly connected (every node is reachable from every other node).

4 Find a cycle (non-empty path fromu tou for anyu ∈V; multiple visits of nodes are allowed).

5 Find a cycle that visits allnodes.

6 Find a cycle that visits agiven nodeu.

7 Find a path that visits all nodeswithout repeating a node.

8 Find a path that uses all edges without repeating an edge.

(13)

D1. Nondeterministic Algorithms, P and NP How to Measure Runtime?

D1.2 How to Measure Runtime?

(14)

How to Measure Runtime?

I Time complexity is a way to measure how much time it takes to solve a problem.

I How can we definesuch a measure appropriately?

German: Zeitkomplexit¨at/Zeitaufwand

(15)

Example Statements about Runtime

Example statements about runtime:

I “Running sort /usr/share/dict/words on the computer dakar takes 0.035 seconds.”

I “With a 1 MiB input file, sort takes at most 1 second on a modern computer.”

I “Quicksort is faster than sorting by insertion.”

I “Sorting by insertion is slow.”

Very different statements with different pros and cons.

(16)

Precise Statements vs. General Statements

Example Statement about Runtime

“Runningsort /usr/share/dict/words on the computer dakartakes 0.035 seconds.”

advantage: veryprecise disadvantage: notgeneral

I input-specific:

What if we want to sort other files?

I machine-specific:

What happens on a different computer?

I evensituation-specific:

Will we get the same result tomorrow that we got today?

(17)

General Statements about Runtime

In this course we want to makegeneralstatements about runtime. We accomplish this in three ways:

1. General Inputs

Instead ofconcrete inputs, we talk about general types of input:

I Example: runtime to sort an input of sizen in the worst case

I Example: runtime to sort an input of sizen in the average case

here: runtime for input sizen in the worst case

(18)

General Statements about Runtime

2. Ignoring Details

Instead ofexact formulas for the runtime we specify theorder of magnitude:

I Example: instead of saying that we need time

d1.2nlogne −4n+ 100, we say that we need timeO(nlogn).

I Example: instead of saying that we need timeO(nlogn), O(n²) orO(n⁴), we say that we needpolynomial time.

here: What can be computed inpolynomial time?

(19)

General Statements about Runtime

3. Abstract Cost Measures

Instead of theruntime on a concrete computer we consider amore abstract cost measure:

I Example: count the number of executed machine code statements

I Example: count the number of executed Java byte code statements

I Example: count the number ofelement comparisons

(20)

D1. Nondeterministic Algorithms, P and NP Decision Problems

D1.3 Decision Problems

(21)

Decision Problems

I As before, we simplify our investigation

by restricting our attention to decision problems.

I More complex computational problems can be solved with multiple queries for an appropriately defined decision problem (“playing 20 questions”).

I Formally, decision problems arelanguages (as before), but we use an informal“given”/“question” notation where possible.

(22)

Example: Decision vs. General Problem (1)

Definition (Hamilton Cycle)

LetG =hV,Eibe a (directed or undirected) graph.

AHamilton cycleof G is a sequence of vertices inV, π=hv₀, . . . ,vni, with the following properties:

I π is a path: there is an edge fromvi to vi+1 for all 0≤i <n I π is a cycle: v0 =vn

I π is simple: v_i 6=v_j for all i 6=j with i,j <n I π is Hamiltonian: all nodes ofV are included in π German: Hamiltonkreis/Hamiltonzyklus

(23)

Example: Decision vs. General Problem (2)

Example (Hamilton Cycles in Directed Graphs) P: general problem DirHamiltonCycleGen

I Input: directed graphG =hV,Ei

I Output: a Hamilton cycle of G or a message that none exists D: decision problemDirHamiltonCycle

I Given: directed graph G =hV,Ei

I Question: DoesG contain a Hamilton cycle?

These problems arepolynomially equivalent:

from a polynomial algorithm for one of the problems

(24)

Algorithms for Decision Problems

Algorithms for decision problems:

I Where possible, we specify algorithms for decision problems in pseudo-code.

I Since they are only yes/no questions, we do not have to return a general result.

I Instead we use the statements

I ACCEPTtoaccept the given input (“yes” answer) and I REJECTtorejectit (“no” answer).

I Where we must be more formal, we use Turing machines and the notion of accepting from chapter B9.

(25)

D1. Nondeterministic Algorithms, P and NP Nondeterminism

D1.4 Nondeterminism

(26)

Nondeterminism

I To develop complexity theory, we need the algorithmic concept ofnondeterminism.

I already known forTuring machines ( chapter B10):

I An NTM can havemore than one possible successor configurationfor a given configuration.

I Inputx is accepted if there isat least one possible computation (configuration sequence) that leads to the accept state.

I Here we analogously introduce nondeterminism for pseudo-code.

German: Nichtdeterminismus

(27)

Nondeterministic Algorithms

nondeterministic algorithms:

I All constructs of deterministic algorithms are also allowed in nondeterministic algorithms: IF,WHILE, etc.

I Additionally, there is a nondeterministic assignment:

GUESS xi ∈ {0,1}

wherex_i is a program variable.

German: nichtdeterministische Zuweisung

(28)

Nondeterministic Algorithms: Acceptance

I Meaning of GUESSxi ∈ {0,1}:

x_i is assigned either the value0 or the value1.

I This implies that the behavior of the program on a given input is no longer uniquely defined:

there are multiple possible execution paths.

I The program accepts a given input ifat least one execution path leads to an ACCEPT statement.

I Otherwise, the input is rejected.

Note: asymmetry between accepting and rejecting!

Note:

(cf. Turing-recognizability)

(29)

More Complex GUESS Statements

I We will also guess more than one bit at a time:

GUESS x∈ {1,2, . . . ,n}

or more generally GUESS x∈S for a setS.

I These are abbreviations and can be split intodlog₂ne (ordlog₂|S|e) “atomic” GUESSstatements.

(30)

Example: Nondeterministic Algorithms (1)

Example (DirHamiltonCycle) input: directed graphG =hV,Ei start := an arbitrary node from V current :=start

remaining := V \ {start}

WHILEremaining6=∅:

GUESS next∈remaining IF hcurrent,nexti∈/ E:

REJECT

remaining := remaining\ {next}

current := next IFhcurrent,starti ∈E:

ACCEPT ELSE:

REJECT

(31)

Example: Nondeterministic Algorithms (2)

I With appropriate data structures, this algorithm solves the problem in O(nlogn) program steps,

wheren =|V|+|E|is the size of the input.

I How many steps would a deterministicalgorithm need?

(32)

Guess and Check

I The DirHamiltonCycleexample illustrates

a general design principle for nondeterministic algorithms:

guess and check I In general, nondeterministic algorithms can

solve a problem by first guessing a “solution”

and then verifying that it is indeed a solution.

(In the example, these two steps are interleaved.) I If solutions to a problem can be efficiently verified,

then the problem can also be efficiently solved if nondeterminism may be used.

German: Raten und Pr¨ufen

(33)

The Power of Nondeterminism

I Nondeterministic algorithms are very powerful

because they can “guess” the “correct” computation step.

I Or, interpreted differently: they go through many possible computations “in parallel”, and it suffices ifone of them is successful.

I Can they solve problems efficiently (in polynomial time) which deterministic algorithms cannotsolve efficiently?

I This is the big question!

(34)

D1. Nondeterministic Algorithms, P and NP P and NP

D1.5 P and NP

(35)

Impact of Nondeterminism?

I We earlier established that deterministic and non-deterministic Turing machines recognize the same class of languages.

→ For this aspect, non-determinism did not make a difference.

I Now we consider what decision problems can be solved in polynomial time.

I Does it make a difference whether we allow non-determinism?

This is the famous P vs. NP question!

(36)

Runtime of a Deterministic Turing Machine

Definition (Runtime of a DTM)

LetM be a DTM that halts on all inputs. The running time or time complexityof M if the functionf :N→N, wheref(n) is the maximum number of steps thatM uses on any input of lengthn.

We say that

I M runs in time f and that I M is an f time Turing machine.

(37)

Big-O

Definition (Big-O)

Letf andg be functionsf,g :N→R⁺.

We say thatf ∈O(g)if positive integers c andn₀ exist such that for every integern≥n0

f(n)≤cg(n).

(38)

Complexity Class P

Definition (Time Complexity Class TIME) Lett :N→R⁺ be a function.

Define thetime complexity class TIME(t(n)) to be the collection of all languages that are decidable by anO(t) time Turing machine.

Definition (P)

Pis the class of languages that aredecidable in polynomial time by a deterministic single-tape Turing machine. In other words,

P =[

k

TIME(n^k).

(39)

Runtime of a Non-deterministic Turing Machine

Definition (Runtime of a NTM)

LetM be a NTM that is a decider, i. e. all its computation branches halt on all inputs.

Therunning timeor time complexity ofM if the function f :N→N, wheref(n) is the maximum number of steps thatM uses onany branch of its computation on any input of lengthn.

(40)

Complexity Class NP

Definition (Time Complexity Class NTIME) Lett :N→R⁺ be a function.

Define the time complexity classNTIME(t(n)) to be the collection of all languages that are

decidable by anO(t) time nondeterministic Turing machine.

Definition (NP)

NPis the class of languages that are decidable in polynomial time by anon-deterministicsingle-tape Turing machine. In other words,

NP =[

k

NTIME(n^k).

(41)

P and NP: Remarks

I Sets of languages like P and NP that are defined in terms of computation time of TMs

(or other computation models) are called complexity classes.

I We know that P⊆NP. (Why?)

I Whether the converse is also true is an open question:

this is the famousP-NP problem.

German: Komplexit¨atsklassen, P-NP-Problem

(42)

Example: DirHamiltonCycle ∈ NP

Example (DirHamiltonCycle∈NP)

The nondeterministic algorithm of the previous section solves the problem and can be implemented on an NTM in polynomial time.

I Is DirHamiltonCycle∈P also true?

I The answer is unknown.

I So far, only exponential deterministic algorithms for the problem are known.

(43)

Simulation of NTMs with DTMs

I Unlike DTMs, NTMs are not arealistic computation model:

they cannot be directly implemented on computers.

I But NTMs can besimulated by systematically trying all computation paths, e. g., with a breadth-first search.

More specifically:

I Let M be an NTM that decides languageL in timef, wheref(n)≥n for alln ∈N0.

I Then we can specify a DTMM⁰ that decides L in timef⁰, wheref⁰(n) = 2^O(f⁽ⁿ⁾⁾.

I without proof

(44)

D1. Nondeterministic Algorithms, P and NP Summary

Summary (1)

I Complexity theorydeals with the question which problems can be solved efficientlyand which ones cannot.

I here: focus on what can be computedin polynomial time I To formalize this, we use Turing machines,

but other formalisms are polynomially equivalent.

I We consider decision problems, but the results often directly transfer to general computational problems.

(45)

Summary (2)

important concept: nondeterminism

I Nondeterministic algorithmscan“guess”,

i. e., perform multiple computations “at the same time”.

I An input receives a “yes” answer ifat least one computation pathaccepts it.

I in NTMs: with nondeterministic transitions (δ(q,a) contains multiple elements)

I in pseudo-code: with GUESSstatements

(46)

Summary (3)

I P: languages decidable byDTMsin polynomial time I NP: languages decidable byNTMs in polynomial time I P⊆NPbut it is anopen question whether P = NP.