Foundations of Artiﬁcial Intelligence 36. Automated Planning: Delete Relaxation Heuristics Malte Helmert

(1)

Foundations of Artificial Intelligence

36. Automated Planning: Delete Relaxation Heuristics

Malte Helmert

University of Basel

May 5, 2021

(2)

Relaxed Planning Graphs Maximum and Additive Heuristics FF Heuristic Summary

Automated Planning: Overview

Chapter overview: automated planning 33. Introduction

34. Planning Formalisms

35.–36. Planning Heuristics: Delete Relaxation 35. Delete Relaxation

36. Delete Relaxation Heuristics 37. Planning Heuristics: Abstraction 38.–39. Planning Heuristics: Landmarks

(3)

Relaxed Planning Graphs

(4)

Relaxed Planning Graphs

relaxed planning graphs: representwhich variables in Π⁺ can be reached and how

graphs withvariable layers Vⁱ andaction layersAⁱ

variable layerV⁰ contains thevariable vertexv⁰ for allv∈I action layerAⁱ⁺¹ contains theaction vertexaⁱ⁺¹ for actiona ifVⁱ contains the vertexvⁱ for allv∈pre(a)

variable layerVⁱ⁺¹ contains the variable vertexvⁱ⁺¹ if previous variable layer containsvⁱ,

or previous action layer containsaⁱ⁺¹ withv ∈add(a) German: relaxierter Planungsgraph, Variablenknoten, Aktionsknoten

(5)

Relaxed Planning Graphs (Continued)

goal verticesGⁱ ifvⁱ ∈Vⁱ for all v ∈G

graph can be constructed for arbitrary many layers but stabilizes after a bounded number of layers

Vⁱ⁺¹=Vⁱ andAⁱ⁺¹=Aⁱ (Why?) directed edges:

fromvⁱ toaⁱ⁺¹ ifv∈pre(a) (precondition edges) fromaⁱ tovⁱ if v∈add(a) (effect edges)

fromvⁱ toGⁱ ifv ∈G (goal edges) fromvⁱ tovⁱ⁺¹ (no-op edges)

German: Zielknoten, Vorbedingungskanten, Effektkanten, Zielkanten, No-Op-Kanten

(6)

Illustrative Example

We will write actionsa with pre(a) ={p₁, . . . ,pk}, add(a) ={a₁, . . . ,a_l},del(a) =∅andcost(a) =c asp₁, . . . ,p_k −→^c a₁, . . . ,a_l

V ={a,b,c,d,e,f,g,h}

I ={a}

G ={c,d,e,f,g}

A={a₁,a₂,a₃,a₄,a₅,a₆} a₁ =a−→³ b,c

a₂ =a,c −→¹ d a3 =b,c −→¹ e a4 =b −→¹ f a₅ =d −→¹ e,f a₆ =d −→¹ g

(7)

Illustrative Example: Relaxed Planning Graph

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

(8)

Illustrative Example: Relaxed Planning Graph

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁

(9)

Illustrative Example: Relaxed Planning Graph

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁

a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

(10)

Illustrative Example: Relaxed Planning Graph

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁

a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ a₂ a3

a4

(11)

Illustrative Example: Relaxed Planning Graph

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁

a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ a₂ a3

a4

a² b² c² d² e² f² g² h²

(12)

Illustrative Example: Relaxed Planning Graph

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁

a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ a₂ a3

a4

a² b² c² d² e² f² g² h²

a₁ a₂ a3

a4

a5

a6

(13)

Illustrative Example: Relaxed Planning Graph

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁

a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ a₂ a3

a4

a² b² c² d² e² f² g² h²

a₁ a₂ a3

a4

a5

a6

a³ b³ c³ d³ e³ f³ g³ h³

(14)

Illustrative Example: Relaxed Planning Graph

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁

a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ a₂ a3

a4

a² b² c² d² e² f² g² h²

a₁ a₂ a3

a4

a5

a6

a³ b³ c³ d³ e³ f³ g³ h³ c³ d³ e³ f³ g³

(15)

Illustrative Example: Relaxed Planning Graph

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁

a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ a₂ a3

a4

a² b² c² d² e² f² g² h²

a₁ a₂ a3

a4

a5

a6

a³ b³ c³ d³ e³ f³ g³ h³

G

(16)

Generic Relaxed Planning Graph Heuristic

Heuristic Values from Relaxed Planning Graph functiongeneric-rpg-heuristic(hV,I,G,Ai,s):

Π⁺:=hV,s,G,A⁺i for k ∈ {0,1,2, . . .}:

rpg:=RPG_k(Π⁺) [relaxed planning graph to layer k]

if rpg contains a goal node:

Annotate nodes ofrpg.

if termination criterion is true:

returnheuristic value from annotations else if graph has stabilized:

return∞

general templatefor RPG heuristics

to obtain concrete heuristic: instantiate highlighted elements

(17)

Concrete Examples for Generic RPG Heuristic

Many planning heuristics fit this general template.

In this course:

maximum heuristic h^max (Bonet & Geffner, 1999) additive heuristic h^add (Bonet, Loerincs & Geffner, 1997) Keyder & Geffner’s (2008) variant of the FF heuristic h^FF (Hoffmann & Nebel, 2001)

German: Maximum-Heuristik, additive Heuristik, FF-Heuristik remark:

The most efficient implementations of these heuristics do not use explicit planning graphs,

but rather alternative (equivalent) definitions.

(18)

Maximum and Additive Heuristics

(19)

Maximum and Additive Heuristics

h^max andh^add are the simplest RPG heuristics.

Vertex annotations arenumerical values.

The vertex values estimate the costs to make a given variable true to reach and apply a given action to reach the goal

(20)

Maximum and Additive Heuristics: Filled-in Template

h^max andh^add

computation of annotations:

costs of variable vertices:

0 in layer 0;

otherwise minimumof the costs of predecessor vertices costs of action and goal vertices:

maximum(h^max) or sum(h^add) of predecessor vertex costs;

for action vertices aⁱ, also addcost(a) termination criterion:

stability: terminate ifVⁱ =Vⁱ⁻¹ and costs of all vertices in Vⁱ equal corresponding vertex costs in Vⁱ⁻¹

heuristic value:

value of goal vertex in the last layer

(21)

Maximum and Additive Heuristics: Intuition

intuition:

variable vertices:

choosecheapestway of reaching the variable action/goal vertices:

h^max isoptimistic: assumption:

when reaching themost expensiveprecondition variable, we can reach the other precondition variables in parallel (hence maximization of costs)

h^add ispessimistic: assumption:

all precondition variables must be reached completely independently of each other (hence summation of costs)

(22)

Illustrative Example: h

^max

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G

h^max({a}) = 5

(23)

Illustrative Example: h

^max

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

h^max({a}) = 5

(24)

Illustrative Example: h

^max

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

h^max({a}) = 5

(25)

Illustrative Example: h

^max

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

0 3 3

h^max({a}) = 5

(26)

Illustrative Example: h

^max

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

0 3 3

3

h^max({a}) = 5

(27)

Illustrative Example: h

^max

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

0 3 3

3 4

h^max({a}) = 5

(28)

Illustrative Example: h

^max

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

0 3 3

3 4 4 4

h^max({a}) = 5

(29)

Illustrative Example: h

^max

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

0 3 3

3 4 4 4

0 3 3 4 4 4

h^max({a}) = 5

(30)

Illustrative Example: h

^max

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

0 3 3

3 4 4 4

0 3 3 4 4 4

3 4 4 4 5 5

h^max({a}) = 5

(31)

Illustrative Example: h

^max

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

0 3 3

3 4 4 4

0 3 3 4 4 4

3 4 4 4 5 5

0 3 3 4 4 4 5

h^max({a}) = 5

(32)

Illustrative Example: h

^max

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

0 3 3

3 4 4 4

0 3 3 4 4 4

3 4 4 4 5 5

0 3 3 4 4 4 5

5

h^max({a}) = 5

(33)

Illustrative Example: h

^add

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G

h^add({a}) = 21

(34)

Illustrative Example: h

^add

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

h^add({a}) = 21

(35)

Illustrative Example: h

^add

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

h^add({a}) = 21

(36)

Illustrative Example: h

^add

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

0 3 3

h^add({a}) = 21

(37)

Illustrative Example: h

^add

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

0 3 3

3

h^add({a}) = 21

(38)

Illustrative Example: h

^add

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

0 3 3

3 4

h^add({a}) = 21

(39)

Illustrative Example: h

^add

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

0 3 3

3 4 7 4

h^add({a}) = 21

(40)

Illustrative Example: h

^add

a⁰ b⁰ c⁰ d⁰ e⁰ f⁰ g⁰ h⁰

a₁ ⁺³ a¹ b¹ c¹ d¹ e¹ f¹ g¹ h¹

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹

a² b² c² d² e² f² g² h²

a₁ ⁺³ a₂ ⁺¹ a₃ ⁺¹ a₄ ⁺¹ a₅ ⁺¹ a₆ ⁺¹

a³ b³ c³ d³ e³ f³ g³ h³

G 0

3

0 3 3

3 4 7 4

0 3 3 4 7 4

h^add({a}) = 21