Planning and Optimization D7. M&S: Generic Algorithm and Heuristic Properties Gabriele R¨oger and Thomas Keller

(1)

Planning and Optimization

D7. M&S: Generic Algorithm and Heuristic Properties

Gabriele R¨oger and Thomas Keller

Universit¨at Basel

November 7, 2018

(2)

Generic Algorithm Heuristic Properties Summary

Content of this Course

Planning

Classical

Tasks Progression/

Regression Complexity Heuristics

Probabilistic

MDPs Uninformed Search

Heuristic Search Monte-Carlo

Methods

(3)

Content of this Course: Heuristics

Heuristics

Delete Relaxation

Abstraction

Abstractions in General

Pattern Databases

Merge &

Shrink Landmarks

Potential Heuristics Cost Partitioning

(4)

Generic Algorithm

(5)

Content of this Course: Merge & Shrink

Merge & Shrink

Synchronized Product Merge & Shrink Algorithm

Heuristic Properties Strategies Label Reduction

(6)

Generic Merge-and-shrink Abstractions: Outline

Using the results from the previous chapter, we can develop the ideas of ageneric abstraction computation procedurethat takes all state variables into account:

Initialization step: Compute all abstract transition systems for atomic projections to form the initial abstraction collection.

Merge steps: Combine two abstract systems in the collection by replacing them with their synchronized product. (Stop once only one transition system is left.)

Shrink steps: If the abstractions in the collection are too large to compute their synchronized product, make them smaller by abstracting them further (applying an arbitrary abstraction to them).

We explain these steps with our running example.

(7)

Back to the Running Example

LRR LLL LLR

LRL

ALR

ALL

BLL

BRL

ARL

ARR

BRR

BLR

RRR RRL

RLR

RLL

Logistics problem with one package, two trucks, two locations:

state variablepackage: {L,R,A,B} state variabletruck A:{L,R}

state variabletruck B:{L,R}

(8)

Initialization Step: Atomic Projection for Package

T^π^{package}:

L

A

B

R

M???

PAL DAL

M???

DAR PAR

M???

PBR DBR

M???

DBL PBL

(9)

Initialization Step: Atomic Projection for Truck A

T^π^{{truck A}}:

L R

PAL,DAL,MB??, PB?,DB?

MALR

MARL

PAR,DAR,MB??, PB?,DB?

(10)

Initialization Step: Atomic Projection for Truck B

T^π^{{truck B}}:

L R

PBL,DBL,MA??, PA?,DA?

MBLR

MBRL

PBR,DBR,MA??, PA?,DA?

current collection: {T^π^{package},T^π^{{truck A}},T^π^{{truck B}}}

(11)

First Merge Step

T₁ :=T^π^{package}⊗ T^π^{{truck A}}:

LL LR

AL AR

BL BR

RL RR

MALR MARL

MALR MARL PAL

DAL

PADARR

PBR DBL DBR

PBL

PBL DBL

DBR PBR

MB?? MB??

current collection: {T₁,T^π^{{truck B}}}

(12)

Need to Simplify?

If we have sufficient memory available, we can now compute T₁⊗ T^π^{{truck B}}, which would recover the complete transition system of the task.

However, to illustrate the general idea, let us assume that we do not have sufficient memory for this product.

More specifically, we will assume that after each product operation we need to reduce the result transition system to four states to obey memory constraints.

So we need to reduce T₁ to four states. We have a lot of leeway in deciding how exactly to abstractT₁.

In this example, we simply use an abstraction that leads to a good result in the end.

(13)

First Shrink Step

T₂ := some abstraction of T₁

LL LR

AL AR

BL BR

RL RR

MALR MARL

MALR MARL PAL

DAL

PADARR

PBR DBL DBR

PBL

PBL DBL

DBR PBR

MB?? MB??

current collection: {T₂,T^π^{{truck B}}}

(14)

First Shrink Step

LL LR

AL AR

BL BR

RL RR

MALR MARL

MALR MARL PAL

DAL

PADARR

PBR DBL DBR

PBL

PBL DBL

DBR PBR

MB?? MB??

(15)

First Shrink Step

LL LR

AL AR

BL BR

R

MALR MARL

MALR MARL PAL

DAL

DAR PAR

DBRPBR DBL

PBL

PBL DBL

DBR PBR MB??

MB?? MB??

MB??

M???

MB??

(16)

First Shrink Step

LL LR

AL AR

BL BR

R

MALR MARL

MALR MARL PAL

DAL

DAR PAR

DBRPBR DBL

PBL

PBL DBL

DBR PBR MB??

MB?? MB??

MB??

M???

MB??

(17)

First Shrink Step

LL LR

A

BL BR

R

MALR MARL

MALR MARL PAL

DAL

DAR PAR

DBRPBR DBL

PBL

PBL DBL

DBR PBR MB??

M???

MB??

M???

MB??

(18)

First Shrink Step

LL LR

A

BL BR

R

MALR MARL

MALR MARL PAL

DAL

DAR PAR

DBRPBR DBL

PBL

PBL DBL

DBR PBR MB??

M???

MB??

M???

MB??

(19)

First Shrink Step

LL LR

A

B

R

MALR MARL

PAL DAL

DAR PAR

PBR DBL DBR

PBL PBL DBL MB??

M???

MB??

M???

(20)

First Shrink Step

LL LR

A A

B B

R

MALR MARL

PAL DAL

DAR PAR

PBR DBL DBR

PBL PBL DBL MB??

M???

MB??

M???

(21)

First Shrink Step

LL LR I R

MALR MARL MB??

MB??

D?R M???

P?R M???

PBL DBL P?L D?L

(22)

First Shrink Step

LL LR I R

MALR MARL MB??

MB??

D?R M???

P?R M???

PBL DBL P?L D?L

(23)

Second Merge Step

T₃ :=T₂⊗ T^π^{{truck B}}:

LRL

LRR

LLL

LLR

IL

IR

RL

RR

MBLR MBRL

MBLR MBRL DAR

PAR

D?R P?R P?L

D?L

PAL DAL MALR MARL MALR MARL

PBL DBL

MA??

MA?? MA??

MA??

current collection: {T₃}

(24)

Another Shrink Step?

Normally we could stop now and use the distances in the final abstract transition system as our heuristic function.

However, if there were further state variables to integrate, we would simplify further, e.g. leading to the following

abstraction (again with four states):

LRR ^LLL^LRL

LLR I R

M??? M???

M???

M?RL M?LR

P?L D?L

D?R P?R

We get a heuristic value of 3 for the initial state, better than any PDB heuristic that is a proper abstraction.

The example generalizes to more locations and trucks, even if we stick to the size limit of 4 (after merging).

(25)

Generic Algorithm Template

Generic Merge & Shrink Algorithm abs := {T^π^{v} |v ∈V}

while abs contains more than one abstract transition system:

select A₁,A₂ fromabs

shrink A₁ and/or A₂ untilsize(A₁)·size(A₂)≤N abs := abs\ {A₁,A₂} ∪ {A₁⊗ A₂}

returnthe remaining abstract transition system in abs N: parameter bounding number of abstract states Questions for practical implementation:

Which abstractions to select? merging strategy How to shrink an abstraction? shrinking strategy How to chooseN? usually: as high as memory allows

(26)

Heuristic Properties

(27)

Content of this Course: Merge & Shrink

Merge & Shrink

Synchronized Product Merge & Shrink Algorithm

Heuristic Properties Strategies Label Reduction

(28)

Heuristic Properties

Each iteration of the algorithm corresponds to a

transformation of the collection absof transition systems.

The exact transformation depends on the specific instantiation of the generic algorithm

(e.g. of the merging and the shrinking strategy).

For analyzing the properties of the resulting heuristic, we analyze properties of the individual transformations.

(29)

Collections of Transition Systems

Definition (Collection of Transition Systems)

A setX of transition systems is acollection of transition systems if allT ∈X have the same set of labels and the same cost function.

Thecombined systemis T_X :=N

T ∈X T.

Remark: Strictly speaking, the combined system is not well-defined as the Cartesian product is neither commutative nor associative.

For our purpose it is sufficient that the results of all possible combination orders are isomorphic.

(30)

Safe Transformations

Definition (Safe Transformation)

LetX andX⁰ be collections of transition systems with label setsL andL⁰ and cost functions c andc⁰, respectively.

The transformation fromX toX⁰ is safe if there exist functionsσ andλmapping the states and labels ofT_X to the states and labels ofT_X⁰ such that

c⁰(λ(`))≤c(`) for all`∈L,

if hs, `,ti is a transition ofT_X then hσ(s), λ(`), σ(t)i is a transition of T_X⁰, and

if s is a goal state ofT_X thenσ(s) is a goal state of T_X⁰.

(31)

Examples

X: Collection of transition systems

Replacement with Synchronized Product is Safe

LetT₁,T₂ ∈X with T₁6=T₂. The transformation fromX to X⁰:= (X\ {T₁,T₂})∪ {T₁⊗ T₂} is safe with σ= id andλ= id.

Abstraction is Safe

Letα be an abstraction forT_i ∈X. The transformation from X to X⁰:= (X\ {T_i})∪ {T_i^α} is safe withλ= id and

σ(hs₁, . . . ,s_ni) =hs₁, . . . ,si−1, α(s_i),s_i+1, . . . ,s_ni.

(Proofs omitted.)

(32)

Heuristic Properties (1)

Theorem

Let X and X⁰ be collections of transition systems. If the

transformation from X to X⁰ issafe with functions σ andλthen h(s) =h^∗_T

X0(σ(s))is a safe, goal-aware, admissible, and consistent heuristic forT_X.

Proof.

We prove goal-awareness and consistency, the other properties follow from these two.

Goal-awareness: For all goal statess_? of T_X, state σ(s_?) is a goal state ofT_X⁰ and therefore h(s?) =h^∗_T

X0(σ(s?)) = 0. . . .

(33)

Heuristic Properties (2)

Proof (continued).

Consistency: Letc andc⁰ be the label cost functions of X andX⁰, respectively. Consider states of T_X and transitionhs, `,ti.

AsT_X⁰ has a transition hσ(s), λ(`), σ(t)i, it holds that h(s) =h^∗_T

X0(σ(s))

≤c⁰(λ(`)) +h^∗_T

X0(σ(t))

=c⁰(λ(`)) +h(t)

≤c(`) +h(t)

The second inequality holds due to the requirement on the label costs.

(34)

Exact Transformations

Definition (Exact Transformation)

LetX andX⁰ be collections of transition systems with label setsL andL⁰ and cost functions c andc⁰, respectively.

The transformation fromX to X⁰ is exactif there exist functions σ andλmapping the states and labels ofT_X to the states and labels ofT_X⁰ such that

1 σ andλsatisfy the requirements of safe transformations,

2 ifhs⁰, `⁰,t⁰i is a transition ofT_X⁰ then hs, `,tiis a transition of T_X for all s ∈σ⁻¹(s⁰),t∈σ⁻¹(t⁰) and some`∈λ⁻¹(`⁰),

3 if s⁰ is a goal state of T_X⁰ then all states s ∈σ⁻¹(s⁰) are goal states of T_X, and

4 c(`) =c⁰(λ(`)) for all`∈L.

no “new” transitions and goal states, no cheaper labels

(35)

Examples

Replacement with Synchronized Product is Exact

LetT₁,T₂ ∈X with T₁6=T₂. The transformation fromX to X⁰:= (X\ {T₁,T₂})∪ {T₁⊗ T₂} is exact with σ= id and λ= id.

(Proof omitted.)

More examples will follow.

(36)

Heuristic Properties with Exact Transformations (1)

Theorem

Let X and X⁰ be collections of transition systems. If the

transformation from X to X⁰ isexact with functionsσ and λthen h^∗_T

X(s) =h^∗_T

X0(σ(s)).

Proof.

As the transformation is safe,h^∗_T

X0(σ(s)) is admissible for T_X and thereforeh^∗_T

X(s)≥h^∗_T

X0(σ(s)).

For the other direction, we show that for every states⁰ ofT_X⁰ and goal pathπ⁰ for s⁰, there is for each s ∈σ⁻¹(s⁰) a goal path inT_X

that has the same cost. . . .

(37)

Heuristic Properties with Exact Transformations (2)

Proof (continued).

Proof via induction over the length ofπ⁰.

|π⁰|= 0: Ifs⁰ is a goal state ofT_X⁰ then eachs ∈σ⁻¹(s⁰) is a goal state ofT_X and the empty path is a goal path fors in T_X.

|π⁰|=i+ 1: Let π⁰ =hs⁰, `⁰,t⁰iπ_t⁰0, where π⁰_t0 is a goal path of lengthi fromt⁰. Then there is for eacht ∈σ⁻¹(t⁰) a goal pathπt

of the same cost inT_X. Furthermore, for alls ∈σ⁻¹(s⁰) there is a label`∈λ⁻¹(`⁰) such thatT_X has a transitionhs, `,tiwith t∈σ⁻¹(t⁰). The path π=hs, `,tiπ_t is a solution for s in T. As` and`⁰ must have the same cost andπ_t andπ⁰_t0 have the same cost,π has the same cost asπ⁰.

(38)

Sequences of Transformations

Theorem (Sequences of Transformations)

Let X1, . . . ,Xn be collections of transition systems.

If for i∈ {1, . . . ,n−1} the transformation from X_i to X_i₊₁ is safe (exact) then the transformation from X₁ to X_nis safe (exact).

Proof idea: The composition of the σ andλfunctions of each step satisfy the required conditions.

(39)

Consequences

Generic Merge & Shrink Algorithm abs := {T^π^{v} |v ∈V} =: X₀

while abs contains more than one abstract transition system:

select A₁,A₂ fromabs

shrink A₁ and/or A₂ untilsize(A₁)·size(A₂)≤N abs := abs\ {A₁,A₂} ∪ {A₁⊗ A₂}

returnthe remaining abstract transition system in abs Initially T_abs is the concrete transition system.

Each iteration performs a safe transformation ofabs.

→ the corresponding abstraction heuristic is safe, goal-aware,

→ consistent, and admissible.

If shrinking is exact, the corresponding heuristic is perfect.

(40)

Summary

(41)

Summary

Projections perfectly reflecta fewstate variables.

Merge-and-shrink abstractions are ageneralization that can reflect allstate variables, but in apotentially lossy way.

The merge stepscombine two abstract transition systems by replacing them with theirsynchronized product.

The shrink stepsmake an abstract system smaller by abstracting it further.

As we only use safe transformations, the resulting heuristic is alwaysadmissible.

If we use only exacttransformations, the resulting heuristic is perfect.