Planning and Optimization D8. M&S: Strategies and Label Reduction Gabriele R¨oger and Thomas Keller

(1)

Planning and Optimization

D8. M&S: Strategies and Label Reduction

Gabriele R¨oger and Thomas Keller

Universit¨at Basel

November 7, 2018

G. R¨oger, T. Keller (Universit¨at Basel) Planning and Optimization November 7, 2018 1 / 47

Planning and Optimization

November 7, 2018 — D8. M&S: Strategies and Label Reduction

D8.1 Merging Strategies D8.2 Shrinking Strategies D8.3 Label Reduction D8.4 Summary

D8.5 Literature

Content of this Course

Planning

Classical

Tasks Progression/

Regression Complexity Heuristics

Probabilistic

MDPs Uninformed Search

Heuristic Search

Content of this Course: Heuristics

Heuristics

Delete Relaxation

Abstraction

Abstractions in General

Pattern Databases

Merge &

Shrink Landmarks

Potential Heuristics Cost Partitioning

(2)

D8. M&S: Strategies and Label Reduction Merging Strategies

D8.1 Merging Strategies

Content of this Course: Merge & Shrink

Merge & Shrink

Synchronized Product Merge & Shrink Algorithm

Heuristic Properties Strategies Label Reduction

Generic Algorithm Template

Generic M&S computation algorithm abs := {T^π^{v} |v ∈V}

whileabs contains more than one abstract transition system:

selectA₁,A₂ from abs

shrinkA₁ and/orA₂ until size(A₁)·size(A₂)≤N abs :=abs \ {A₁,A₂} ∪ {A₁⊗ A₂}

return the remaining abstract transition system inabs Remaining question:

I Which abstractions to select? merging strategy

Linear Merging Strategies

Linear Merging Strategy

In each iteration after the first, choose the abstraction computed in the previous iteration as A₁.

Rationale: only maintains one “complex” abstraction at a time Fully defined by an ordering of atomic projections.

(3)

Linear Merging Strategies: Choosing the Ordering

Use similar causal graph criteria as for growing patterns.

Example: Strategy of h_HHH

h_HHH: Ordering of atomic projections

I Start with a goal variable.

I Add variables that appear in preconditions of operators affecting previous variables.

I If that is not possible, add a goal variable.

Rationale: increasesh quickly

Non-linear Merging Strategies

I Non-linear merging strategies only recently gained more interest in the planning community.

I One reason: Better label reduction techniques (later in this chapter) enabled a more efficient computation.

I Examples:

I DFP: preferrably merge transition systems that must synchronize on labels that occur close to a goal state.

I UMCandMIASM: Build clusters of variables with strong interactions and first merge variables within each cluster.

I Each merge-and-shrink heuristic computed with a non-linear merging strategy can also be computed with a linear merging strategy.

I However, linear merging can require a super-polynomial blow-up of the final representation size.

D8. M&S: Strategies and Label Reduction Shrinking Strategies

D8.2 Shrinking Strategies

Content of this Course: Merge & Shrink

Merge & Shrink

(4)

Generic Algorithm Template

Generic M&S computation algorithm abs := {T^π^{v} |v ∈V}

whileabs contains more than one abstraction:

selectA₁,A₂ from abs

shrinkA₁ and/orA₂ until size(A₁)·size(A₂)≤N abs :=abs \ {A₁,A₂} ∪ {A₁⊗ A₂}

return the remaining abstraction inabs

N: parameter bounding number of abstract states Remaining Questions:

I Which abstractions to select? merging strategy

I How to shrink an abstraction? shrinking strategy

Shrinking Strategies

How to shrink an abstraction?

We cover two common approaches:

I f-preserving shrinking

I bisimulation-based shrinking

f -preserving Shrinking Strategy

f-preserving Shrinking Strategy

Repeatedly combine abstract states with identical abstract goal distances (h values) and identical abstract initial state distances (g values).

Rationale: preserves heuristic value and overall graph shape Tie-breaking Criterion

Prefer combining states whereg +h is high.

In case of ties, combine states where h is high.

Rationale: states with high g+h values are less likely to be explored by A^∗, so inaccuracies there matter less

Bisimulation

Definition (Bisimulation)

LetT =hS,L,c,T,s₀,S_?i be a transition system. An equivalence relation∼on S is abisimulation for T if for everyhs, `,s⁰i ∈T and everyt ∼s there is a transitionht, `,t⁰i ∈T with t⁰ ∼s⁰. A bisimulation∼isgoal-respectingif s ∼t implies that either s,t ∈S_? ors,t 6∈S_?.

(5)

Bisimulation: Example

1

2

3

4

5

o p

o

o p

q o q

o

p

∼with equivalence classes {{1,2,5},{3,4}}is a goal-respecting bisimulation.

Bisimulations as Abstractions

Theorem (Bisimulations as Abstractions)

LetT =hS,L,c,T,s₀,S_?i be a transition system and∼be a bisimulation forT. Thenα_∼:S → {[s]_∼|s ∈S}with α_∼(s) = [s]_∼ is an abstraction of T .

Note: [s]_∼ denotes the equivalence class ofs.

Note: Surjectivity follows from the definition of the codomain Note: as the image ofα_∼.

Abstractions as Bisimulations

Definition (Abstraction as Bisimulation)

Let T =hS,L,c,T,s₀,S_?ibe a transition system andα:S →S⁰ be an abstraction of T. The abstraction induces the equivalence relation∼_α as s ∼_αt iff α(s) =α(t).

We say that αis a (goal-respecting) bisimulation for T if ∼_α is a (goal-respecting) bisimulation forT.

Abstraction as Bisimulations: Example

Abstractionα with

α(1) =α(2) =α(5) =Aandα(3) =α(4) =B is a goal-respecting bisimulation forT.

T

1

2

3

4

5

o p

o

o p

q o q

o

p

T^α

A B

o p

o,q

(6)

Goal-respecting Bisimulations are Exact (1)

Theorem

Let X be a collection of transition systems. Let α be an

abstraction for T_i ∈X . Ifα is a goal-respecting bisimulation then the transformation from X to X⁰ := (X \ {T_i})∪ {T_i^α}is exact.

Proof.

Let T_X =T₁⊗ · · · ⊗ T_n =hS,L,c,T,s₀,S_?iand w.l.o.g.

T_X⁰ =T₁⊗ · · · ⊗ Ti−1⊗ T_i^α⊗ T_i+1⊗ · · · ⊗ T_n =hS⁰,L⁰,c⁰,T⁰,s₀⁰,S_?⁰i.

Consider σ(hs₁, . . . ,s_ni) =hs₁, . . . ,s_i−1, α(s_i),s_i+1, . . . ,s_ni for the mapping of states andλ= id for the mapping of labels.

1 Mappingsσ andλsatisfy the requirements of safe transformations becauseα is an abstraction and we have chosen the mapping functions as before.

. . .

Goal-respecting Bisimulations are Exact (2)

Proof (continued).

2 If hs⁰, `,t⁰i ∈T⁰ with s⁰ =hs₁⁰, . . . ,s_n⁰i andt⁰=ht₁⁰, . . . ,t_n⁰i, then for j 6=i transition system T_j has transitionhs_j⁰, `,t_j⁰i (*) andT_i^α has transitionhs_i⁰, `,t_i⁰i. This implies thatT_i has a transitionhs_i⁰⁰, `,t_i⁰⁰i for some s_i⁰⁰∈α⁻¹(s_i⁰) andt_i⁰⁰∈α⁻¹(t_i⁰).

As αis a bisimulation, there must be such a transition for all such s_i⁰⁰ andt_i⁰⁰ (**).

Each s ∈σ⁻¹(s⁰) has the form s =hs₁, . . . ,s_ni with s_j =s_j⁰ for j 6=i ands_i ∈α⁻¹(s_i⁰). Analogously for each

t =ht₁, . . . ,t_ni ∈σ⁻¹(t⁰). From (*) and (**) follows thatT_j has a transitionhs_j, `,t_ji for allj ∈ {1, . . . ,n}, so for each such s andt,T contains the transitionhs, `,ti.

. . .

Goal-respecting Bisimulations are Exact (3)

Proof (continued).

3 Fors_?⁰ =hs₁⁰, . . . ,s_n⁰i ∈S_?⁰, each s_j⁰ with j 6=i must be a goal state of T_j (*) ands_i⁰ must be a goal state ofT_i^α. The latter implies that at least ons_i⁰⁰∈α⁻¹(s_i⁰) is a goal state ofT_i. As α is goal-respecting, all states fromα⁻¹(s_i⁰) are goal states of T_i (**).

Considers_? =hs₁, . . . ,s_ni ∈σ⁻¹(s_?⁰). By the definition ofσ, s_j =s_j⁰ forj 6=i ands_i ∈α⁻¹(s_i⁰). From (*) and (**), eachs_j (j ∈ {1, . . . ,n}) is a goal state ofT_j and, hence, s_? a goal state of T_X.

4 Asλ= id and the transformation does not change the label cost function, c(`) =c⁰(λ(`)) for all`∈L.

Bisimulations: Discussion

I As all bisimulations preserve all relevant information, we are interested in the coarsestsuch abstraction (to shrink as much as possible).

I There is always a unique coarsest bisimulation for T and it can be computed efficiently (from the explicit representation).

I In some cases, computing the bisimulation is still too expensive or it cannot sufficiently shrink a transition system.

(7)

Greedy Bisimulations

Definition (Greedy Bisimulation)

Let T =hS,L,c,T,s₀,S_?ibe a transition system. An equivalence relation∼on S is agreedy bisimulation forT if it is a bisimulation for the system hS,L,c,T^G,s₀,S_?i, where

T^G ={hs, `,ti | hs, `,ti ∈T,h^∗(s) =h^∗(t) +c(`)}.

Greedy bisimulation only considers transitions that are used in an optimal solution of some state of T.

Greedy Bisimulation is h-preserving

Theorem

LetT be a transition system and let αbe an abstraction of T. If

∼_α is a goal-respecting greedy bisimulation forT then h^∗_Tα =h_T^∗. (Proof omitted.)

Note: This does not mean that replacingT with T^α in a collection of transition systems is a safe transformation! Abstraction α preserves solution costs “locally” but not “globally”.

D8. M&S: Strategies and Label Reduction Label Reduction

D8.3 Label Reduction

Content of this Course: Merge & Shrink

Merge & Shrink

(8)

Label Reduction: Motivation (1)

T

5

o,o⁰ p

o

o p

q

o,o⁰ q

o

p⁰

T⁰

o,o⁰ o,o⁰,p,p⁰,q

Whenever there is a transition with labelo⁰ there is also a

transition with label o. If o⁰ is not cheaper thano, we can always use the transition with o.

Idea: Replaceo ando⁰ with labelo⁰⁰ with cost of o

Label Reduction: Motivation (2)

T

s t

o⁰⁰ p

o⁰⁰

o p

q

o⁰⁰ q

o⁰⁰

p⁰

T⁰

o⁰⁰

o⁰⁰,p,p⁰,q

Statess andt are not bisimilar due to labels p andp⁰. InT⁰ they label the same (parallel) transitions. Ifp andp⁰ have the same cost, in such a situation there is no need for distinguishing them.

Idea: Replacep andp⁰ with labelp⁰⁰ with same cost.

Label Reduction: Motivation (3)

T

s t

o⁰⁰ p⁰⁰

o⁰⁰

p⁰⁰ o

q

o⁰⁰ q

o⁰⁰

p⁰⁰

T⁰

o⁰⁰ o⁰⁰,p⁰⁰,q

Label reductions reduce the time and memory requirement for merge and shrink steps and enable coarser bisimulation abstractions.

When is label reduction a safe transformation?

Label Reduction: Definition

Definition (Label Reduction)

LetX be a collection of transition systems with label setLand label cost functionc. Alabel reduction hλ,c⁰i for X is given by a functionλ:L→L⁰, where L⁰ is an arbitrary set of labels, and a label cost functionc⁰ on L⁰ such that for all`∈L,c⁰(λ(`))≤c(`).

ForT =hS,L,c,T,s₀,S_?i ∈X the label-reduced transition system isT^hλ,c⁰ⁱ =hS,L⁰,c⁰,{hs, λ(`),ti | hs, `,ti ∈T},s₀,S_?i.

Thelabel-reduced collectionis X^hλ,c⁰ⁱ={T^hλ,c⁰ⁱ| T ∈X}.

L⁰∩L6=∅ andL⁰ =Lare allowed.

(9)

Label Reduction is Safe (1)

Theorem (Label Reduction is Safe)

Let X be a collection of transition systems and hλ,c⁰i be a

label-reduction for X . Thetransformation from X to X^hλ,c⁰ⁱ is safe.

Proof.

We show that the transformation is safe, using σ= id for the mapping of states andλ for the mapping of labels.

The label cost function of T_Xhλ,c0i isc⁰ and has the required property by the definition of label reduction. . . .

Label Reduction is Safe (2)

Theorem (Label Reduction is Safe)

Let X be a collection of transition systems andhλ,c⁰ibe a

label-reduction for X . Thetransformation from X to X^hλ,c⁰ⁱis safe.

Proof (continued).

By the definition of synchronized products,T_X has a transition hhs₁, . . . ,s_|X_|i, `,ht₁, . . . ,t_|X|iiif for alli,T_i ∈X has a transition hs_i, `,t_ii. By the definition of label-reduced transition systems, this implies thatT^hλ,c⁰ⁱ has a corresponding transitionhs_i, λ(`),t_ii, so

T_Xhλ,c0i has a transitionhs, λ(`),ti=hσ(s), λ(`), σ(t)i (definition

of synchronized products).

For each goal state s_? of T_X, stateσ(s_?) =s_? is a goal state of

T_Xhλ,c0i because the transformation replaces each transition system

with a system that has the same goal states.

More Terminology

Let X be a collection of transition systems with labelsL. Let

`, `⁰ ∈Lbe labels and let T ∈X.

I Label `isalive in X if allT⁰ ∈X have some transition labelled with`. Otherwise, `is dead.

I Label `locally subsumeslabel`⁰ in T if for all transitions hs, `⁰,ti ofT there is also a transition hs, `,ti inT.

I `globally subsumes`⁰ if it locally subsumes `⁰ in all T⁰∈X.

I `and`⁰ are locally equivalentin T if they label the same transitions inT, i.e.`locally subsumes`⁰ in T and vice versa.

I `and`⁰ are T-combinableif they are locally equivalent in all transition systemsT⁰ ∈X \ {T }.

Exact Label Reduction

Theorem (Criteria for Exact Label Reduction)

Let X be a collection of transition systems with cost function c and label set L that contains no dead labels.

Lethλ,c⁰ibe a label-reduction for X such that λ combines labels

`₁ and`₂ and leaves other labels unchanged. The transformation from X to X^hλ,c⁰ⁱ is exactiff c(`₁) =c(`₂), c⁰(λ(`)) =c(`) for all

`∈L, and

I `₁ globally subsumes`₂, or

I `₂ globally subsumes`₁, or

I `₁ and`₂ are T-combinable for someT ∈X . (Proof omitted.)

(10)

Back to Example (1)

T

5

o,o⁰ p

o

o p

q

o,o⁰ q

o

p⁰

T⁰

o,o⁰ o,o⁰,p,p⁰,q

Label o globally subsumes labelo⁰.

Back to Example (2)

T

s t

o⁰⁰ p

o⁰⁰

o p

q

o⁰⁰ q

o⁰⁰

p⁰

T⁰

o⁰⁰

o⁰⁰,p,p⁰,q

Labels p andp⁰ are T-combinable.

Computation of Exact Label Reduction (1)

I For given labels`₁, `₂, the criteria can be tested in low-order polynomial time.

I Finding globally subsumed labels involves finding subset relationsships in a set family.

no linear-time algorithms known

I The following algorithm exploits onlyT-combinability.

Computation of Exact Label Reduction (2)

eq_i := set of label equivalence classes ofT_i ∈X Label-reduction based onT_i-combinability

eq:={L}

forj ∈ {1, . . . ,|X|} \ {i}

Refineeq with eq_j

// two labels are in the same set ofeq

// iff they are locally equivalent in allT_j 6=T_i. λ= id

forB ∈eq

samecost := {[`]∼_c |`∈B, `⁰ ∼_c `⁰⁰ iff c(`⁰) =c(`⁰⁰)}

for L⁰ ∈samecost

`_new := new label

c⁰(`_new) := cost of labels in L⁰ for `∈L⁰

λ(`) =`_new

(11)

Application in Merge-and-Shrink Algorithm

Generic M&S Computation Algorithm with Label Reduction abs := {T^π^{v} |v ∈V}

whileabs contains more than one abstract transition system:

selectT₁,T₂ from abs

possiblylabel-reduce allT ∈abs

(e.g. based onT₁- and/orT₂-combinability).

shrinkT₁ and/orT₂ until size(T₁)·size(T₂)≤N possiblylabel-reduce allT ∈abs

abs :=abs \ {T₁,T₂} ∪ {T₁⊗ T₂}

return the remaining abstract transition system inabs

D8. M&S: Strategies and Label Reduction Summary

D8.4 Summary

D8. M&S: Strategies and Label Reduction Summary

Summary

I Bisimulationis anexactshrinking method.

I There is a wide range of merging strategies. We only covered some important ones.

I Label reductionis crucial for the performance of the

merge-and-shrink algorithm, especially when using bisimilarity for shrinking.

D8. M&S: Strategies and Label Reduction Literature

D8.5 Literature

(12)

Literature (1)

References on merge-and-shrink abstractions:

Klaus Dr¨ager, Bernd Finkbeiner and Andreas Podelski.

Directed Model Checking with Distance-Preserving Abstractions.

Proc. SPIN 2006, pp. 19–34, 2006.

Introduces merge-and-shrink abstractions (for model-checking) andDFP merging strategy.

Malte Helmert, Patrik Haslum and J¨org Hoffmann.

Flexible Abstraction Heuristics for Optimal Sequential Planning.

Proc. ICAPS 2007, pp. 176–183, 2007.

Introduces merge-and-shrink abstractionsfor planning.

Literature (2)

Raz Nissim, J¨org Hoffmann and Malte Helmert.

Computing Perfect Heuristics in Polynomial Time: On Bisimulation and Merge-and-Shrink Abstractions in Optimal Planning.

Proc. IJCAI 2011, pp. 1983–1990, 2011.

Introducesbisimulation-based shrinking.

Malte Helmert, Patrik Haslum, J¨org Hoffmann and Raz Nissim.

Merge-and-Shrink Abstraction: A Method for Generating Lower Bounds in Factored State Spaces.

Journal of the ACM 61 (3), pp. 16:1–63, 2014.

Detailedjournal versionof the previous two publications.

Literature (3)

Silvan Sievers, Martin Wehrle and Malte Helmert.

Generalized Label Reduction for Merge-and-Shrink Heuristics.

Proc. AAAI 2014, pp. 2358–2366, 2014.

Introduces label reductionas covered in these slides (there has been a more complicated version before).

Gaojian Fan, Martin M¨uller and Robert Holte.

Non-linear merging strategies for merge-and-shrink based on variable interactions.

Proc. AAAI 2014, pp. 2358–2366, 2014.

Introduces UMC and MIASM merging strategies