Merging Quality Estimation for Binary Decision Diagrams with Binary Classiﬁers

(1)

Merging Quality Estimation for Binary Decision Diagrams with Binary Classifiers

LOD 2019, September 10-13, Certosa di Pontignano, Siena, Tuscany, Italy

Nikolaus Frohner and G¨unther R. Raidl

September 11, 2019

N. Frohner and G.R. Raidl Merging Quality Estimation September 11, 2019 1 / 25

(2)

Binary Decision Diagrams (BDDs)

Introduced by Lee in 1959 as compact representation of boolean functions and further elaborated on by Akers in eponymous “Binary decision

diagrams” (1978) as rooted, directed, acyclic, multigraphs.

Figure: adapted from “Binary decision diagrams” by Akers, page 2

(3)

BDDs in Combinatorial Optimization

Introduced into the field of combinatorial optimization by Hadzic and Hooker (2006), for post-optimality analysis.

Representation of solution space where paths represent solutions with associated objective value and longest paths correspond to maxima.

(4)

Top-Down Construction of BDDs

(from Hooker (2016))

(5)

Top-Down Construction of BDDs

(6)

Top-Down Construction of BDDs

(7)

Relaxed BDDs

• Provides new means for relaxation, besides for example Linear Programming based or Lagrangian.

• Relaxed BDD represents superset of all feasible solutions.

• BDD kept compact by merging also nodes for which states are not the same →longest paths then usually correspond to upper bound for represented problem instance.

Decision of which nodes to merge is job of merging heuristic.

(8)

Top-Down Construction of Relaxed BDDs

(9)

Top-Down Construction of Relaxed BDDs

(10)

Classical minimum longest path (minLP) Selection

r₁

{5,6} {4} {4,5} {5} {6,7} {7,8}

Order nodes in given layer by longest path (LP) length from root (r₁).

States of nodes are represented by sets of elements that can still be selected.

(11)

Classical minLP Merging

r₁

{5,6} {4} {4,5} {5,6,7,8}

Merges nodes from the back into one node.

May result into nodes with large states, yielding higher upper bounds for the resulting nodes, since more infeasible pathsare likely to be

introduced.

(12)

Similarity Based Merging

• We introduced minLP and state similarity based hybrid merging heuristic that improved bounds for small width BDDs for the Maximum Independent Set Problem (MISP) and the Set Cover Problem (SCP) (see our LION 13 paper) via tie breaking.

• Issue for weighted MISP instances, where ties are less likely to occur.

(13)

Merging Quality

Idea: do not always (i.e., in each layer) apply the same merging heuristic, instead go for the “locally best” one out of a set of merging heuristics.

We define the locally best as: For which the completition of the decision diagram using minLP merging would result in the tightest bound.

(14)

Perfect Lookahead

Given the nodes of a layer that needs to be reduced in width:

• Apply all available merging heuristics, including minLP, on shallow BDD copy and finish construction using minLP.

• Finally, apply the merging heuristic that yielded best bound.

(15)

k -layers Lookahead

Lookahead used by Bergman et al. (2012) for dynamic variable ordering, we use it in the context of merging.

Perfect lookahead too expensive but gives us ground truth:

f(H,H⁰) = 1, ifH yields a strictly tighter bound thanH⁰, otherwise 0.

(16)

k -layers Lookahead

Only look k-layers ahead and gather p features by layerY for two merging heuristics and estimate which one will result in a better final bound.

Binary classification function:

h˜:R^p×k×R^p×k →[0,1].

h_α(Y,Y⁰) =

(0, h(Y,˜ Y⁰)< α 1, h(Y,˜ Y⁰)≥α

(17)

Possible Features

• min/mean/max of longest path values of nodes in layer

• min/mean/max of problem specific upper bound values of nodes in layer

• layer progress l/l_max

(18)

Linear Approximation of Bound Growth

Noisy, linear growth of bounds (maximum longest path values) over layers.

0 25 50 75 100 125 150 175 200

layer 0

1000 2000 3000 4000 5000

max lp

brock200_1.clq, = 10, r²= 0.9953

0 200 400 600 800 1000

layer 0

500 1000 1500 2000 2500 3000

mean lp

san1000.clq, = 10, r²= 0.9973

(19)

Classifiers

• Linear regression considering differences ∆Y=Y_minLP−Y_H.

• max-maxLP: compare the maximum of the maximum of the longest path values over all looked-ahead layers.

• Wilcoxon signed rank sum test on paired features YminLP,Y_H.

• Neural network based classifier.

(20)

LR Examples

• Left a true positive (minLP worse than H).

• Right a true negative (minLP not worse thanH).

0 10 20 30 40 50

looked ahead layer 100

0 100 200 300 400 500

max LP difference

p-val=1.000 r²=0.899

max LP differences over looked ahead layers for minLP vs H

0 10 20 30 40 50

looked ahead layer 150

100 50 0 50 100

max LP difference

p-val=0.529 r²=0.104

max LP differences over looked ahead layers for minLP vs H

(21)

Train & Test Data

• Take random layer in random graph and apply hybrid merging with random parameters.

• Finish construction of BDD to see which one performs better, yielding the features and ground truth.

• Created 21000 training & test samples, approximately balanced, from 1000 random weighted graphs.

(22)

Neural Network Classifier

Use more features

• max of the longest path valuesz^lp(u)

• max of the upper bound values z_MISP^ub (u)

• layer progress l/l_max

resulting in input dimension of 2k+ 1 NN outputs value between 0.0 and 1.0.

(23)

Classifier Comparison

Precision-recall comparison on training data.

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 recall

0.40 0.45 0.50 0.55 0.60 0.65 0.70

precision

Precision-recall curve for classifiers for k=50 LRWC NNmax

10 20 30 40 50 60 70 80 90

look-ahead layer length k 0.40

0.45 0.50 0.55 0.60 0.65 0.70

precision-recall AUC

Precision-recall AUC over look ahead layers for classifiers maxLR WCNN

(24)

Classifiers on Weighted DIMACS

Baseline merging heuristic is minLP with tie breaking and the competing algorithm uses the raced parameter set (0.185,0.043), evaluated by k = 50 layers lookahead with Wilcoxon classifier.

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5

A: minLP 0.0

0.5 1.0 1.5 2.0 2.5 3.0 3.5

B: 50-layers lookahead Wilcoxon

MedA= 1.77 MedB= 1.61 Med = 0.12 Wilcoxon p-val = 0.000

Relative Upper Bound Comparison per weighted DIMACS instance

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0

A: minLP with state similarity (0.185, 0.043) 0.0

0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0

B: 50-layers lookahead Wilcoxon

MedA= 1.67 MedB= 1.61 Med = 0.05 Wilcoxon p-val = 0.000

Relative Upper Bound Comparison per weighted DIMACS instance

(25)

Results on Weighted DIMACS

¹

Weighted MISP relative bound improvements for different classifiers, parameters, and compared to different baseline approaches.

∆,e ∆ : median/mean improvement of relative bounds.

comparing approach PLA k max LR WC NN

∆e ∆ ∆e ∆ ∆e ∆ ∆e ∆ ∆e ∆

pure minLP 0.16 0.17

30 0.09 0.11 0.07 0.08 0.09 0.11 0.11 0.11 minLP with state similarity 0.09 0.11 0.04 0.06 0.02 0.03 0.04 0.06 0.04 0.06

1https://github.com/jamestrimble/max- weight-clique-instances/tree/master/DIMACS

(26)

Conclusion & Future Work

We could improve relaxed BDD bounds of weighted MISP instances using lookahead mechanism as compared to pure classic minLP merging

heuristic.

• Main issue: computationally very expensive.

• Search for stronger features to identity “locally best” merging heuristic with less effort.

• Improve classification to reduce lookahead length.

• Test with reduced BDD width for lookahead.

• Test on other problems, weighted set cover problem as next goal.

(27)

References

David Bergman, Andre A Cire, Willem-Jan van Hoeve, and John N Hooker.

Variable ordering for the application of BDDs to the maximum independent set problem.

InInternational Conference on Integration of Artificial Intelligence (AI) and Operations Research (OR) Techniques in Constraint Programming, pages 34–49. Springer, 2012.

David Bergman, Andre A Cire, Willem-Jan van Hoeve, and John N Hooker.

Optimization bounds from binary decision diagrams.

INFORMS Journal on Computing, 26(2):253–268, 2013.

Nikolaus Frohner and G¨unther R. Raidl.

Towards improving merging heuristics for binary decision diagrams.

In Nikolaos F. Matsatsinis, Yannis Marinakis, and Panos Pardalos, editors,Learning and Intelligent Optimization – 13th International Conference, LION 13, volume 11968 ofLNCS, pages 30–45. Springer, 2019.

URLhttps://www.ac.tuwien.ac.at/files/pub/frohner-19a.pdf.

(28)

Merging Quality Estimation for Binary Decision Diagrams with Binary Classiﬁers

Merging Quality Estimation for Binary Decision Diagrams with Binary Classifiers

Binary Decision Diagrams (BDDs)

BDDs in Combinatorial Optimization

Top-Down Construction of BDDs

Top-Down Construction of BDDs

Top-Down Construction of BDDs

Relaxed BDDs

Top-Down Construction of Relaxed BDDs

Top-Down Construction of Relaxed BDDs

Classical minimum longest path (minLP) Selection

Classical minLP Merging

Similarity Based Merging

Merging Quality

Perfect Lookahead

k -layers Lookahead

k -layers Lookahead

Possible Features

Linear Approximation of Bound Growth

Classifiers

LR Examples

Train & Test Data

Neural Network Classifier

Classifier Comparison

Classifiers on Weighted DIMACS

Results on Weighted DIMACS

Conclusion & Future Work

References

Thank you