https://doi.org/10.1007/s10844-021-00666-5
An overview of machine learning techniques in constraint solving
Andrei Popescu1·Seda Polat-Erdeniz1·Alexander Felfernig1·Mathias Uta2· M ¨usl ¨um Atas1·Viet-Man Le1·Klaus Pilsl3·Martin Enzelsberger3·
Thi Ngoc Trang Tran1
Received: 16 April 2021 / Revised: 5 August 2021 / Accepted: 9 August 2021 /
©The Author(s) 2021
Abstract
Constraint solving is applied in different application contexts. Examples thereof are the con- figuration of complex products and services, the determination of production schedules, and the determination of recommendations in online sales scenarios. Constraint solvers apply, for example, search heuristics to assure adequate runtime performance and prediction qual- ity. Several approaches have already been developed showing that machine learning (ML) can be used to optimize search processes in constraint solving. In this article, we provide an overview of the state of the art in applying ML approaches to constraint solving problems including constraint satisfaction, SAT solving, answer set programming (ASP) and appli- cations thereof such as configuration, constraint-based recommendation, and model-based diagnosis. We compare and discuss the advantages and disadvantages of these approaches and point out relevant directions for future work.
Keywords Constraint satisfaction·Boolean satisfiability·Constraint solving·Answer set programming·Machine learning·Applications
1 Introduction
Over several decades,constraint solving(we use this as a general term referring to specific problem solving approaches such as constraint satisfaction (Freuder,1997), SAT solving (Gu et al., 1996), and answer set programming (Brewka et al.,2011)) has shown to be a core technology of Artificial Intelligence (Apt,2003; Brailsford et al.,1999; Rossi et al., 2006). Its widespread use led to the development of algorithms and tools that help to tackle
Andrei Popescu
andrei.popescu@ist.tugraz.at
Extended author information available on the last page of the article.
Published online: 30 August 2021
a variety of combinatorial problems. Basic examples thereof are the n-queens problem and car sequencing (Adorf & Johnston,1990; Tsang,1993). Then-queens problemis related to the task of placingnqueens onndistinct squares of ann×nchessboard in such a way that no pair of queens attacks each other. This problem has been extensively used to illustrate the modeling of constraint solving tasks and related search algorithms. Thecar sequencing problemis another problem that can be tackled on the basis of constraint solving tech- niques. This problem is related to the task of scheduling a car production sequence in such a way that capacity constraints of the assembly line are satisfied. Cars can be of different type and thus need different features to be installed and, as a consequence, also different production steps need to be taken into account. For example, atype-Acar might need a sunroof installedbutno skibag. An assembly line is often composed of different sections, each specialized in installing one or more specific features. Each section, however, has a limited capacity which has to be taken into account by the solver. Another application area of constraint-based knowledge representations and reasoning is knowledge-based configu- ration (Felfernig et al.,2014). Configuration has become a core technology for businesses depending on the mass customization paradigm (Felfernig et al.,2014). Configuration tech- nologies help to tackle important challenges triggered by mass customization, for example, due to the correctness of configurations, errors in processing orders and related follow-up costs due to the production of faulty configurations can be avoided. Beyond configura- tion, constraint-based technologies are applied in a variety of further scenarios such as scheduling, vehicle routing, and recommender systems just to mention a few (Burke,2000;
Felfernig & Burke,2008; Rossi et al.,2006).
As indicated by a couple of research results, combinatorial problems such as configura- tion problems have no universal best solving approach (Bengio et al.,2021; Kotthoff,2014;
Loreggia et al.,2016; O’Mahony et al.,2013; Spieker & Gotlieb,2018; Xu et al.,2008;
Xu et al.,2009). To solve such problems in an efficient fashion, state of the art techniques exploitsearch heuristics(Arbelaez et al.,2010; Beck et al.,2004; Da Col & Teppan,2017;
Erdeniz et al.,2017; Jannach,2013; Johnston & Minton,1994; Mouhoub & Jafari,2011;
Pearl,1984; Sadeh & Fox,1996) or try to find the best solver (algorithm) for the problem instance at hand.Algorithm selectionapproaches have shown to be applicable in various scenarios – related approaches to predict a solver’s performance become an increasingly active field of research (Kotthoff,2014). Furthermore, the development of machine learn- ing methods that are capable oflearning search heuristicsin a domain-dependent fashion is relevant in the context of minimizing search efforts and optimizing the prediction quality (Raedt et al.,2011). Machine learning can also be used to predict the satisfiability of a spe- cific problem and thus can help to significantly reduce the overall search effort. We have identifiedthe following main application areasof machine learning techniques in the con- text of constraint solving: (1)direct solution search, (2)constraint learning, (3)learning search heuristics (and/or parameters)(e.g., related to possible variable and value orderings), (4)satisfiability prediction (by learning relevant problem features that help to correctly classify problems as (un-)satisfiable), and (5) algorithm selection (based on problem properties/features).
There exist differentsurveysrelated to the topics ofconstraint satisfaction and SAT solv- ing (Apt,2003; Rossi et al., 2006; Tsang,1993), algorithm portfolios(Kotthoff, 2014), and alsointeractions between data mining and constraint programming(Bessiere et al., 2016). A survey of different algorithmic approaches to solve constraint satisfaction prob- lems is provided, for example, by Kumar (1992). An overview of different approaches to SAT solving is provided, for example, by Gu et al. (1996). Compared to the literature study
presented in this article, commonalities exist with the overviews provided by Kotthoff (2014), Bischl et al. (2015), and Bessiere et al. (2016). Kotthoff (Kotthoff,2014) provides a survey of algorithm portfolios that allow to select the “best” algorithm for a specific problem instance. Bischl et al. (2015) propose an algorithm benchmark including different datasets for evaluation purposes. Finally, Bessiere et al. (2016) focus on synergies between data mining and constraint satisfaction. They focus on related aspects such as approaches to algorithm selection, constraint learning, and representing data mining problems as con- straint satisfaction problems. In this line of research, Alves Pereira et al. (2019) motivate the application of machine learning methods for the purpose of configuration space learn- ing. The authors propose the application of machine learning methods for the purpose of predicting non-functional system properties such as response time or system failure proba- bilities. Furthermore, overviews of the state of the art in algorithm runtime prediction and selection can be found in Hutter et al. (2014) and Lombardi and Milano (2018). Finally, an overview of recent research related to the application of machine learning methods in the context of combinatorial optimization can be found in Bengio et al. (2021).
The major contribution of this article is the following. There does not exist an in-depth overview of machine learning methods aiming at improving constraint solving (in areas such as constraint satisfaction, SAT solving, and answer set programming). Although the interest in the field of applying machine learning to the solution of combinatorial problems is growing (Bengio et al.,2021), an overview specifically focusing on the application of machine learning approaches to constraint solving is still missing. With this article, we provide such an overview and also include an in-depth discussion of challenges and open issues for future work.
The remainder of this article is organized as follows. In Section2, we present our method- ological approach to the literature analysis on machine learning and constraint solving.
Section3provides an introduction to major relevant concepts from the areas ofconstraint solvingandmachine learningthat are regarded as a basis for being able to understand the discussions in Section4. In Section4, we summarize the results of our literature analysis on the integration of machine learning techniques into different types of constraint solving and provide selected examples to increase understandability. Specifically, we focus on related work in constraint satisfaction, SAT solving, answer set programming, and related appli- cations in configuration, diagnosis, and constraint-based recommendation. In Section 5, we provide an overview of relevant issues for future research. The article is concluded in Section6.
2 Research methodology
Our goal is to provide a comprehensive overview of the state of the art in the application of machine learning methods in constraint solving. We want to provide an in-depth overview of the field and thus also contribute to further bridge building between the two research fields of machine learning and constraint solving. This article is based on a literature review com- prised of paper selection, review, and a discussion of reviewed approaches. Specifically, our work involved an initial step ofqueryingleadingresearch portalswith relevant keywords, an intermediary phase consisting ofclassifying approachesbased on theircontributionto Constraint Solvingand usedmachine learning methods, and a final phase with adiscussion of current challengesand relevantfuture research directions.
We performed queries on well-known research platforms such as Google Scholar1, ResearchGate2, ScienceDirect3, SpringerLink4, and Elsevier5 with the initial keywords
“machine learning” + “constraint satisfaction”. Taking into account the initial query results regarded as relevant, we established further search criteria (in terms of keywords) that combined in a systematic fashion “machine learning” and “constraint satisfaction”
with related terms such as “deep learning”, “data mining”, “matrix factorization” and “con- straint solving”, “SAT solving”, “Answer Set Programming”, and related meta-criteria such as “overview”, “review”, and “survey”. In addition, we analyzed topic-related work from the last 15 years published in theInternational Joint Conference on Artificial Intelligence (IJCAI), theAAAI Conference on Artificial Intelligence, theEuropean Conference on Artifi- cial Intelligence (ECAI), theConference on Theory and Applications of Satisfiability Testing (SAT), and theConstraint Programming (CP)conference. Following the snowballing tech- nique (Wohlin,2014), we further analyzed the reference sections of the identified papers with the goal to find further topic-related work. The major criterion for regarding a ref- erence as relevant was a topic-wise match, i.e., the reference needed to cover the topic of integrating machine learning (ML) techniques with constraint solving. We have identified 41 publications as a basis for our overview. We did not include contributions focusing on the integration of constraint solving into machine learning (see, e.g., Lallouet & Legtchenko, 2005). The majority of the identified publications is based on supervised ML (see Table1).
3 Preliminaries
In the following, we present relevantconstraint solving paradigms, such as constraint satis- faction (Freuder,1997), SAT solving (Gu et al.,1996) and answer set programming (Brewka et al.,2011), and key concepts from machine learning which constitute the basic knowledge required to be able to understand the discussions in this overview.
3.1 Constraint satisfaction problems (CSPs)
A constraint on a set of variables defines a restriction on possible combinations of vari- able assignments (Apt,2003). Constraint Satisfaction Problems (CSPs) specify a problem in terms of relevant variables and corresponding constraints on possible value combinations (Apt,2003). Ideally, an assignment should instantiate each variable and satisfy all the con- straints defining the problem. Examples of CSPs are the configuration of telecommunication switches (Fleischanderl et al.,1998), the recommendation of financial services (Felfernig et al.,2006), and reconfiguration (Felfernig et al.,2018). As a basis for the following dis- cussions, we now introduce a definition of a constraint satisfaction problem (CSP) (Rossi et al.,2006; Tsang,1993).
Definition (CSP and Solution). A CSP is defined as a triple (V , D, C)where V = {v1..vn} is a set of variables, D = {domain(v1)..domain(vn)} are the corresponding
1https://scholar.google.com
2https://www.researchgate.net
3https://www.sciencedirect.com
4https://link.springer.com
5https://www.elsevier.com
Table1Overviewofexampleintegrationsofmachinelearningwithconstraintsolvingcategorizedbasedonproblemtype(CSP,SAT,andASP)problemsandrelatedapplications suchasconfiguration,diagnosis,andconstraint-basedrecommendation ProblemtypePaperMLMethodApplicationEvaluationMetrics CSP(Gentetal.,2010)DecisionTreesConstraintLearningRuntime (Adorf&Johnston,1990)NeuralNetworksSolutionSearchCompleteness (Galassietal.,2018)DeepLearningSolutionSearchCompleteness (Xuetal.,2018)DeepLearningSatisfiabilityPredictionAccuracy (Gentetal.,2010)ClassificationAlgorithmSelectionRuntime (Wang&Tsang,1991)NeuralNetworksSolutionSearchRuntime (Xuetal.,2009)ReinforcementLearningLearningHeuristicsRuntime (Erdeniz&Felfernig,2018)Clustering,GeneticAlgorithmsLearningHeuristicsRuntime (Arbelaezetal.,2010)ClassificationLearningHeuristicsRuntime (Guerri&Milano,2004)DecisionTreesAlgorithmSelectionRuntime (Bonfiettietal.,2015)DecisionTreesandRandomForestsConstraintLearningRuntime (Belloetal.,2017)ReinforcementLearningSolutionSearchOptimality (Cappartetal.,2020)ReinforcementLearningLearningHeuristicsOptimality (Nareyek,2004)ReinforcementLearningLearningHeuristicsOptimality SAT(Loreggiaetal.,2016)DeepLearningAlgorithmSelectionAccuracy (O’Mahonyetal.,2013)Case-basedReasoningAlgorithmSelectionCompleteness (Xuetal.,2008)RidgeRegressionAlgorithmSelectionRuntime (B¨unz&Lamm,2017)GraphNeuralNetworksSatisfiabilityPredictionAccuracy (Selsametal.,2019)NeuralNetworksSatisfiabilityPredictionAccuracy (Hutteretal.,2006)LinearRegressionParameterTuningRuntime (Samulowitz&Memisevic,2007)LogisticRegressionLearningHeuristicsRuntime
Table1(continued) ProblemtypePaperMLMethodApplicationEvaluationMetrics (Lagoudakis&Littman,2001)ReinforcementLearningLearningHeuristicsRuntime (Yolcu&P´o,2019)ReinforcementLearningLearningHeuristicsCompleteness (Ans´oteguietal.,2009)GeneticAlgorithmsHyperParameterTuningRuntime (Cameronetal.,2020)DeepLearningSatisfiabilityPredictionAccuracy (Kurinetal.,2019)ReinforcementLearningLearningHeuristicsRuntime (Xuetal.,2012)DecisionTreesSatisfiabilityPredictionAccuracy (Haim&Walsh,2009)LogisticRegressionAlgorithmSelectionRuntime (Liangetal.,2016)ReinforcementLearningLearningHeuristicsRuntime ASP(Gebseretal.,2011)SupportVectorRegressionHyperParameterTuningRuntime (Marateaetal.,2012)Nearest-NeighbourClassificationAlgorithmSelectionRuntime (Yangetal.,2020)NeuralNetworksLearningHeuristicsAccuracy Configuration(Templeetal.,2017;Templeetal.,2016)DecisionTreesProblemModelLearningAccuracy (Acheretal.,2018)DecisionTreesConstraintLearningAccuracy (DaCol&Teppan,2017)GeneticAlgorithmsLearningHeuristicsRuntime (Uta&Felfernig,2020)NeuralNetworksSolutionSearchAccuracy Diagnosis(Erdenizetal.,2018)GeneticAlgorithmsLearningHeuristicsRuntime (Erdenizetal.,2019)MatrixFactorizationLearningHeuristicsAccuracy Constraint-basedRecommendation(Erdenizetal.,2019)MatrixFactorizationLearningHeuristicsAccuracy (Zanker,2008)CollaborativeFilteringConstraintLearningAccuracy
domain definitions, andC= {c1..ck}is a set of constraints. A solution to a CSP is a vari- able assignmentA = {v1 =valv1..vn =valvn}wherevalvj denotes the value of variable vj withvalvj ∈domain(vj)and consistent(C∪A).
A basic approach to solve a CSP is to applybacktracking searchcombined withfor- ward checking(analyzing whether a variable value assignment has an impact on the set of possible values of other variables) and different approaches assuring consistency proper- ties such asnode consistency(each value of a variable domain has to be consistent with all unary constraints defined on that variable) and arc consistency(for each binary con- straintca connecting two variables, e.g.,x1andx2, it must be the case that each value of domain(x1)must have a corresponding values indomain(x2)such that the corresponding assignment is consistent withca). For a general overview of different approaches to solve constraint satisfaction problems we refer to Apt (2003). Example evaluation criteria that can be used for evaluating the performance of solvers areruntime(how long does it take to find a solution),optimality(solution that achieves an optimum with regard to a predefined opti- mization function),accuracy(share of correct classifications), andcompleteness(degree to which existing solutions can be found).
3.2 Boolean satisfiability problems (SAT & ASP problems)
A Boolean Satisfiability (SAT) problem is a specific type of CSP where variables have Boolean domains and constraints are represented as Boolean formulas (Rossi et al.,2006).
As a consequence, CSPs and SAT problems are in a close relationship which allows for a reformulation of CSPs into corresponding SAT representations (Gu et al.,1996). In the line of the definition of a CSP, a SAT problem is defined by a set of variables V = {v1..vn} where each variable represents a specific domain value. For example, if V = {x1} and domain(x1) = {a, b} in the CSP representation, the corresponding SAT representation would beV = {xa, xb}withdomain(xa)=domain(xb)={true, f alse}. Constraints in SAT solving are represented in terms of clauses where each clause is represented by a dis- junction of literals (a positive literal represents the assumption that the variable value is
“true”, a negative literal represents the assumption that the variable value is “false”) (Gu et al.,1996). A solution to a SAT problem is a consistent variable assignment for the equiv- alent conjunctive normal form (CNF) which represents a conjunction of the mentioned clauses. For an overview of different approaches to solve SAT problems, we refer to Gu et al.
(1996). Importantly, answer set programming (ASP) (Brewka et al.,2011) is an expressive declarative knowledge representation that allows to model complex domains in a predicate logic fashion. On the reasoning level, answer set programs are translated into a SAT rep- resentation, i.e., ASP problems can be solved on the basis of SAT algorithms. It is worth mentioning that one of the first real-world applications of answer set programming was product configuration– for details we refer to Myll¨arniemi et al. (2014) and Simons et al.
(2002).
3.3 Machine learning
The basic idea of machine learning is to exploit data from past events (represented in terms of training data) to build a (hopefully) reliable model that is able to generalize the given data. On a high level, machine learning is used to solve two basic tasks. First,classification is used to figure out for a given test case if it falls under a given category or not. For exam- ple, in the context of a constraint-based configuration scenario, an estimation whether a user will be interested in a specific potentially new product component (or a new service) can be
regarded as a classification task, i.e., to figure out whether an item is relevant or irrelevant.
Second,predictionis used to estimate a specific value. For example, the estimation of the upper price limit of a user can be regarded as a prediction task (predicting user-individual price limits). Depending on the machine learning task to be solved (classification or pre- diction), corresponding evaluation metrics can be applied, i.e., classification metrics such asprecisionandprediction metricssuch asmean squared error(MSE) (Gunawardana &
Shani,2015; Uta et al.,2021). In addition to the basic differentiation between classification and prediction, machine learning algorithms can be further differentiated as follows. First, supervised learningis based on the idea of learning from labeled data (often denoted as
“datasets”) where datasets are divided into a training dataset (used for learning purposes) and a test dataset (used for evaluation purposes). Examples of machine learning algorithms implementing supervised learning arelinear regression,decision tree based classification, matrix factorization, andneural networks(Bishop,2006). Second, unsupervised learning focuses on the identification of patterns in unlabeled data. A basic example of an algorithmic approach supporting unsupervised learning are differentclustering algorithms. Third,rein- forcement learninglearns from its past experience by trying to maximise a reward function with respect to the next state and actions available for selection (Sutton & Barto,1998).
4 Machine learning in constraint solving
The majority of research contributions related to the integration of machine learning with constraint solving are based on supervised machine learning. In the following, we will discuss these approaches in detail and – where applied – also discuss the corresponding unsupervised learning and reinforcement learning approaches. Table1provides an overview of relevant research contributions related to the topic of machine learning and constraint solving. In the line of the basic classification of machine learning methods as discussed in Carbonell et al. (1983), we categorize the identified research contributions according to the used machine learning method/algorithm (e.g., decision trees and neural networks), the application in the constraint solving context (e.g., algorithm selection and learning of heuristics), and the used evaluation metrics (e.g., runtime and accuracy).
4.1 Constraint satisfaction
Research contributions related to the integration of machine learning and constraint satis- faction problem solving (Freuder,1997; Tsang,1993) support different goals. In the context of solution search,constraint learningsupports the learning of constraints that will help to avoid redundant search and thus improve the search performance.Solution predictionis related to the goal of directly applying machine learning to derive a solution for a given constraint satisfaction problem (CSP). Furthermore,satisfiability predictionfocuses on pre- dicting the satisfiability of a CSP without the need of activating a solver.Heuristics learning focuses on the learning of search heuristics that improve the performance of a constraint solver with regard to criteria such as runtime and prediction quality. Finally, constraint learningin the context of knowledge acquisition supports the derivation of constraints from a given dataset or on the basis of the feedback from expert users. Research contributions related to these aspects will be discussed in the following paragraphs.
The idea of constraint learning is to identify previously unknown constraints of a CSP and – through the inclusion of these constraints – speed up the search process.
Gent et al. (2010) introduce aclassification modelthat helps to identify CSP instances on
which constraint learning is likely to improve a solver’s performance. The motivation behind this work is that not every CSP instance is amenable of constraint learning which means, it cannot be guaranteed that constraint learning helps to improve performance in every case.
In order to better understand the utility of constraint learning, Gent et al. (2010) introduce a decision tree based approach that helps to evaluate a given CSP instance. Specifically, the authors uselazy learningin the context ofnogood learningwhere a nogood can be inter- preted as a combination of attribute values that do not allow the determination of a solution.
Such nogoods can be interpreted as additional constraints to be taken into account by the solver. Examples of features used in the decision tree based approach of Gent et al. (2010) arenumber of problem constraints,number of variables,number of attributes,number of constraints that refer to a specific variable, and theproportion of tuples not allowed by a specific constraint(constraint tightness).
A neural network architecturethat helps to solve constraint satisfaction problems is proposed in Wang and Tsang (1991). CSP variables and potential values are encoded as nodes of the neural network. Based on the results of their performance evaluation, the authors point out that CSP solution search based on neural network approaches is feasible also for CSPs with up to 170 variables. As a working example, the authors present the domain ofcar scheduling. A neural network can learn relevant variable assign- ments on the basis of a training data set which is represented by a collection of already solved CSPs. In order to escape local minima (e.g., in terms of the number of still vio- lated constraints), the authors propose a so-called heuristic repair method. More in-depth evaluations are needed to have a clearer view in which contexts neural network based approaches can outperform standard constraint solving. A multi neural network archi- tecture for solving binary CSPs is proposed by Adorf and Johnston (1990) where the problem of escaping local minima is solved by the integration of so-called auxiliary guard networks.
A deep learning based approachto predict the satisfiability of constraint satisfaction problems is presented by Xu et al. (2018). Their approach exploits basic properties of CSPs (e.g., individual variable values) as input for the neural network which has to decide whether the given CSP is satisfiable or not. Besides satisfiability, further labels of data points are k-consistency, best algorithm to solve, and required time to solve the problem. The input for the neural network is the CSP represented in matrix formwhere each entry indicates whether the corresponding value combination is allowed or not. The rationale behind this representation is its similarity to a 2-D representation of a grayscale image which is also used as a basis for pattern recognition in image analysis. The major motivation behind the work of Xu et al. (2018) is to accurately predict the satisfiability of CSPs to achieve the goal of minimizing the need of backtracking in solution search. In the context of the presented evaluation settings (synthesized dataset that includes data points labeled in terms of satisfiability or unsatisfiability), the authors report to have achieved an accuracy rate of higher than 99.99%. A similar line of research is followed by Wen et al. (2020) who apply deep learning for improving the performance of constraint solving in the context of symbolic execution. Symbolic execution is used, for example, in software testingand test case generationwith the goal to identify in an efficient fashion executable paths in a program.
The objective ofcontinuous searchis to permanently improve the search behavior of a solver (Arbelaez et al.,2010). Related approaches includetwo basic modes: first, the so- called functioning mode which is used to solve concrete user problems (ako productive environment) whereas theexploration modefocuses on continuously improving the search
performance by reusing the information from already completed solver sessions fortraining the heuristics model. The underlying machine learning approach is based on a binary classi- fication model. As mentioned by Arbelaez et al. (2010), the proposed approach follows the idea of lifelong learning which means to successively become an expert in the application domain. Based on a set ofstatic features(e.g.,#variables, variable domain sizes, and con- straint degrees) anddynamic features(e.g.,used heuristics for variable selection, variable domain size during search, and involvement degree of variables in failed constraints), the underlying machine learning task is defined as a binary classification problem. A related neural network approach to the training of heuristics is introduced in Galassi et al. (2018).
Permanently improving the search performance is also the idea ofreinforcement learning which follows the idea of continuously exploring new problem solving approaches with the overall goal of continuously improving problem solving behavior and corresponding outputs (Bello et al.,2017; Lagoudakis & Littman,2001; Lederman et al.,2020). Xu et al. (Xu et al.,2009) and Spieker and Gotlieb (2018) introduce reinforcement learning approaches for efficiently solving constraint satisfaction problems. Adaptive behavior is achieved, for example by the continuous adaptation of individual variable ordering heuristics with the goal to improve the performance of the problem solving behavior. Reinforcement learning does not primarily rely on datasets used for evaluation purposes but focuses more on an instance by instance trainingwith adaptations based on the feedback ofreward functions.
A simple example of such a reward function is thevalidity of a solutionor theshare of violated constraintsin a proposed solution. Consequently,two major rolescan be observed in reinforcement learning based approaches: first,actorsare responsible for theselection of the next actions(e.g., adaptation of the domain ordering heuristic for a specific variable), second,critiquing agentsare responsible forestimating the reward after an action has been performed(Nareyek,2004; Spieker & Gotlieb,2018).
Combinatorial optimization is related to the task offinding an optimal solutionfor a given (constrained) problem (Cappart et al.,2020; Korte & Vygen,2000). Example algo- rithmic approaches to solve such problems range from solving integer linear programs to basic formulations as constraint satisfaction problems. Guerri and Milano (2004) apply deci- sion tree learning in the context of theBid Evaluation Problem(BEP) which is regarded as a combinatorial optimization problem. They propose the application of a decision tree to decide about the solving approach to be used for a problem instance. There are two basic problem solving approaches taken into account in the reported evaluation scenario which areconstraint programming(CP) andinteger programming(IP). Each entry in the machine learning training set consists of 25 features and – as an annotation – the best algorithm to solve the problem.
In the context of combinatorial optimization, Bonfietti et al. (2015) propose an approach to the integration of Decision Trees (DTs) and Random Forests (RFs) in a constraint programming model. The major motivation for this integration is that in many domains con- straints and heuristics part of the model are not completely clear. The overall idea is to learn additional constraints that are then integrated into the constraint model. This approach can be regarded as a kind of empirical model learning where data from past solutions can be used to infer the mentioned classification models. In the work of Bonfietti et al. (2015), different embedding techniques are used which basically represent different ways of inte- grating the knowledge from decision trees and random forests into the constraint-based representation. An example thereof is arule-based embeddingwhere rules become part of the constraint-based representation.
Runtime prediction and optimization is important in different contexts. For example, predicting the performance of a solver given a set of solver settings and a corresponding problem instance can help to avoid suboptimal solver parametrizations but also support the optimization of algorithm parametrization, i.e., parameter tuning. Furthermore, runtime predictions in the context of solution space learning can help to efficiently estimate the applicability of a specific (constraint-based) knowledge base in interactive settings. With the goal to support the prediction of solver runtimes, Hutter et al. (2006) propose a regression- based prediction model where empirical hardness aspects derived from a problem instance are used as basic indicators for runtime estimation.
Gent et al. (2010) introduce a classification based approach to select the optimal implementation of thealldifferent constraint with the overall goal of solver performance optimization. The semantics of thealldifferentconstraint associated with a set of variables V = {x1, x2,.., xn} can be defined as ∀(xi,xj) : val(xi) = val(xj) with i = j. The features used by the classification approach areconstraint and variable statisticsdirectly derived from each given CSP instance. The training data was automatically labeled on the basis of the performance of a specificalldifferentimplementation given a set of problem- specific constraint and variable statistics (features). Examples of such features areaverage constraint arity,proportion of constraint pairs that share more than one variable, andcon- straint tightness in terms of the portion of disallowed tuples. The approach showed to clearly outperform CSP implementations with one specificalldifferentvariant.
In contrast to the previously discussed work, Erdeniz and Felfernig (2018) show how to make the learning of CSP heuristics more applicable to interactive settings. In their approach, a dataset of already collected user requirements is forwarded to ak-means clus- tering process (Burkardt,2009) which is used to group similar sets of user requirements.
Thereafter, in the line of the work of Da Col and Teppan (2017), a genetic algorithm is used to determine cluster-specific heuristics that cause the best runtime behavior for the problem instances (user requirements) in the individual cluster. After having determined the cluster- specific heuristics, the resulting models (clusters) can be applied in interactive settings in the context of new user sessions: if a user specifies a set of preferences, the system can determine the most similar cluster and use the cluster-specific heuristics for determining a solution. Based on example map coloring problems (Fritsch et al.,1998; Leighton,1979), Erdeniz and Felfernig (2018) show that their cluster-based approach outperforms settings with globally defined variable and value ordering heuristics. For visualization purposes, a simple example of learning of variable value orderings in the context of constraint-based recommendation scenarios is depicted in Tables6,7,8and9.
An application of case-based reasoning (CBR) for theselection of well-performing con- straint solvers is introduced by O’Mahony et al. (2013). CBR is based on the idea of exploiting existing problem solving knowledge from previous cases for solving new prob- lems. In this context, old cases are retrieved that are similar to the new problem and the problem solving practices of the old cases are reused to solve the new one. In the approach proposed by O’Mahony et al. (2013), solvers are recommended which have a high prob- ability of efficiently solving the given problem. The similarity between old cases and the new ones is determined on the basis of similarity metrics defined on features such asmax- imum constraint arity,maximum variable domain size, number of variables,number of constraints, andratio of different types of constraints(e.g., extensional and intensional).
Table2depicts a simplesolver recommendation scenariowhere individualsessionsreflect stored data about already completed constraint solving sessions and the task is to find an appropriate solver for the current session (scurrent).
Table 2 A sketch of a case-based reasoning scenario focusing on the identification of appropriate solver parameter settings (pirepresent problem features such as #constraints, average variable domain sizes, and used variable ordering heuristics),solverdenotes the selected solver,runtimedenotes the time (inmsec) needed by the solver to determine a solution
session p1 p2 p3 p4 p5 solver runtime
s1 1 2 1 3 2 solver5 191.2
s2 3 10 1 5 1 solver1 1012.7
s3 3 12 1 7 1 solver2 78.1
s4 1 2 1 3 2 solver5 33.82
scurrent 3 10 1 5 1 ? ?
The nearest neighbor in terms of the problem featurespi of thecurrent session is ses- sions2, the second nearest neighbor is session s3. Since the solver chosen for sessions3 showed a significantly better performance, solversolver2 (used in sessions3) should be recommended/used for/in the current session.
Constraint learning Raedt et al. (2018) in the context of knowledge acquisition scenarios (i.e., not during solution search) can be interpreted as inductive learning task of identifying a constraint set that supports a training/test dataset. This problem is different from learning processes within the scope of a solver session where the learned constraints are primarily used to improve solver performance (Gent et al.,2010). Learning constraints from data is a relevant approach to tackle issues in the context of theknowledge acquisition bottleneck which can be a major obstacle when deploying constraint-based systems in industrial con- texts. Another example task of constraint learning is thelearning of preferenceswhich is used, for example, when learning search heuristics for improving the prediction quality of a solver (Erdeniz et al.,2019). Constraints can also be learned in an interactive fashion within the scope of so-calledactive learningprocesses where the machine learning component is not limited to the analysis of a given dataset but also interacts with expert users who give feedback on the relevance of learned constraints. In this line of research, also human com- putation based approaches to constraint learning (Ulz et al.,2016) are used to exploit the feedback of expert users.
4.2 SAT solving
In the SAT solving literature, there are two basic method categories how to integrate machine learning for the purpose of problem solving (Zhang et al.,2020). First, machine learning can be used directly to solve individual SAT instances, i.e., given a SAT problem as input, the machine learning algorithm learns to solve the problem instance itself – see, for example, Galassi et al. (2018). Second, so-called heuristic methods use machine learn- ing for the purpose of predicting relevant heuristics which are then used by solvers to find a solution for a given instance (Zhang et al.,2020). Beyond solution search, there are fur- ther applications of machine learning techniques in SAT solving which include the aspects ofpredicting satisfiability andselecting the best algorithm for solving a given problem instance. Related scientific contributions will be discussed in the following paragraphs.
The task of algorithm selection is to select the best algorithm to solve a specific problem instance. An in-depth survey of algorithm selection techniques is given in Kotthoff (2014).
One of the most prominent environments supporting algorithm selection is SATZILLA
(Xu et al., 2008). It can be regarded as one of the first systems propagating the idea of algorithm portfoliosas a reasonable alternative to the optimization of individual algorithms.
The application context of SATZILLAis SAT solving scenarios whereruntime per prob- lem instanceis used as evaluation criteria (see also Haim & Walsh,2009where machine learning is applied to determine solver restart strategies). Algorithm portfolios can bestatic (a portfolio is comprised of a fixed set of algorithms which comes along with limitations regarding flexibility) or dynamic(algorithms in a portfolio can be modified, e.g., on the basis of parametrization). A dynamic approach proposed in the context of constraint solving is the ADAPTIVECONSTRAINTENGINE(ACE) (Epstein & Freuder,2001) which supports the adaptive selection of variable orderings based on a multi-tier adaptive voting architec- ture with the goal to optimize solution search. Related approaches to heuristic selection in SAT solving are proposed in Ans´otegui et al. (2009) and Samulowitz and Memisevic (2007).
Finally, Da Col and Teppan (Da Col & Teppan,2017) show how to apply genetic algorithms to automatically figure out optimal search heuristics (in terms of variable orderings, value orderings, and pruning strategies) for a set of predefined benchmark problems.
An approach to the application of neural networks forclassifying SAT problems in terms of satisfiability is introduced by Selsam et al. (2019). In this context, a SAT problem is encoded as a undirected graph where each node represents one literal of the SAT problem.
Furthermore, each clause is represented as a node of the neural work which is connected to those literals that are part of the clause. In this scenario, the neural network can be applied for two purposes: first, it can be applied for predicting the satisfiability of a given SAT problem. Second, it can be applied fordetermining a solution. If one exists, it can be directly derived from the individual activation level of the nodes in the neural network. The proposed approach is also flexible in terms of solving SAT problems in different domains such as graph coloring(e.g., in map coloring, no two adjacent map areas have the same color) and clique detection(e.g., in social network analysis, the goal could be to identify subgraphs of people where everybody knows each other). Another classification approach for estimating the satisfiability of a given SAT problem is discussed in B¨unz and Lamm (2017). Regarding the used structural properties of SAT formulae, the authors point out that neural networks fail to produce a different output given certain highly particular symmetries between two different inputs (SAT problems).
In many application contexts, there is no single solver that provides the optimal per- formance for a range of different problem settings (Loreggia et al., 2016). The goal of algorithm selectionis to identify the best problem solving approach (algorithm) for a given problem at hand. Such an algorithm selection is often based on a manually defined set of features which are then used by different machine learning techniques to predict the optimal solver for a given problem. The deep learning approach presented in Loreggia et al. (2016) supports the automated determination of an informative set of machine learning features thus making the algorithm selection a completely automated process. Interestingly, the deep learning approach is based on the idea of transforming input text files representing problem instances into corresponding grayscale square images that serve as an input for the training of the neural network. The presented approach outperforms single solvers. A basic example of a related neural network architecture supporting the selection of solvers on the basis of a given set of problem properties (features) is depicted in Fig.2.
The termphase transitiondenotes the transition from a region of search spaces where the large majority of problems have many solutions (the problems are under-constrained) to a region where the large majority of problems has no solution (the problems become
increasingly over-constrained). Xu et al. (2012) introduce a decision tree based classifica- tion approach that supports the classification of SAT problems with regard to satisfiability.
The classification approach is based on a set of features derived from a given SAT prob- lem. An example of such a feature is the averageimbalance of the non-negated and negated occurrences of SAT variables. Following a similar idea, Yolcu and P´o (2019), Kurin et al.
(2019), and Liang et al. (2016) introduce reinforcement learning approaches with the goal of improving the performance of SAT solving through the repeated adaptation of search heuristics.
With the focus ofpredicting the satisfiability of individual 3-SAT problems(maximum three literals per clause), Cameron et al. (2020) introduce a deep neural network based approach based on the concept of end-to-end learning (all model parameters are learned at the same time). In this context, the authors introduce two different network architectures, one focusing on the prediction of satisfiability, the other one supporting the prediction of satisfiable variable assignments. These variants were evaluated on SAT instances ranging from 100 to 600 variables and showed an increased prediction accuracy compared to related work (Xu et al.,2012). For visualization purposes, we provide a simple example of a neural network architecture (see Fig.1).
4.3 Answer set programming (ASP)
An approach tosolver selection in the context of ASP scenariosis introduced by Gebser et al.
(2011). More precisely, the authors show how to select a specific solver configuration given a specific answer set program (ASP). The problem features used by the machine learning approach (support vector regression) are, for example,number of constraints,number of variables,average backjump length, andlength of learned clauses. These features are used to map a given problem to the most promising solver configuration.
Yang et al. (2020) present a framework (NeurASP) for the integration of neural net- works into answer set programs. The underlying idea is to exploit the output of a neural network for the purpose of ranking atomic facts in answer set programs. As mentioned by the authors, this integration is a good example of approaches combining symbolic AI (ASP) and sub-symbolic AI (neural networks). It helps to exploit synergy effects by combining the strengths of both worlds. For example, in the context of solving Sudoku, neural networks can be used to recognize the digits on the game board whereas ASP logic can be used to rep- resent the constraints of the game and to find solutions. The approach of Yang et al. (2020) easily generalizes to differentSudokuinstances.
Fig. 1 A simple neural network architecture with one hidden layer that can classify a SAT problem as sat- isfiable or not. Examples of propertiespropi(features) are problem size, variables per clause, and balance between negated an non-negated occurrences of SAT variables
As mentioned, there is rarely one specific problem solving approach/algorithm that per- forms best in all of a given set of problem instances. More often, the best-performing algorithm differs depending on the problem setting. In the line of the research of Gebser et al. (2011) and Maratea et al. (2012) present a classification-based (a.o., nearest neighbor based classification) machine learning approach toper problem instance solver selectionin a multi ASP solver environment. Examples of features used by the classification approach are the following ones: number of rules,number of atoms,share of unary/binary/ternary rules, andnumber of true facts. For visualization purposes, we provide a simple example of a related neural network architecture (see Fig.2).
4.4 Configuration
Highly-variant products, services, and systems allow for a potentially huge number of possible configurations (Felfernig et al.,2014; Felfernig et al.,2021). The corresponding configuration space could also allow faulty configurations simply due to the fact that some relevant constraints have not been integrated into the configuration model. Examples of such faulty configurations are non-compilable software components and non-producible products. Configuration problems are typically represented on the basis of the already dis- cussed approaches of constraint satisfaction, SAT solving, and answer set programming (ASP). In the context of feature model development, Temple et al. (2016) introduce a classification-based (decision trees) learning approach that supports the derivation of addi- tional constraints that help to reduce the space of possible configurations. In their approach, an oracle is used to estimate the correctness of generated configurations. These annotated configurations are then used to build a decision tree that classifies correct and faulty con- figurations depending on the configuration-individual attribute values. This decision tree is then used for the derivation of the mentioned additional constraints. Intuitively, new relevant constraints can be extracted by negating those decisions that led to faulty configurations (Temple et al.,2016). In a related work, Temple et al. (2017) propose a machine learning approach that helps to relate context information (represented in terms of feature selections) with corresponding parts of a software and thus replaces manual modeling efforts. Based
Fig. 2 A simple neural network architecture with one hidden layer that can be used for learning solver selection (solveri) based on basic properties (propj) of a given ASP. Examples of such properties (features) are: number of rules in the ASP, number of atoms, share of unary/binary/ternary rules, and number of true facts
on the criteria of runtime performance of generated configurations, these problem instances (configurations) can be evaluated as acceptable or unacceptable. On the basis of this anno- tated dataset, decision trees can be learned which again serve as a basis for the derivation of additional constraints that reduce the search space.
In the context ofconfiguring highly-variant productswith around 40 different product properties, Uta and Felfernig (2020) introduce a machine learning approach that helps to predict variable values a user would regard as relevant. The learning approach is based on a three-layer neural network with one input, output, and hidden layer. In the presented approach, variable value predictions do not take into account domain constraints which can result in situations where the recommendations can become inconsistent with the underlying product model. Related approaches based on recommendation techniques are summarized a.o. in Falkner et al. (2011) where techniques such as collaborative filtering (Konstan et al., 1997) are applied to predict variable values of relevance for a user. A basic method to avoid inconsistencies between recommended variable values and the underlying product domain knowledge is to avoid direct variable value predictions but forward value selection probabilities to a configurator (e.g., a constraint solver) which can integrate this information, for example, in terms of variable value ordering heuristics.
Writing a paper, a project proposal, or a curriculum vitae that meets different criteria such obligatory contents, maximum and minimum sizes of paragraphs, upper and lower bounds in terms of pages and words used in a document is often a challenging task. Reasons for making such tasks challenging are time pressure when submitting the document or simply cognitive overheads that are triggered by complex rules regarding the correct format of a submitted document. In order to tackle these challenges, Acher et al. (2018) introduce VARYLATEX which enriches LATEX document editing tasks with the concepts of variability modeling (in terms of feature models) and binary classification which classifies a configured document as acceptable or unacceptable. Features used by classification are allvariability propertiesdefined for the document under consideration.
For an overview of the application of machine learning and recommender systems tech- niques in configuration scenarios (especially in feature-modeling related scenarios), we refer to Felfernig et al. (2021). The example depicted in Table3sketches a basic approach forvalue prediction in configuration scenarios. For example, a user interacting with a con- figurator has already specified some parameters and the task is to recommend values for parameters that have not been specified up to now. Following the principles of collabora- tive filtering (Konstan et al.,1997), thenearest neighborof the current user (ucurrent) is the
Table 3 A simple example of applying collaborative filtering (Konstan et al.,1997) in configuration contexts user x1(a, b) x2(u, v) x3(1,2,3) x4(1,2) x5(3,4,5)
u1 a u 2 1 3
u2 b v 3 2 5
u3 b v 1 2 4
ucurrent a u 2 2 ?→3
The nearest neighbor (in terms of an overlap in the individual preferences) of the current userucurrent is the useru1. In this scenario, we are interested in a prediction of the value ofx5to be recommended to userucurrent. Following the preferences of the nearest neighboru1, the recommended value forucurrentfor variablex5would be 3
useru1. His/her preference with regard to the variablex5is 3. Consequently, 3 is taken as recommendation for variablex5for userucurrent.
4.5 Diagnosis
If no solution can be identified for a given constraint setC = {c1, c2,.., cn}, one or more conflicts exist inC. In this context, a minimal conflict (set)CS⊆Crepresents a minimal set of constraints which is inconsistent (Junker,2004), i.e., inconsistent(CS). IfCSis minimal, only one constraint has to be deleted fromCS to restore consistency. IfC entails more than one conflict, each individual conflict has to be resolved to restore consistency inC.
A minimal set of constraints that needs to be deleted fromCto restore consistency is also denoted asminimal hitting set(for the given conflict sets) or minimal diagnosis (Felfernig et al.,2012; Reiter,1987). Adiagnosis(often denoted asΔ) is minimal if¬∃Δ :Δ ⊂Δ andΔ is still a diagnosis.
In most scenarios, there exists more than one diagnosis for a given inconsistent constraint setC(e.g., a CSP). In such cases, we need to find the most relevant diagnosis, for example, a diagnosis that will be with a high probability of relevance for a user. Erdeniz et al. (2018) introduce an approach based onclustering and genetic algorithmsthat help to optimize the prediction quality for diagnoses which is especially relevant in interactive settings. Assum- ing the existence of a dataset comprised of sets of inconsistent user requirements, clustering can be applied to form individual groups of similar (inconsistent) user requirements. On each of the individual clusters, a genetic algorithm can determine a cluster-specific impor- tance ranking of constraints (user requirements) that helps to maximize the diagnosis prediction quality.
For diagnosis purposes, Erdeniz et al. (2018) apply a specific variant of FASTDIAG
(Felfernig et al., 2012) which is a divide-and-conquer based direct diagnosis algorithm.
FASTDIAGis based on a constraint ordering that has an impact on the determined diagnosis.
For example, if we assumeC= [c1, c2, c3](ordered set notation) and there exists a conflict CS = {c1, c3}, the resulting diagnosis would beΔ= {c1}. If we assumeC = [c3, c2, c1], the corresponding diagnosis determined by FASTDIAGwould beΔ= {c3}. In the approach of Erdeniz et al. (2018), optimal cluster-specific constraint orderings are determined by a genetic algorithm. The optimization criteria in this context is prediction quality, for exam- ple, in terms ofprecisionwhich is |correctlypredicteddiagnoses|
|predictions| . In a new user session where user requirements become inconsistent, the cluster most similar to the current set of user requirements has to be identified. Using the cluster-specific constraint ordering, FASTDIAG
can be activated with the current set of inconsistent user requirements. More details about the used diagnostic approach can be found in Felfernig et al. (2012) and Felfernig et al.
(2018). A further development of the approach of Erdeniz et al. (2018) is introduced in (Erdeniz et al.,2019) where the task of adaptive constraint ordering (in diagnosis contexts) is represented as a matrix factorization task (Koren et al.,2009).
A simple example of the impact of different constraint orderings on the outcome of a diagnosis task is depicted in Tables4and5. In this example, we assume a knowledge base consisting of the constraintsC= {c1:x1=x2, c2:x2=x3}and the additional constraints {c3, c4, c5}representing, for example, user requirements. In Table4, the constraint with the lowest importance is assumed to bec3whereas in Table5, the constraint with the lowest importance is assumed to bec5. The different rankings result in different corresponding diagnosis (Δ1, Δ2). In Tables4and5,×indicates that the corresponding constraint has been selected for conflict resolution, i.e., is part of a diagnosisΔi.
