Learn to Relax:
Integrating Integer Linear
Programming with
ConflictDriven Search
Jo Devriendt †*, Ambros Gleixner ‡, Jakob Nordström *†
Acknowledgement: Jan Elffers †*
† Lund University, Sweden
‡ Zuse Institut Berlin, Germany
* University of Copenhagen, Denmark
jodevriendt.com
Conflictdriven search for 01 ILP
an exampledriven intro
ConflictDriven Search Loop
 Input: 01 IL program

Output: either
 solution
 no solution exists

Search state:
 set of constraints φ
 partial assignment α

Initial state:
 φ = input
 α = {}
ConflictDriven Search Loop
Unit propagation
 Under α, if a constraint c ∈ φ would be falsified by assuming x=0 (resp. x=1), extend α with x=1 (resp. x=0)
 Propagate until fixpoint
currently no unit propagation
Unit propagation
Conflict?
 Conflict: some c ∈ φ falsified by α
ConflictDriven Search Loop
Unit propagation
Conflict?
Decide unassigned variable
no
 Only if unit propagation did not lead to a conflict
 If no unassigned variable left, return solution
 Resume unit propagation
ConflictDriven Search Loop
Unit propagation
Conflict?
Decide unassigned variable
no
 Only if unit propagation did not lead to a conflict
 If no unassigned variable left, return solution
 Resume unit propagation
ConflictDriven Search Loop
Unit propagation
Conflict?
Decide unassigned variable
no
ConflictDriven Search Loop
Unit propagation
Conflict?
Decide unassigned variable
no
ConflictDriven Search Loop
Unit propagation
Conflict?
Decide unassigned variable
no
ConflictDriven Search Loop
Unit propagation
Conflict?
Decide unassigned variable
no
ConflictDriven Search Loop
Unit propagation
Conflict?
Decide unassigned variable
no
ConflictDriven Search Loop
Unit propagation
Conflict?
Decide unassigned variable
no
ConflictDriven Search Loop
Unit propagation
Conflict?
Decide unassigned variable
no
 Conflict: some constraint in φ falsified by α
ConflictDriven Search Loop
Unit propagation
Conflict?
Learn constraint
Decide unassigned variable
yes
no
 From falsified constraint and reasons, construct learned constraint
 implied by φ, should prevent conflict
 Add learned constraint to φ
 → learned constraint database
ConflictDriven Search Loop
Unit propagation
Conflict?
Learn constraint
Decide unassigned variable
yes
no
 From falsified constraint and reasons, construct learned constraint
 implied by φ, should prevent conflict
 Add learned constraint to φ
 → learned constraint database
ConflictDriven Search Loop
Unit propagation
Conflict?
Learn constraint
Backjump
Decide unassigned variable
yes
no
 Backtrack based on learned constraint
 Resume unit propagation
ConflictDriven Search Loop
 Backtrack based on learned constraint
 Resume unit propagation
Unit propagation
Conflict?
Learn constraint
Backjump
Decide unassigned variable
yes
no
 Backtrack based on learned constraint
 Resume unit propagation
ConflictDriven Search Loop
Unit propagation
Conflict?
Learn constraint
Backjump
Decide unassigned variable
yes
no
ConflictDriven Search Loop
Unit propagation
Conflict?
Learn constraint
Backjump
Decide unassigned variable
yes
no
ConflictDriven Search Loop
Unit propagation
Conflict?
Learn constraint
Backjump
Decide unassigned variable
yes
no
ConflictDriven Search Loop
Unit propagation
Conflict?
Learn constraint
Backjump
Decide unassigned variable
yes
no
ConflictDriven Search Loop
Unit propagation
Conflict?
Learn constraint
Backjump
Decide unassigned variable
yes
no
ConflictDriven Search Loop
Unit propagation
Conflict?
Learn constraint
Backjump
Decide unassigned variable
yes
no
ConflictDriven Search Loop
Unit propagation
Conflict?
Learn constraint
Backjump
Decide unassigned variable
yes
no
ConflictDriven Search Loop
Unit propagation
Conflict?
Learn constraint
Backjump
Decide unassigned variable
yes
no
Unsatisfiable!
ConflictDriven Search Loop
Unit propagation
Conflict?
Learn constraint
Backjump
Decide unassigned variable
yes
no
 Learning constraints pushes search forward
 Thousands of conflicts per second
 Highly optimized unit propagation
 Originates from Boolean satisfiability (SAT) [MS96,BS97,MMZZM01]
 Generalized to pseudoBoolean (PB) solving [CK05,SS06,LP10,EN18]
 many variations possible
here be dragons
ConflictDriven Search Loop
Another look at our example...
 It's rationally infeasible!
 Could be solved without search
 In theory: rationally infeasible programs are easy for conflictdriven PB search
 In practice: PB solvers timeout on certain rationally infeasible programs [EGNV18]
 unit propagation is local
 wrong constraints are learned
How about integrating an LP solver?
our work
Linear Programming (LP) solver
 Input:
 LP relaxation of φ
 variable bounds α
 objective function
 Output: either
 optimal rational solution

Farkas multipliers
 define a positive linear combination of constraints
in φ, falsified by α
 define a positive linear combination of constraints
ConflictDriven Search Loop
with LP solver call
Unit Propagation
Conflict?
Backjump
Decide unassigned variable
no
Rational infeasibility?
Query LP solver
yes
no
Learn constraint
Extract Farkas multipliers
yes
Learn Farkas constraint
Rational infeasibility example
Farkas multipliers
conflicting
Farkas constraint
 LP solvers are slow compared to conflictdriven search loop
 limit calls to LP solver
 limit LP solver running time
 deterministic measure: balance #conflicts in conflictdriven solver to #pivots in LP solver
 LP solver uses inexact floating point arithmetic
 learned constraint must be implied by φ
 calculate Farkas constraint with exact arithmetic
 verify Farkas constraint is still conflicting
 postprocess Farkas constraint to eliminate noise
Two technical hurdles
 Every once in a while, run LP solver to completion at root
 use (optimal) rational solution as value heuristic
 generate ChvátalGomory (CG) cuts
 add to both LP solver and learned constraint set
 Generate "deep" ChvátalGomory cuts from internal search nodes
 valid at root node, so safe to add as learned constraint
 Add learned constraints as cuts to the LP solver
Further ideas
 PB solver RoundingSat [EN18]
 Strong ILP constraint learning
 Performed well in past PB competitions
 Linear optimization built on top of conflictdriven search loop
 LP solver SoPlex [ZIB]
 SCIP's native LP solver
 Stateoftheart open source
Working implementation
 #pivots/#conflicts ≤ 1
 128 bit precision to calculate CG cuts and Farkas constraints
 Cut parallelism check
 for decision instances, minimize sum of variables in SoPlex
 for pure CNFs, deactivate LP techniques
Design choices
Compare stateoftheart
 RoundingSat
 Sat4J
 NaPS
 SCIP
 Gurobi
 CPLEX
to implementations
 RS+SPX
 RS+SPX+GC
 RS+SPX+GC+LC
Experiments!
[PBCOMP]
[PBCOMP]
[PBCOMP]
[PBCOMP]
[P04]
[P04]
[MIPLIB]
[MIPLIB]
[MIPLIB]
[MIPLIB]
Related work
 Combined conflict analysis and LP solving in other solving paradigms:
 SAT: LPSAT [WW00]
 CP: LCGGlucose [D16]
 SMT: deciding linear arithmetic [Dd06]
 MIP: SATstyle learning, e.g. SCIP [ABKW08]
 Key differences of our approach:
 run LP solver incrementally
 not until completion

native conflict analysis on all linear constraints
 exponentially stronger than clausal conflict analysis
 run LP solver incrementally
Summary
Conflictdriven search for 01 ILPs
 Generates cuts from search conflicts
 Does not always find short refutations for rational infeasibility
 Allows sound & efficient integration with LP solver
 Is further improved by value heuristic & Gomory cut generation
Thanks for your attention!
Experiments approach best of both worlds
[MS96] GRASP  a new search algorithm for satisfiability  MarquesSilva, Sakallah
[BS97] Using CSP lookback techniques to solve realworld SAT instances  Bayardo, Schrag
[WW00] Combining Linear Programming and Satisability Solving for Resource Planning  Wolfman, Weld
[MMZZM01] Chaff: Engineering an efficient SAT solver  Moskewicz, Madigan, Zhao, Zhang, Malik
[P04] Where are the hard knapsack problems?  Pisinger
[CK05] A fast pseudoBoolean constraint solver  Chai, Kuehlmann
[SS06] Pueblo: A hybrid pseudoBoolean SAT solver  Sheini, Sakallah
[Dd06] A Fast LinearArithmetic Solver for DPLL(T)  Dutertre, de Moura
[ABKW08] Constraint Integer Programming: A New Approach to Integrate CP and MIP  Achterberg, Berthold, Koch, Wolter
[LP10] The Sat4j library, release 2.2  Le Berre, Parrain
[D16] Scheduling and rostering with learning constraint solvers  Downing
[EN18] Divide and conquer: Towards faster pseudoboolean solving  Elffers, Nordström
[EGNV18] Using Combinatorial Benchmarks to Probe the Reasoning Power of pseudoBoolean Solvers  Elffers, GiráldezCru, Nordström, Vinyals
[ZIB] SoPlex  soplex.zib.de
[PBCOMP] Latest PB competition  www.cril.univartois.fr/PB16/
[MIPLIB] The Mixed Integer Programming Library  miplib.zib.de
References
Learn to Relax: Integrating Integer Linear Programming with ConflictDriven Search  CPAIOR
By jod
Learn to Relax: Integrating Integer Linear Programming with ConflictDriven Search  CPAIOR
Learn to Relax: Integrating Linear Programming with ConflictDriven Search
 311