Solving 0-1 integer linear programming problems with core-guided optimization
Jo Devriendt ¹²³ Stephan Gocht ¹² Emir Demirović ⁴ Jakob Nordström ²¹ Peter J Stuckey ⁵
¹ Lund University, Sweden
² University of Copenhagen, Denmark
³ KU Leuven, Belgium
⁴ TU Delft, The Netherlands
⁵ Monash University, Australia
Currently post-doc with Joost Vennekens
jodevriendt.com
slides.com/jod/cgo-eavise
Based on the AAAI21 paper
"Cutting to the Core of Pseudo-Boolean Optimization: Combining Core-Guided Search with Cutting Planes Reasoning"
- All variables are 0-1 (Boolean)
- Constraints C are linear inequalities
- Formula φ is a conjunction of constraints
- Negation:
- A literal is a variable or its negation
-
Linear objective function O to be minimized
- WLOG, objective has only positive coefficients
Prelims:
0-1 Integer Linear Programming (with negations)
x, y
\overline{x}=1-x
l
\sum_{i} a_i x_i \geq A
\sum_{i} c_i x_i
Prelims:
0-1 Integer Linear Programming (with negations)
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
\text{Minimize}~~~y+2v+w
x=1,y=1,z=1,v=1,w=1
A solution:
x=0,y=0,z=0,v=1,w=1
Optimal solution:
Solutions
Linear Optimization
Objective
(to minimize)
Solutions
Linear Optimization
Objective
(to minimize)
Solutions
Linear Optimization
Objective
(to minimize)
Solutions
Linear Optimization
Objective
(to minimize)
Solutions
Linear Optimization
Objective
(to minimize)
Solutions
Linear Optimization
Objective
(to minimize)
Solutions
Linear Optimization
Objective
(to minimize)
Solutions
Core-guided Optimization (CGO)
Objective
(to minimize)
Solutions
Objective
(to minimize)
Core-guided Optimization (CGO)
Solutions
Objective
(to minimize)
Core-guided Optimization (CGO)
Solutions
Objective
(to minimize)
Core-guided Optimization (CGO)
Solutions
Objective
(to minimize)
Core-guided Optimization (CGO)
Core-guided Optimization (CGO)
Solutions
Objective
(to minimize)
Prelims:
0-1 Integer Linear Programming (with negations)
Special types of constraints:
-
Cardinality: all coefficients are 1 or -1, e.g.,
-
Clause: cardinality representing disjunction of literals, e.g.,
x-y+z-w \geq 0 \\
\Leftrightarrow x + \overline{y} + z + \overline{w} \geq 2
x-y+z-w \geq -1 \\
\Leftrightarrow x + \overline{y} + z + \overline{w} \geq 1
Prelims
\alpha \colon \{l, \ldots\}
\varphi
Solver abstraction:
- Input
- Formula
-
Assumptions
- Output, either
- Solution satisfying under
- Core constraint implied by and violated by
\varphi
\alpha
\alpha
\varphi
This abstraction applies to conflict-driven, constraint learning solvers
I
Classic
Core-guided Optimization
Solve under assumptions
Extract core
Reformulate problem
Increase lower bound
- Original core-guided idea [FM06] (MaxSAT)
- CGO also succesfully applied in ASP [AKMS12] and CP [GBDS20]
- Described technique: OLL [AKMS12, MDM14] and ONE [ADR15]
CGO loop
Solve under assumptions
- Assume all variables in objective to be false
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
Extract core
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
- Classic case: conflict-driven clause learning solver
- Core is a clause
\mathit{Core}\colon x_2+x_3+x_4 \geq 1
Reformulate problem
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
- Determine :
smallest coefficient in objective with variable in core - Enlarge core to extension constraint Ext with one fewer auxiliary variables
-
Add Ext to with order constraints
- Sound as implies Core
-
Reformulate objective:
Multiply Ext by ,
subtract lhs and add rhs to O
\mathit{Core}\colon x_2+x_3+x_4 \geq 1
\mathbf{c^*}
c^*=2
\varphi
y_i \geq y_{i+1}
\mathit{Ext}\colon x_2+x_3+x_4 = 1+y_1+y_2
\varphi
O'\colon x_1+x_3+2x_4+2+2y_1+2y_2\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
c^*
\varphi' = \varphi \wedge \mathit{Ext} \wedge y_1 \geq y_2
Increase lower bound
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
- Lower bound increase is simply
\mathit{Core}\colon x_2+x_3+x_4 \geq 1
c^*=2
\mathit{Ext}\colon x_2+x_3+x_4 = 1+y_1+y_2
O'\colon x_1+x_3+2x_4+2+2y_1+2y_2\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
c^*
\varphi' = \varphi \wedge \mathit{Ext} \wedge y_1 \geq y_2
Rinse and repeat
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
- Repeat procedure with reformulated problem
\mathit{Core}\colon x_2+x_3+x_4 \geq 1
c^*=2
\mathit{Ext}\colon x_2+x_3+x_4 = 1+y_1+y_2
O'\colon x_1+x_3+2x_4+2+2y_1+2y_2\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
\varphi' = \varphi \wedge \mathit{Ext} \wedge y_1 \geq y_2
\varphi', O', \alpha'=\{\overline{x_1},\overline{x_3},\overline{x_4},\overline{y_1},\overline{y_2}\}
II
Cutting-planes
Core-guided Optimization
(our contribution)
Solve under assumptions
- Assume all variables in objective to be false
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
Extract cardinality core
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
- With a conflict-driven cutting-planes
solver, core is a linear inequality - Can contain non-assumptions or negative coefficients
- Eliminate by adding Boolean axioms
- Can have arbitrarily large coefficients
- Reduce to cardinality core, e.g., by division with largest coefficient
\mathit{Core}\colon x_0-x_1+3x_2+3x_3+x_4 \geq 5
1 \geq x, x \geq 0
\mathit{Core'}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card}\colon x_2+x_3+x_4 \geq 2
Reformulate problem
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
- Determine
- Determine cardinality degree d
- Enlarge core to extension constraint Ext with d fewer auxiliary variables
- Add Ext to with order constraints
- Reformulate objective
c^*
c^*=2
y_i \geq y_{i+1}
\mathit{Ext}\colon x_2+x_3+x_4 = 2+y_1
\varphi
O'\colon x_1+x_3+2x_4+4+2y_1\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
\varphi' = \varphi \wedge \mathit{Ext}
\mathit{Core}\colon x_0-x_1+3x_2+3x_3+x_4 \geq 5
\mathit{Core'}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card}\colon x_2+x_3+x_4 \geq 2
d=2
Increase lower bound
- Lower bound increase is
- Rinse and repeat
c^*\cdot d
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
c^*=2
\mathit{Ext}\colon x_2+x_3+x_4 = 2+y_1
O'\colon x_1+x_3+2x_4+4+2y_1\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
\varphi' = \varphi \wedge \mathit{Ext}
\mathit{Core}\colon x_0-x_1+3x_2+3x_3+x_4 \geq 5
\mathit{Core'}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card}\colon x_2+x_3+x_4 \geq 2
d=2
Advantage of cutting-planes vs clausal
- Stronger (cardinality) core
- Stronger lower bound per core
- No encoding of Ext to clauses
Why a cardinality core?
\varphi \\
O\colon \sum_i c_ix_i \\
\varphi \models \sum_{i} a_i x_i \geq A
Given
\forall i\colon 0 \leq a_i \leq c_i
If
Then
\varphi \models O \geq A
Any core constraint is violated by setting all objective coefficients to 0, so it can be transformed to a constraint satisfying the if-condition.
Why a cardinality core?
Extension constraint would require too many auxiliary variables (assuming unary encoding)
\sum_i a_ix_i = A + \sum_{j\colon 1}^{(\sum_{i}a_i)-A} y_j
\sum_{i\colon 1}^n x_i = A + \sum_{j\colon 1}^{n-A} y_j
compared to
III
Cutting-planes
Core-guided Optimization
Optimizations
Cardinality reduction optimization
O\colon x_1+2x_2+3x_3+4x_4\\
\mathit{Core}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card_1}\colon x_2+x_3+x_4 \geq 2\\
\mathit{Card_2}\colon x_3+x_4 \geq 1
- Multiple implied cardinalities
- Pick one that leads to best
lower bound increase
- Pick one that leads to best
2*2\\
1*3
c_1^* = 2\\
c_2^*=3
Lower bounds:
Lazy extension encoding
\mathit{Card}\colon x_1+x_2+x_3+x_4 \geq 2\\
\mathit{Ext}\colon x_1+x_2+x_3+x_4 = 2+y_1+y_2\\
y_1 \geq y_2
- Auxiliary variables represent bounds on lhs
- Extension constraint can be split along the auxiliary variables
- Setting to false implies must be false
- As long as is false, the constraint for is not needed
- Lazily introduce variable and corresponding constraint
- Similar in spirit to [MJML14]
y_1 \Leftrightarrow x_1+x_2+x_3+x_4 \geq 3\\
y_2 \Leftrightarrow x_1+x_2+x_3+x_4 \geq 4\\
y_i
y_{i+1}
y_i
y_{i+1}
y_{i+1}
Interleaving core-guided and linear optimization
Solutions
Objective
(to minimize)
Related ideas in [ADMR15]
Solutions
Objective
(to minimize)
Interleaving core-guided and linear optimization
Related ideas in [ADMR15]
Solutions
Objective
(to minimize)
Interleaving core-guided and linear optimization
Related ideas in [ADMR15]
Solutions
Objective
(to minimize)
Interleaving core-guided and linear optimization
Related ideas in [ADMR15]
Solutions
Objective
(to minimize)
Interleaving core-guided and linear optimization
Related ideas in [ADMR15]
Solutions
Objective
(to minimize)
Interleaving core-guided and linear optimization
Related ideas in [ADMR15]
Solutions
Objective
(to minimize)
Interleaving core-guided and linear optimization
Related ideas in [ADMR15]
IV
Detailed example
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
CGO lower bound derivation
\alpha=\{\}
Simple solution:
x = y = z = v = w = 1
Optimal rational solution:
x = z = 0, y = v = 1/3, w = 2/3
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
Min~~~y+2v+w
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
\alpha=\{\}
Min~~~y+2v+w
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
\alpha=\{y=0,\\v=0,w=0\}
Min~~~y+2v+w
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
\alpha=\{y=0,\\v=0,w=0\}
Min~~~y+2v+w
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
\alpha=\{y=0,\\v=0,w=0\}
Min~~~y+2v+w
y+w \geq 1
\textcolor{red}{\text{add}~x \geq 0}
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
\alpha=\{y=0,\\v=0,w=0\}
Min~~~y+2v+w
y+w \geq 1
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
\textcolor{red}{y+w = u+1}
\textcolor{red}{\text{introduce}~u}
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
\alpha=\{y=0,\\v=0,w=0\}
Min~~~\textcolor{red}{2v+u+1}
y+w \geq 1
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
\textcolor{red}{y+w = u+1}
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
\alpha=\{v=0,u=0\}
Min~~~2v+u+1
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
y+w = u+1
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
Min~~~2v+u+1
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
y+w = u+1
\alpha=\{v=0,u=0,\\w=0\}
\textcolor{red}{\text{decision}}
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
Min~~~2v+u+1
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
y+w = u+1
\alpha=\{v=0,u=0,\\w=0,y=1\}
\textcolor{red}{\text{propagates}~y=1}
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
Min~~~2v+u+1
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
y+w = u+1
\alpha=\{v=0,u=0,\\w=0,y=1,z=0\}
\textcolor{red}{\text{propagates}~z=0}
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
Min~~~2v+u+1
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
y+w = u+1
\alpha=\{v=0,u=0,\\w=0,y=1,z=0\}
\textcolor{red}{\text{conflict!}}
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
Min~~~2v+u+1
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
y+w = u+1
\alpha=\{v=0,u=0,\\w=0,y=1,z=0\}
3v \geq 2u+1
\textcolor{red}{\text{division}}
3v \geq 1
v \geq1
\textcolor{red}{+2\times u\geq0}
v= 1
\textcolor{red}{+2\times}
\textcolor{red}{+2\times}
\textcolor{red}{+}
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
y+w = u+1
\alpha=\{v=0,u=0,\\w=0,y=1,z=0\}
v= 1
Min~~~\textcolor{red}{u+3}
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
y+w = u+1
\alpha=\{u=0\}
v= 1
Min~~~u+3
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
y+w = u+1
\alpha=\{u=0\}
v= 1
Min~~~u+3
x=0,y=0,z=0,\\
v=1,w=1,u=0
CGO lower bound derivation
+x+y-z \geq 0 \\
-2y+2z-v \geq -1 \\
-z+2v-w \geq 0 \\
-x+y+w \geq 1
Assume best case objective
Call yields SAT?
Extract core
Reformulate objective,
increase lower bound
Optimal
no
yes
y+w = u+1
\alpha=\{u=0\}
v= 1
Min~~~u+3
x=0,y=0,z=0,\\
v=1,w=1,u=0
CGO lower bound derivation
Proof complexity view
V
Experiments
Overall performance
[PB16]
Overall performance
[PB16]
Overall performance
[PB16]
Overall performance
[PB16]
Stronger cores reach optimality faster
Fewer # variables due to lazy encoding
Take-away
Our contribution
- Core-guided optimization with cutting-planes reasoning instead of clausal reasoning
- Investigated different ways to optimize process
Cutting-planes reasoning advantages
- More efficient search for cores
- Can find stronger cores
- No re-encoding to clauses needed
- Proposed optimizations are easy to implement
Conclusion
- Huge performance improvement in RoundingSat
- Still behind MIP solvers on many instances
- Future work: incorporate MIP preprocessing techniques?
References
- [FM06] On Solving the Partial MAX-SAT Problem; Fu, Malik; 2006
- [AKMS12] Unsatisfiability-Based Optimization in clasp; Andres, Kaufmann, Matheis, Schaub; 2012
- [MDM14] Core-Guided MaxSAT with Soft Cardinality Constraints; Morgado, Dodaro, Marques-Silva; 2014
- [MJML14] Martins, Joshi, Manquinho, Lynce; Incremental Cardinality Constraints for MaxSAT; 2014
- [ADMR15] Alviano, Dodaro, Marques-Silva, Ricca; Optimum stable model search: algorithms and implementation; 2015
- [ADR15] Alviano, Dodaro, Ricca; A MaxSAT Algorithm Using Cardinality Constraints of Bounded Size; 2015
- [PB16] Latest PB competition - www.cril.univ-artois.fr/PB16/
- [GBDS20] Core-Guided and Core-Boosted Search for CP; Gange, Berg, Demirović, Stuckey; 2020
Thanks for your attention!
Solving 0-1 Integer Linear Programs with Core-guided Optimisation
By Jo Devriendt
