Solving 0-1 integer linear programming problems with core-guided optimization

Jo Devriendt ¹²³ Stephan Gocht ¹² Emir Demirović ⁴ Jakob Nordström ²¹ Peter J Stuckey ⁵
¹ Lund University, Sweden

² University of Copenhagen, Denmark

³ KU Leuven, Belgium

⁴ TU Delft, The Netherlands

⁵ Monash University, Australia

Currently post-doc with Joost Vennekens


jodevriendt.com
slides.com/jod/cgo-eavise

Based on the AAAI21 paper
"Cutting to the Core of Pseudo-Boolean Optimization: Combining Core-Guided Search with Cutting Planes Reasoning"

  • All variables          are 0-1 (Boolean)
  • Constraints C are linear inequalities
  • Formula φ is a conjunction of constraints
  • Negation:
    • A literal    is a variable or its negation
  • Linear objective function O to be minimized
    • WLOG, objective has only positive coefficients

Prelims:

0-1 Integer Linear Programming (with negations)

x, y
\overline{x}=1-x
l
\sum_{i} a_i x_i \geq A
\sum_{i} c_i x_i

Prelims:

0-1 Integer Linear Programming (with negations)

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
\text{Minimize}~~~y+2v+w
x=1,y=1,z=1,v=1,w=1

A solution:

x=0,y=0,z=0,v=1,w=1

Optimal solution:

Solutions

Linear Optimization

Objective
(to minimize)

Solutions

Linear Optimization

Objective
(to minimize)

Solutions

Linear Optimization

Objective
(to minimize)

Solutions

Linear Optimization

Objective
(to minimize)

Solutions

Linear Optimization

Objective
(to minimize)

Solutions

Linear Optimization

Objective
(to minimize)

Solutions

Linear Optimization

Objective
(to minimize)

Solutions

Core-guided Optimization (CGO)

Objective
(to minimize)

Solutions

Objective
(to minimize)

Core-guided Optimization (CGO)

Solutions

Objective
(to minimize)

Core-guided Optimization (CGO)

Solutions

Objective
(to minimize)

Core-guided Optimization (CGO)

Solutions

Objective
(to minimize)

Core-guided Optimization (CGO)

Core-guided Optimization (CGO)

Solutions

Objective
(to minimize)

Prelims:

0-1 Integer Linear Programming (with negations)

Special types of constraints:

  • Cardinality: all coefficients are 1 or -1, e.g.,


     
  • Clause: cardinality representing disjunction of literals, e.g.,

     
x-y+z-w \geq 0 \\ \Leftrightarrow x + \overline{y} + z + \overline{w} \geq 2
x-y+z-w \geq -1 \\ \Leftrightarrow x + \overline{y} + z + \overline{w} \geq 1

Prelims

\alpha \colon \{l, \ldots\}
\varphi

Solver abstraction:

  • Input
    • Formula
    • Assumptions
       
  • Output, either
    • Solution satisfying      under
    • Core constraint implied by     and violated by
\varphi
\alpha
\alpha
\varphi

This abstraction applies to conflict-driven, constraint learning solvers

I

Classic

Core-guided Optimization

Solve under assumptions

Extract core

Reformulate problem

Increase lower bound

  • Original core-guided idea [FM06] (MaxSAT)
  • CGO also succesfully applied in ASP [AKMS12] and CP [GBDS20]
  • Described technique: OLL [AKMS12, MDM14] and ONE [ADR15]

CGO loop

Solve under assumptions

  • Assume all variables in objective to be false
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\

Extract core

\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
  • Classic case: conflict-driven clause learning solver
  • Core is a clause
\mathit{Core}\colon x_2+x_3+x_4 \geq 1

Reformulate problem

\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
  • Determine     :
    smallest coefficient in objective with variable in core
  • Enlarge core to extension constraint Ext with one fewer auxiliary variables
  • Add Ext to     with order constraints
    • Sound as     implies Core
  • Reformulate objective:
    Multiply Ext by    ,
    subtract lhs and add rhs to O
\mathit{Core}\colon x_2+x_3+x_4 \geq 1
\mathbf{c^*}
c^*=2
\varphi
y_i \geq y_{i+1}
\mathit{Ext}\colon x_2+x_3+x_4 = 1+y_1+y_2
\varphi
O'\colon x_1+x_3+2x_4+2+2y_1+2y_2\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
c^*
\varphi' = \varphi \wedge \mathit{Ext} \wedge y_1 \geq y_2

Increase lower bound

\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
  • Lower bound increase is simply
\mathit{Core}\colon x_2+x_3+x_4 \geq 1
c^*=2
\mathit{Ext}\colon x_2+x_3+x_4 = 1+y_1+y_2
O'\colon x_1+x_3+2x_4+2+2y_1+2y_2\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
c^*
\varphi' = \varphi \wedge \mathit{Ext} \wedge y_1 \geq y_2

Rinse and repeat

\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
  • Repeat procedure with reformulated problem
\mathit{Core}\colon x_2+x_3+x_4 \geq 1
c^*=2
\mathit{Ext}\colon x_2+x_3+x_4 = 1+y_1+y_2
O'\colon x_1+x_3+2x_4+2+2y_1+2y_2\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
\varphi' = \varphi \wedge \mathit{Ext} \wedge y_1 \geq y_2
\varphi', O', \alpha'=\{\overline{x_1},\overline{x_3},\overline{x_4},\overline{y_1},\overline{y_2}\}

II

Cutting-planes

Core-guided Optimization

(our contribution)

Solve under assumptions

  • Assume all variables in objective to be false
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\

Extract cardinality core

\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
  • With a conflict-driven cutting-planes
    solver, core is a linear inequality
  • Can contain non-assumptions or negative coefficients
    • Eliminate by adding Boolean axioms
  • Can have arbitrarily large coefficients
    • Reduce to cardinality core, e.g., by division with largest coefficient
\mathit{Core}\colon x_0-x_1+3x_2+3x_3+x_4 \geq 5
1 \geq x, x \geq 0
\mathit{Core'}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card}\colon x_2+x_3+x_4 \geq 2

Reformulate problem

\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
  • Determine    
  • Determine cardinality degree d
  • Enlarge core to extension constraint Ext with d fewer auxiliary variables
  • Add Ext to     with order constraints
  • Reformulate objective
c^*
c^*=2
y_i \geq y_{i+1}
\mathit{Ext}\colon x_2+x_3+x_4 = 2+y_1
\varphi
O'\colon x_1+x_3+2x_4+4+2y_1\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
\varphi' = \varphi \wedge \mathit{Ext}
\mathit{Core}\colon x_0-x_1+3x_2+3x_3+x_4 \geq 5
\mathit{Core'}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card}\colon x_2+x_3+x_4 \geq 2
d=2

Increase lower bound

  • Lower bound increase is
  • Rinse and repeat
c^*\cdot d
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
c^*=2
\mathit{Ext}\colon x_2+x_3+x_4 = 2+y_1
O'\colon x_1+x_3+2x_4+4+2y_1\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
\varphi' = \varphi \wedge \mathit{Ext}
\mathit{Core}\colon x_0-x_1+3x_2+3x_3+x_4 \geq 5
\mathit{Core'}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card}\colon x_2+x_3+x_4 \geq 2
d=2

Advantage of cutting-planes vs clausal

  • Stronger (cardinality) core
  • Stronger lower bound per core
  • No encoding of Ext to clauses

Why a cardinality core?

\varphi \\ O\colon \sum_i c_ix_i \\ \varphi \models \sum_{i} a_i x_i \geq A

Given

\forall i\colon 0 \leq a_i \leq c_i

If

Then

\varphi \models O \geq A

Any core constraint is violated by setting all objective coefficients to 0, so it can be transformed to a constraint satisfying the if-condition.

Why a cardinality core?

Extension constraint would require too many auxiliary variables (assuming unary encoding)

\sum_i a_ix_i = A + \sum_{j\colon 1}^{(\sum_{i}a_i)-A} y_j
\sum_{i\colon 1}^n x_i = A + \sum_{j\colon 1}^{n-A} y_j

compared to

III

Cutting-planes

Core-guided Optimization
Optimizations

Cardinality reduction optimization

O\colon x_1+2x_2+3x_3+4x_4\\ \mathit{Core}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card_1}\colon x_2+x_3+x_4 \geq 2\\ \mathit{Card_2}\colon x_3+x_4 \geq 1
  • Multiple implied cardinalities
    • Pick one that leads to best
      lower bound       increase
2*2\\ 1*3
c_1^* = 2\\ c_2^*=3

Lower bounds:

Lazy extension encoding

\mathit{Card}\colon x_1+x_2+x_3+x_4 \geq 2\\ \mathit{Ext}\colon x_1+x_2+x_3+x_4 = 2+y_1+y_2\\ y_1 \geq y_2
  • Auxiliary variables represent bounds on lhs
  • Extension constraint can be split along the auxiliary variables
  • Setting      to false implies         must be false
  • As long as      is false, the constraint for         is not needed
  • Lazily introduce variable         and corresponding constraint
  • Similar in spirit to [MJML14]
y_1 \Leftrightarrow x_1+x_2+x_3+x_4 \geq 3\\ y_2 \Leftrightarrow x_1+x_2+x_3+x_4 \geq 4\\
y_i
y_{i+1}
y_i
y_{i+1}
y_{i+1}

Interleaving core-guided and linear optimization

Solutions

Objective
(to minimize)

Related ideas in [ADMR15]

Solutions

Objective
(to minimize)

Interleaving core-guided and linear optimization

Related ideas in [ADMR15]

Solutions

Objective
(to minimize)

Interleaving core-guided and linear optimization

Related ideas in [ADMR15]

Solutions

Objective
(to minimize)

Interleaving core-guided and linear optimization

Related ideas in [ADMR15]

Solutions

Objective
(to minimize)

Interleaving core-guided and linear optimization

Related ideas in [ADMR15]

Solutions

Objective
(to minimize)

Interleaving core-guided and linear optimization

Related ideas in [ADMR15]

Solutions

Objective
(to minimize)

Interleaving core-guided and linear optimization

Related ideas in [ADMR15]

IV
Detailed example

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

CGO lower bound derivation

\alpha=\{\}

Simple solution:
x = y = z = v = w = 1

Optimal rational solution:

x = z = 0, y = v = 1/3, w = 2/3

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
Min~~~y+2v+w
+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
\alpha=\{\}
Min~~~y+2v+w

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
\alpha=\{y=0,\\v=0,w=0\}
Min~~~y+2v+w

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
\alpha=\{y=0,\\v=0,w=0\}
Min~~~y+2v+w

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
\alpha=\{y=0,\\v=0,w=0\}
Min~~~y+2v+w
y+w \geq 1
\textcolor{red}{\text{add}~x \geq 0}

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
\alpha=\{y=0,\\v=0,w=0\}
Min~~~y+2v+w
y+w \geq 1

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

\textcolor{red}{y+w = u+1}
\textcolor{red}{\text{introduce}~u}

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
\alpha=\{y=0,\\v=0,w=0\}
Min~~~\textcolor{red}{2v+u+1}
y+w \geq 1

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

\textcolor{red}{y+w = u+1}

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
\alpha=\{v=0,u=0\}
Min~~~2v+u+1

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

y+w = u+1

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
Min~~~2v+u+1

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

y+w = u+1
\alpha=\{v=0,u=0,\\w=0\}
\textcolor{red}{\text{decision}}

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
Min~~~2v+u+1

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

y+w = u+1
\alpha=\{v=0,u=0,\\w=0,y=1\}
\textcolor{red}{\text{propagates}~y=1}

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
Min~~~2v+u+1

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

y+w = u+1
\alpha=\{v=0,u=0,\\w=0,y=1,z=0\}
\textcolor{red}{\text{propagates}~z=0}

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
Min~~~2v+u+1

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

y+w = u+1
\alpha=\{v=0,u=0,\\w=0,y=1,z=0\}
\textcolor{red}{\text{conflict!}}

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1
Min~~~2v+u+1

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

y+w = u+1
\alpha=\{v=0,u=0,\\w=0,y=1,z=0\}
3v \geq 2u+1
\textcolor{red}{\text{division}}
3v \geq 1
v \geq1
\textcolor{red}{+2\times u\geq0}
v= 1
\textcolor{red}{+2\times}
\textcolor{red}{+2\times}
\textcolor{red}{+}

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

y+w = u+1
\alpha=\{v=0,u=0,\\w=0,y=1,z=0\}
v= 1
Min~~~\textcolor{red}{u+3}

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

y+w = u+1
\alpha=\{u=0\}
v= 1
Min~~~u+3

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

y+w = u+1
\alpha=\{u=0\}
v= 1
Min~~~u+3
x=0,y=0,z=0,\\ v=1,w=1,u=0

CGO lower bound derivation

+x+y-z \geq 0 \\ -2y+2z-v \geq -1 \\ -z+2v-w \geq 0 \\ -x+y+w \geq 1

Assume best case objective

Call yields SAT?

Extract core

Reformulate objective,

increase lower bound

Optimal

no

yes

y+w = u+1
\alpha=\{u=0\}
v= 1
Min~~~u+3
x=0,y=0,z=0,\\ v=1,w=1,u=0

CGO lower bound derivation

Proof complexity view

V
Experiments

Overall performance

[PB16]

Overall performance

[PB16]

Overall performance

[PB16]

Overall performance

[PB16]

Stronger cores reach optimality faster

Fewer # variables due to lazy encoding

Take-away

Our contribution

  • Core-guided optimization with cutting-planes reasoning instead of clausal reasoning
  • Investigated different ways to optimize process

Cutting-planes reasoning advantages

  • More efficient search for cores
  • Can find stronger cores
  • No re-encoding to clauses needed
  • Proposed optimizations are easy to implement

Conclusion

  • Huge performance improvement in RoundingSat
  • Still behind MIP solvers on many instances
  • Future work: incorporate MIP preprocessing techniques?

References

  • [FM06] On Solving the Partial MAX-SAT Problem; Fu, Malik; 2006
  • [AKMS12] Unsatisfiability-Based Optimization in clasp; Andres, Kaufmann, Matheis, Schaub; 2012
  • [MDM14] Core-Guided MaxSAT with Soft Cardinality Constraints; Morgado, Dodaro, Marques-Silva; 2014
  • [MJML14] Martins, Joshi, Manquinho, Lynce; Incremental Cardinality Constraints for MaxSAT; 2014
  • [ADMR15] Alviano, Dodaro, Marques-Silva, Ricca; Optimum stable model search: algorithms and implementation; 2015
  • [ADR15] Alviano, Dodaro, Ricca; A MaxSAT Algorithm Using Cardinality Constraints of Bounded Size; 2015
  • [PB16] Latest PB competition - www.cril.univ-artois.fr/PB16/
  • [GBDS20] Core-Guided and Core-Boosted Search for CP; Gange, Berg, Demirović, Stuckey; 2020

Thanks for your attention!