## Cutting to the Core of Pseudo-Boolean Optimization: Combining Core-Guided Search with Cutting Planes Reasoning

Jo Devriendt ¹²³ Stephan Gocht ¹² Emir Demirović Jakob Nordström ²¹ Peter J Stuckey

¹ Lund University, Sweden

² University of Copenhagen, Denmark

³ KU Leuven, Belgium

⁴ TU Delft, The Netherlands

Monash University, Australia

jodevriendt.com
slides.com/jod/coreguidedpb-full

Objective
(to minimize)

Solutions

Solutions

Objective
(to minimize)

Solutions

Objective
(to minimize)

Solutions

Objective
(to minimize)

Solutions

Objective
(to minimize)

Solutions

Objective
(to minimize)

## Core-guided Optimization (CGO)

• All variables          are Boolean (0, 1)
• Constraints C are linear inequalities
• Formula φ is a conjunction of constraints
• Negation:
• A literal    is a variable or its negation
• Linear objective function O to be minimized
• WLOG, objective has only positive coefficients

## Prelims

Basically, we are doing 0-1 integer linear programming in this talk

x, y
\overline{x}=1-x
l
• All variables          are Boolean (0, 1)
• Constraints C are linear inequalities
• Formula φ is a conjunction of constraints
• Negation:
• A literal    is a variable or its negation
• Linear objective function O to be minimized
• WLOG, objective has only positive coefficients
• Solver input
• Formula
• Assumptions
• Solver output, either
• Solution satisfying      under
• Core constraint implied by     and violated by

## Prelims: 0-1 linear programming

x, y
\overline{x}=1-x
l
\alpha \colon \{l, \ldots\}
\varphi
\varphi
\alpha
\alpha
\varphi

## Prelims

\alpha \colon \{l, \ldots\}
\varphi

Solver abstraction:

• Input
• Formula
• Assumptions

• Output, either
• Solution satisfying      under
• Core constraint implied by     and violated by
\varphi
\alpha
\alpha
\varphi

## Core-guided Optimization

Solve under assumptions

Extract core

Reformulate problem

Increase lower bound

• Original core-guided idea [FM06]
• CGO also succesfully applied in ASP [AKMS12] and CP [GBDS20]
• Described technique: OLL [AKMS12, MDM14] and ONE [ADR15]

## CGO loop

Solve under assumptions

Extract core

Reformulate problem

Increase lower bound

• Original core-guided idea [FM06]
• CGO also succesfully applied in ASP [AKMS12] and CP [GBDS20]
• Described technique: OLL [AKMS12, MDM14] and ONE [ADR15]

## Solve under assumptions

• Assume all variables in objective to be false
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\

## Extract core

\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
• Classic case: conflict-driven clause learning solver
• Core is a clause
\mathit{Core}\colon x_2+x_3+x_4 \geq 1

## Reformulate problem

\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
• Determine     :
smallest coefficient in objective with variable in core
• Enlarge core to extension constraint Ext with one fewer auxiliary variables
• Add Ext to     with symmetry breakers
• Sound as     implies Core
• Reformulate objective:
Multiply Ext by    ,
subtract lhs and add rhs to O
\mathit{Core}\colon x_2+x_3+x_4 \geq 1
\mathbf{c^*}
c^*=2
\varphi
y_i \geq y_{i+1}
\mathit{Ext}\colon x_2+x_3+x_4 = 1+y_1+y_2
\varphi
O'\colon x_1+x_3+2x_4+2+2y_1+2y_2\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
c^*
\varphi' = \varphi \wedge \mathit{Ext} \wedge y_1 \geq y_2

## Increase lower bound

\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
• Lower bound increase is simply
\mathit{Core}\colon x_2+x_3+x_4 \geq 1
c^*=2
\mathit{Ext}\colon x_2+x_3+x_4 = 1+y_1+y_2
O'\colon x_1+x_3+2x_4+2+2y_1+2y_2\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
c^*
\varphi' = \varphi \wedge \mathit{Ext} \wedge y_1 \geq y_2

## Rinse and repeat

\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
• Repeat procedure with reformulated problem
\mathit{Core}\colon x_2+x_3+x_4 \geq 1
c^*=2
\mathit{Ext}\colon x_2+x_3+x_4 = 1+y_1+y_2
O'\colon x_1+x_3+2x_4+2+2y_1+2y_2\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
\varphi' = \varphi \wedge \mathit{Ext} \wedge y_1 \geq y_2
\varphi', O', \alpha'=\{\overline{x_1},\overline{x_3},\overline{x_4},\overline{y_1},\overline{y_2}\}
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
\mathit{Core}\colon x_2+x_3+x_4 \geq 1
c^*=2
\mathit{Ext}\colon x_2+x_3+x_4 = 1+y_1+y_2
O'\colon x_1+x_3+2x_4+2+2y_1+2y_2\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
\varphi' = \varphi \wedge \mathit{Ext} \wedge y_1 \geq y_2
• Assume all variables in objective to be false
• Conflict-driven clause learning solver returns clausal core
• Determine     :
smallest coefficient in objective with variable in core
• Enlarge core to extension constraint Ext with one fewer auxiliary variables
• Add Ext to     with symmetry breakers
• Sound as     implies Core
• Reformulate objective:
Multiply Ext by    ,
subtract lhs and add rhs to O
• Lower bound increase is simply
• Repeat procedure with reformulated problem
\mathbf{c^*}
\varphi
y_i \geq y_{i+1}
\varphi
c^*
c^*
\varphi', O', \alpha'=\{\overline{x_1},\overline{x_3},\overline{x_4},\overline{y_1},\overline{y_2}\}

# II

## Solve under assumptions

• Assume all variables in objective to be false
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\

## Extract cardinality core

\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
• With a cutting-planes reasoning
solver, core is a linear inequality
• Can contain non-assumptions or negative coefficients
• Eliminate by adding Boolean axioms
• Can have arbitrarily large coefficients
• Reduce to cardinality core, e.g., by division with largest coefficient
\mathit{Core}\colon x_0-x_1+3x_2+3x_3+x_4 \geq 5
1 \geq x, x \geq 0
\mathit{Core'}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card}\colon x_2+x_3+x_4 \geq 2

## Reformulate problem

\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
• Determine
• Determine cardinality degree d
• Enlarge core to extension constraint Ext with d fewer auxiliary variables
• Add Ext to     with symmetry breakers
• Reformulate objective
c^*
c^*=2
y_i \geq y_{i+1}
\mathit{Ext}\colon x_2+x_3+x_4 = 2+y_1
\varphi
O'\colon x_1+x_3+2x_4+4+2y_1\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
\varphi' = \varphi \wedge \mathit{Ext}
\mathit{Core}\colon x_0-x_1+3x_2+3x_3+x_4 \geq 5
\mathit{Core'}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card}\colon x_2+x_3+x_4 \geq 2
d=2

## Increase lower bound

• Lower bound increase is
• Rinse and repeat
c^*d
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
c^*=2
\mathit{Ext}\colon x_2+x_3+x_4 = 2+y_1
O'\colon x_1+x_3+2x_4+4+2y_1\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
\varphi' = \varphi \wedge \mathit{Ext}
\mathit{Core}\colon x_0-x_1+3x_2+3x_3+x_4 \geq 5
\mathit{Core'}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card}\colon x_2+x_3+x_4 \geq 2
d=2

Advantage of cutting planes vs clausal

• Stronger (cardinality) core
• Stronger lower bound per core
• No encoding of Ext to clauses

## Cutting Planes CGO Example

c^*d
\alpha=\{\overline{x_1},\overline{x_2},\overline{x_3},\overline{x_4}\}
\varphi, O\colon x_1+2x_2+3x_3+4x_4\\
c^*=2
\mathit{Ext}\colon x_2+x_3+x_4 = 2+y_1
O'\colon x_1+x_3+2x_4+4+2y_1\\
O' = O - c^*\mathit{lhs_{Ext}} + c^*\mathit{rhs_{Ext}}
\varphi' = \varphi \wedge \mathit{Ext}
\mathit{Core}\colon x_0-x_1+3x_2+3x_3+x_4 \geq 5
\mathit{Core'}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card}\colon x_2+x_3+x_4 \geq 2
d=2
• Assume all variables in objective to be false
• With a cutting-planes reasoning
solver, core is a linear inequality
• Can contain non-assumptions or negative coefficients
• Eliminate by adding Boolean axioms
• Can have arbitrarily large coefficients
• Reduce to cardinality core, e.g., by division with largest coefficient
• Determine
• Determine cardinality degree d
• Enlarge core to extension constraint Ext with d fewer auxiliary variables
• Add Ext to     with symmetry breakers
• Reformulate objective
• Lower bound increase is
1 \geq x, x \geq 0
c^*
y_i \geq y_{i+1}
\varphi

# III

## Cardinality reduction optimization

O\colon x_1+2x_2+3x_3+4x_4\\ \mathit{Core}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card_1}\colon x_2+x_3+x_4 \geq 2\\ \mathit{Card_2}\colon x_3+x_4 \geq 1
• Multiple implied cardinalities
• Pick one that leads to best
lower bound
increase
2*2\\ 1*3
c_1^* = 2\\ c_2^*=3

Lower bounds:

## Cardinality reduction optimization

O\colon x_1+2x_2+3x_3+4x_4\\ \mathit{Core}\colon 3x_2+3x_3+x_4 \geq 4
\mathit{Card_1}\colon x_2+x_3+x_4 \geq 2\\ \mathit{Card_2}\colon x_3+x_4 \geq 1
• Multiple implied cardinalities
• Pick one that leads to best
lower bound
increase
2*2\\ 1*3
c_1^* = 2\\ c_2^*=3

Lower bounds:

## Lazy extension encoding

\mathit{Card}\colon x_1+x_2+x_3+x_4 \geq 2\\ \mathit{Ext}\colon x_1+x_2+x_3+x_4 = 2+y_1+y_2\\ y_1 \geq y_2
• Auxiliary variables represent bounds on lhs
• Extension constraint can be split along the auxiliary variables
• Setting      to false implies         must be false
• As long as      is false, the constraint for         is not needed
• Lazily introduce variable         and corresponding constraint
• Similar in spirit to [MJML14]
y_1 \Leftrightarrow x_1+x_2+x_3+x_4 \geq 3\\ y_2 \Leftrightarrow x_1+x_2+x_3+x_4 \geq 4\\
y_i
y_{i+1}
y_i
y_{i+1}
y_{i+1}

## Lazy extension encoding

\mathit{Card}\colon x_1+x_2+x_3+x_4 \geq 2\\ \mathit{Ext}\colon x_1+x_2+x_3+x_4 = 2+y_1+y_2\\ y_1 \geq y_2
• Auxiliary variables represent bounds on lhs
• Extension constraint can be split along the auxiliary variables
• Setting      to false implies         must be false
• As long as      is false, the constraint for         is not needed
• Lazily introduce variable         and corresponding constraint
• Similar in spirit to [MJML14]
y_1 \Leftrightarrow x_1+x_2+x_3+x_4 \geq 3\\ y_2 \Leftrightarrow x_1+x_2+x_3+x_4 \geq 4\\
y_i
y_{i+1}
y_i
y_{i+1}
y_{i+1}

Solutions

Objective
(to minimize)

Solutions

Objective
(to minimize)

Solutions

Objective
(to minimize)

Solutions

Objective
(to minimize)

Solutions

Objective
(to minimize)

Solutions

Objective
(to minimize)

Solutions

Objective
(to minimize)

# IV Experiments

[PB16]

[PB16]

[PB16]

[PB16]

## Take-away

Our contribution

• Core-guided optimization with cutting-planes reasoning instead of clausal reasoning
• Investigated different ways to optimize process

• More efficient search for cores
• Can find stronger cores
• No re-encoding to clauses needed
• Proposed optimizations are easy to implement

Conclusion

• Huge performance improvement in RoundingSat
• Still behind MIP solvers on many instances
• Future work: incorporate MIP preprocessing techniques?

## References

• [FM06] On Solving the Partial MAX-SAT Problem; Fu, Malik; 2006
• [AKMS12] Unsatisfiability-Based Optimization in clasp; Andres, Kaufmann, Matheis, Schaub; 2012
• [MDM14] Core-Guided MaxSAT with Soft Cardinality Constraints; Morgado, Dodaro, Marques-Silva; 2014
• [MJML14] Martins, Joshi, Manquinho, Lynce; Incremental Cardinality Constraints for MaxSAT; 2014
• [ADMR15] Alviano, Dodaro, Marques-Silva, Ricca; Optimum stable model search: algorithms and implementation; 2015
• [ADR15] Alviano, Dodaro, Ricca; A MaxSAT Algorithm Using Cardinality Constraints of Bounded Size; 2015
• [PB16] Latest PB competition - www.cril.univ-artois.fr/PB16/
• [GBDS20] Core-Guided and Core-Boosted Search for CP; Gange, Berg, Demirović, Stuckey; 2020