Symmetry exploitation for combinatorial problems

November 6, 2017, KTH, Stockholm

Bart Bogaerts, KU Leuven

(joint work with Jo Devriendt, Maurice Bruynooghe, Marc Denecker)

Setting

• Given:
• A logical representation of knowledge
• (possibly) some instance (a partial structure)
• An inference task to solve
• Search:
• A solution to your problem
• E.g.,
• SAT: determine satisfiability of a propositional theory in CNF
• FO-MX: find a model of a first-order theory that expands some input structure
• CP: find a(n) (optimal) solution of a set of constraints
• MIP, SMT, ASP, ...

Problem

• If your specification is highly symmetric, complete solvers tend to get lost
• How to exploit this symmetry?

Overview

• Improved Static Symmetry Breaking for SAT (and ASP and QBF) SAT 2016
• Symmetric explanation learning: Effective dynamic symmetry handling for SAT SAT 2017
• On Local Domain Symmetry for Model Expansion ICLP 2016

Improved static symmetry breaking for SAT

Or: the story of BreakID

Outline

• What's up with symmetry in SAT?
• Our improvements
• Row interchangeability detection
• Stabilizer chain symmetry breaking
• Efficiency optimizations
• Future ideas for symmetry in SAT

What's up with symmetry in SAT?

• CNF theory T, literal l, variable x, assignment α

• Symmetry σ: permutation of literals

• commutes with negation        σ(l) = ¬σ(¬l)
• preserves satisfaction    σ(α) ⊨ T iff α ⊨ T
• syntactically fixes T                   σ(T) = T

• Set of symmetries Σ generate a mathematical group <Σ> under (composition)

What's up with symmetry in SAT?

• E.g. pigeonhole problem

• Set of holes {h1 ,h2 ,...,hn }
• Set of pigeons {p1 ,p2 ,...,pn+1 }
• Symmetries σ on pigeons and holes
e.g., swapping h1 and h2

• Symmetry group <Σ> of all permutations on pigeons and/or holes
\forall p \colon \exists h \colon Occupies(p,h)
$\forall p \colon \exists h \colon Occupies(p,h)$
\forall p~p'~h\colon Occupies(p,h) \land Occupies(p',h) \Rightarrow p=p'
$\forall p~p'~h\colon Occupies(p,h) \land Occupies(p',h) \Rightarrow p=p'$
\sigma \colon Occupies(p_i,h_1) \leftrightarrow Occupies(p_i,h_2)
$\sigma \colon Occupies(p_i,h_1) \leftrightarrow Occupies(p_i,h_2)$

Symmetry detection in SAT

• Saucy 
• Convert CNF to colored graph
• Automorphisms are syntactical symmetries
• Result: set Σ of generator symmetries for <Σ>

• Very efficient!

Static symmetry breaking in SAT

• lex-leader symmetry breaking formula sbf(σ) for σ
• based on variable order

•
• Shatter preprocessor 
• Given Σ, construct sbf(σ) for each σ∈Σ
• Linear encoding of sbf(σ) into clauses; add to T

avoid symmetrical parts of the search space

\forall i\colon (\forall j < i \colon x_j \Leftrightarrow \sigma(x_j)) \Rightarrow \neg x_i \vee \sigma(x_i)
$\forall i\colon (\forall j < i \colon x_j \Leftrightarrow \sigma(x_j)) \Rightarrow \neg x_i \vee \sigma(x_i)$
x_0 < x_1 < \ldots < x_n
$x_0 < x_1 < \ldots < x_n$

What's up with symmetry in SAT?

Let's try Shatter on the pigeonhole problem...

Only 2 more instances solved?

What's up with symmetry in SAT?

Let's try Shatter on the pigeonhole problem...

Only 2 more instances solved?

• Problem lies with generator symmetries Σ
• <Σ> is not completely broken by conjunction of sbf(σ), σ∈Σ
• For pigeonhole, there does exist some small Σ' for which sbf(σ), σ∈ Σ' breaks < Σ> completely

In general, Σ lacks information on structure of the group < Σ>

BreakID tries to exploit symmetry group structure

• Detect row interchangeability symmetry subgroups of < Σ>
• Symmetry breaking based on stabilizer chain of < Σ>
• Small performance optimizations

BreakID: symmetry breaking preprocessor similar to Shatter

Detecting row interchangeability

• Row interchangeability: common form of symmetry
• Stems from (locally) interchangeable objects
• Variables can be ordered as rows in matrix
• All permutations of rows are symmetries
• Can be broken completely by constructing sbf only for consecutive row swaps 
• Assuming appropriate variable ordering

Detecting row interchangeability

 Occupies(p1,h1) Occupies(p1,h2) Occupies(p1,h3) Occupies(p2,h1) Occupies(p2,h2) Occupies(p2,h3) Occupies(p3,h1) Occupies(p3,h2) Occupies(p3,h3) Occupies(p4,h1) Occupies(p4,h2) Occupies(p4,h3)

Variable rows for 4 pigeons, 3 holes:

Symmetry due to interchangeable pigeons completely broken by

sbf(swap(p1 ,p2 )) sbf(swap(p2 ,p3 )) sbf(swap(p3 ,p4 ))

(Greedily) detecting row interchangeability

• Input: CNF theory T, Σ detected by Saucy
• Output: variable matrix M such that rows are interchangeable and <M>⊆<Σ>
1. extract σ 1 , σ 2 ∈Σ that form 2 subsequent row swaps
• forms initial 3-rowed variable matrix M
2. apply every σ∈Σ to all detected rows r∈M so far
• images σ(r) disjoint of M are candidates to extend M
• test if swap r ↔ σ(r) is a symmetry by syntactical check on T
• if success, extend M with σ(r)
3. use Saucy to extend Σ with new symmetry generators by fixing all variable nodes with variable in M, first row excepted

Let's try BreakID on the pigeonhole problem...

• Detects full pigeon subsymmetry
• Poly performance
• 100+ holes are no problem

Stabilizer chain symmetry breaking

\forall i\colon (\forall j < i \colon x_j \Leftrightarrow \sigma(x_j)) \Rightarrow \neg x_i \vee \sigma(x_i)
$\forall i\colon (\forall j < i \colon x_j \Leftrightarrow \sigma(x_j)) \Rightarrow \neg x_i \vee \sigma(x_i)$
• Recall sbf(σ):
• for i=0:
• Binary symmetry breaking clause

• x is stabilized by <Σ> iff σ(x)=x for all σ∈< Σ>
• x' ∈ orbit(x) under <Σ> iff there exists σ∈<Σ> s.t. σ(x)=x'

• Given symmetry group < Σ> with smallest non-stabilized variable x, with x' ∈ orbit(x) under < Σ>,
is logical consequence of sbf(σ) for all σ∈<Σ>
\neg x_0 \vee \sigma(x_0)
$\neg x_0 \vee \sigma(x_0)$
\neg x \vee x'
$\neg x \vee x'$

Stabilizer chain symmetry breaking

• < Σ> has subgroups that have other smallest non-stabilized variables, depending on variable order
• Create stabilizer chain of < Σ> along variable ordering:

• next subgroup Gi is the stabilizer subgroup stabilizing the next non-stabilized variable in ordering
• Gi have different smallest non-stabilized variables x
• for each i: Orbit(x) under Gi  leads to binary symmetry breaking clauses
• Derives all binary symmetry breaking clauses of <Σ> under variable ordering 
<\Sigma> = G_0 \supset G_1 \supset \ldots \supset 1
$<\Sigma> = G_0 \supset G_1 \supset \ldots \supset 1$

Stabilizer chain symmetry breaking

 Occupies(p1,h1) Occupies(p1,h2) Occupies(p1,h3) Occupies(p2,h1) Occupies(p2,h2) Occupies(p2,h3) Occupies(p3,h1) Occupies(p3,h2) Occupies(p3,h3) Occupies(p4,h1) Occupies(p4,h2) Occupies(p4,h3)

Occupies(p1 ,h1 ) < Occupies(p2 ,h1 ) < Occupies(p3 ,h1 ) < Occupies(p4 ,h1 ) < ...

Stabilizer chain for pigeon symmetry consists of 4 subgroups

G0 permutes rows 1-2-3-4,
G1 permutes rows 2-3-4,
G2 swaps rows 3-4,
G3 =1

Binary symmetry breaking clauses:

¬Occupies(p1 ,h1 ) v Occupies(p2 ,h1 )
...
¬Occupies(p3 ,h1 ) v Occupies(p4 ,h1 )

Further improvements on Shatter

• More compact conversion of sbf to CNF 
• 3 clauses of size 3 instead of
4 clauses of sizes 3-4
• Limit the size of sbf
• Limit symmetry detection time of Saucy

BreakID combines row interchangeability detection with stabilizer chain symmetry breaking (approximative)

Symmetry breaking in SAT competitions?

• BreakID overhead is low enough to be competitive, even on asymmetric instances
• Participated in SAT13, SAT15, SAT16 competitions
• Experimental results on SAT14 instances
(Glucose as base solver)

Summary

• BreakID follows in Shatter's footsteps (after ~10 years!)
• Derive group structure information
• Row interchangeability detection
• Stabilizer chain based symmetry breaking
• Both are approximative algorithms
• Efficiency optimizations ensure competitiveness

Much work remains...

• Combination with computational algebra should improve group structure detection
• Intertwine with graph automorphism detection?
• BreakID is easy to fool with redundant clauses... Insight in properties of real-world problems?

• Extend these structure ideas to dynamic symmetry exploitation?
• Completeness result on row interchangeability for dynamic symmetry breaking?

 Symmetry and Satisfiability: An Update - 2010 - Katebi e.a.
 Efficient Symmetry-Breaking for Boolean Satisfiability - 2006 - Aloul e.a.
 On the importance of row symmetry - 2014 - Devriendt e.a.
 Automatic generation of constraints for partial symmetry breaking - 2011 - Jefferson & Petrie
 Symmetry and satisfiability - 2009 - Sakallah
 On Local Domain Symmetry for Model Expansion - 2016 - Devriendt e.a. (accepted for ICLP)

Symmetric explanation learning: Effective dynamic symmetry handling for SAT

Symmetry exploitation in SAT

Two classes of techniques

• Dynamic symmetry exploitation
• Static symmetry exploitation/breaking

Symmetry exploitation in SAT

• Takes symmetries into account during search
• Requires dedicated solvers
• SymChaff 
• Symmetry propagation for SAT (SP) 
• Symmetric Learning Scheme (SLS) 
• ...

Symmetric learning

• Modern SAT solvers learn new clauses during search
• Typically by resolution, but the idea works for any proof system
• Learnt clauses are logical consequences of the theory
• Whenever c is a consequence of T, so is σ(c)
• Why not also learn σ(c)?

Symmetric learning

• Learning σ(c)  for all   σ ∈<Σ>:
1. infeasible  (simpy too many   σ)
2. How to ensure that no duplicate/subsumed clauses are learnt?
• Various algorithms exist that make some choice on which symmetries to apply (SP, SEL)
• None of them performs as well as state-of-the art static symmetry breaking
• We present a novel algorithm in this family: symmetric explanation learning

Symmetric explanation learning

• Idea: we prefer to learn clauses with a high chance of propagation
• Even better: we prefer to only learn clauses that propagate at least once
• Idea: symmetries typically permute only a fraction of the literals
• Hence, if c is unit, σ(c) has a good chance of being unit as well
• This notion of "interesting" candidate learned clause is dynamic

Symmetric explanation learning

• Proposal: whenever c propagates, store σ(c)  in a separate clause store θ  for each σ ∈Σ
• Propagation on θ happens with low priority: only if standard unit propagation is at a fixpoint
• Whenever a clause in θ propagates, we learn it (we upgrade it to a "normal" learned clause)
• Whenever we backtrack over the level where c propagated, we remove  σ(c) from θ

SEL: Properties

• Duplicate/subsumed clauses are never learnt
• Combinatorial explosion of learnt clauses is contained by:
• Only considering candidates σ(c) for σ∈Σ (not σ∈<Σ>) and c a learnt clause
• Removing candidates upon backtracking
• Still allows for transitive effects (learning σ(σ'(c)) if σ'(c) propagates at least once)

SEL: Optimizations

• Standard two-watched literal scheme applied to θ
• Do not store σ(c) if it is satisfied at the time of c's propagation
• Store a simplification of σ(c) in   θ :
• Remove all false literals at the time of c's propagation
• Restore original σ(c) if needed

SEL vs SLS

• SLS: learn σ(c) for each learned clause c and each σ∈Σ
• SLS: possibly learns useless clauses (shows up in experiments)
• SEL: possibly misses useful clauses
• SLS: no (intrinsic) mechanism to avoid duplicates
• SLS: no transitive effects (possibly misses useful clauses)

SEL vs SP

• SP: similar idea to SEL: heuristic to guarantee that symmetric learnt clause propagates at least once
• SP: based on weak activity
• SP vs SEL: whenever SP learns σ(c), so does SEL (not vice versa)

Notation

• State:
•   : assignment (list of literals)
•   : decision literals
•   : learned clause store
•   : explanation (maps propagated literals to the clause that propagated them)
•   : symmetrical learned clause store

\langle\alpha,\gamma,\Delta,\mathcal{E},\Theta\rangle
$\langle\alpha,\gamma,\Delta,\mathcal{E},\Theta\rangle$
\alpha
$\alpha$
\gamma
$\gamma$
\Delta
$\Delta$
\mathcal{E}
$\mathcal{E}$
\Theta
$\Theta$

SEL: Experiments

• Limits:
• 5000s
• 16GB
• Benchmarks
• app14, hard14
• app16, hard16
• highly symmetric instances
• Solvers:
• Glucose 4.0
• Glucose 4.0 + BreakID
• Glucose 4.0 + SEL
• Glucose 4.0 + SP
• Glucose 4.0 + SLS

If SEL does not outperform static symmetry breaking... Why bother?

Symmetry exploitation in SAT

Static symmetry breaking:

• Compatible with any solver (simple preprocessing!)

Dynamic symmetry exploitation (SEL):

• Simple idea
• Requires dedicated solvers
• Does not discard models a priori
• Compatible with other inference tasks, such as:
• Model counting
• (partial) MAXSAT
• Optimization
• "Lazy"

Summary

• We presented the first general-purpose dynamic symmetry exploitation technique +/- on par with static symmetry breaking
• Based on symmetric learning; propagation-directed selection of learnt clauses
• No completeness results yet. Under which conditions on Σ is SEL guaranteed to never visit two symmetrical assignments?

 Symmetry and Satisfiability: An Update - 2010 - Katebi e.a.
 Efficient Symmetry-Breaking for Boolean Satisfiability - 2006 - Aloul e.a.
 On the importance of row symmetry - 2014 - Devriendt e.a.
 Automatic generation of constraints for partial symmetry breaking - 2011 - Jefferson & Petrie
 Symmetry and satisfiability - 2009 - Sakallah
 On Local Domain Symmetry for Model Expansion - 2016 - Devriendt e.a.
 Improved Static Symmetry Breaking for SAT  - 2016 - Devriendt e.a.

An open research problem:

Do 7825 pigeons fit in 7824 holes?

• impossible for SAT solvers
• trivial by first-order reasoning
\forall p \colon \exists h \colon Occupies(p,h)
$\forall p \colon \exists h \colon Occupies(p,h)$
\forall p~p'~h\colon Occupies(p,h) \land Occupies(p',h) \Rightarrow p=p'
$\forall p~p'~h\colon Occupies(p,h) \land Occupies(p',h) \Rightarrow p=p'$

Background: IDP

• A knowledge base system
• Language: extensions first-order logic
• Types
• Inductive definitions
• Aggregates
• Partial functions
• Multiple types of inference
• Model expansion
• Model checking
• Propagation (various forms)
• Entailment
• Querying
• Definition evaluation

Model expansion in IDP

• Given: theory T, partial structure S (interprets some symbols) with finite domain
• Ground and solve:
1. Eliminate quantifiers in T (using S)
2. Solve the resulting problem using MinisatID
(an extension of Minisat with inductive
definitions, integer variables, ...)
• ​Without special treatment: symmetries pose the same problem as for SAT.

Symmetry exploitation

• We know how to exploit symmetry in the underlying solver (let's assume we break them statically)
• (more could, and should, happen here, e.g., exploit them during grounding; that is not our focus)
• How to detect symmetry?
• Efficiently: preferably using syntactic properties of the first-order representation
• get the "right" symmetries
• yields row interchangeability of the ground theory

Symmetry exploitation

• We know how to exploit symmetry in the underlying solver (let's assume we break them statically)
• (more could happen here, e.g., exploit them during grounding; that is not our focus)
• How to detect symmetry?
• Efficiently: preferably using syntactic properties of the first-order representation
• get the "right" symmetries
• yields row interchangeability of the ground theory

Global domain symmetry

• A domain permutation is a bijection
• Each domain permutation induces a structure transformation (renaming)      such that

• Each such structure transformation is a symmetry of any theory
• If furthermore,      preserves S, then it is also a symmetry of MX(T,S)
\pi: D\to D
$\pi: D\to D$
\sigma_\pi
$\sigma_\pi$
(\pi(d_1),\cdots,\pi(d_n)) \in P^{\sigma_\pi(I)} \text{ iff } (d_1,\cdots,d_n) \in P^I
$(\pi(d_1),\cdots,\pi(d_n)) \in P^{\sigma_\pi(I)} \text{ iff } (d_1,\cdots,d_n) \in P^I$
\sigma_\pi
$\sigma_\pi$

Local domain symmetry

• Sometimes, we can exchange domain elements locally
• Given a set A of argument positions
•    and  A induce a structure transformation      that performs    only on those positions in A
• When is       a symmetry of T?
• When is       a symmetry of MX(T,S)?

•
P|i
$P|i$
\sigma_\pi^A
$\sigma_\pi^A$
\pi
$\pi$
\pi
$\pi$
\sigma_\pi^A
$\sigma_\pi^A$
\sigma_\pi^A
$\sigma_\pi^A$

Local domain symmetry

• If for each occurrence of an expression

in T,
then      is a symmetry of T.
• If furthermore      preserves S, it is a symmetry of MX(T,S).
P(t_1,\cdots, t_n) \text{ with } t_k = f(s)
$P(t_1,\cdots, t_n) \text{ with } t_k = f(s)$
P|k\in A\text{ iff } f|0 \in A
$P|k\in A\text{ iff } f|0 \in A$
\sigma_\pi^A
$\sigma_\pi^A$
\sigma_\pi^A
$\sigma_\pi^A$

Local domain symmetry

• If for each occurrence of an expression

in T, where neither P nor f are interpreted by S

and      preserves S, then      is a symmetry of MX(T,S).
P(t_1,\cdots, t_n) \text{ with } t_k = f(s)
$P(t_1,\cdots, t_n) \text{ with } t_k = f(s)$
P|k\in A\text{ iff } f|0 \in A
$P|k\in A\text{ iff } f|0 \in A$
\sigma_\pi^A
$\sigma_\pi^A$
\sigma_\pi^A
$\sigma_\pi^A$

Efficiently breakable?

• Given a set A of argument positions such that A contains at most one argument position per predicate
• Assume D' is a subset of the domain such that for each permutation    of D',      is a symmetry of MX(T,S)
• Then, the group of all such symmetries can be broken by lex-leader constraints for swaps of two domain elements in D'
• Essentially, boils down to row interchangeability for the grounding
\sigma_\pi^A
$\sigma_\pi^A$
\pi
$\pi$

Experimental results

• Much more efficient than symmetry detection on the ground level.
• However, restricted to certain kinds of symmetries.

Conclusion

• Presented three recent developments in symmetry exploitation:
• SAT: detection & static symmetry breaking
• SAT: dynamic symmetry exploitation (symmetric learning)
• FO: symmetry detection
• Work well to speed up search
• Open questions:
• More robust static symmetry breaking
• Completeness guarantees for dynamic approaches
• More extensive FO detection (different types of symmetries)

By krr

KTH invited talk

• 515