Symmetry exploitation for combinatorial problems

November 6, 2017, KTH, Stockholm


 

Bart Bogaerts, KU Leuven

(joint work with Jo Devriendt, Maurice Bruynooghe, Marc Denecker)

Setting

  • Given:
    • A logical representation of knowledge
    • (possibly) some instance (a partial structure)
    • An inference task to solve 
  • Search:
    • A solution to your problem
  • E.g.,
    • SAT: determine satisfiability of a propositional theory in CNF
    • FO-MX: find a model of a first-order theory that expands some input structure
    • CP: find a(n) (optimal) solution of a set of constraints
    • MIP, SMT, ASP, ... 

Problem

  • If your specification is highly symmetric, complete solvers tend to get lost 
  • How to exploit this symmetry?

Overview

  • Improved Static Symmetry Breaking for SAT (and ASP and QBF) SAT 2016
  • Symmetric explanation learning: Effective dynamic symmetry handling for SAT SAT 2017 
  • On Local Domain Symmetry for Model Expansion ICLP 2016

Part I

Improved static symmetry breaking for SAT

Or: the story of BreakID

Outline

  • What's up with symmetry in SAT?
  • Our improvements
    • Row interchangeability detection
    • Stabilizer chain symmetry breaking
    • Efficiency optimizations
  • Future ideas for symmetry in SAT

What's up with symmetry in SAT?

  • CNF theory T, literal l, variable x, assignment α
     
  • Symmetry σ: permutation of literals
     
    • commutes with negation        σ(l) = ¬σ(¬l)
    • preserves satisfaction    σ(α) ⊨ T iff α ⊨ T
      • syntactically fixes T                   σ(T) = T
         
  • Set of symmetries Σ generate a mathematical group <Σ> under (composition)

What's up with symmetry in SAT?

  • E.g. pigeonhole problem


     
  • Set of holes {h1 ,h2 ,...,hn }
  • Set of pigeons {p1 ,p2 ,...,pn+1 }
  • Symmetries σ on pigeons and holes
    e.g., swapping h1 and h2

     
  • Symmetry group <Σ> of all permutations on pigeons and/or holes
\forall p \colon \exists h \colon Occupies(p,h)
p:h:Occupies(p,h)\forall p \colon \exists h \colon Occupies(p,h)
\forall p~p'~h\colon Occupies(p,h) \land Occupies(p',h) \Rightarrow p=p'
p p h:Occupies(p,h)Occupies(p,h)p=p\forall p~p'~h\colon Occupies(p,h) \land Occupies(p',h) \Rightarrow p=p'
\sigma \colon Occupies(p_i,h_1) \leftrightarrow Occupies(p_i,h_2)
σ:Occupies(pi,h1)Occupies(pi,h2)\sigma \colon Occupies(p_i,h_1) \leftrightarrow Occupies(p_i,h_2)

Symmetry detection in SAT

  • Saucy [1]
    • Convert CNF to colored graph
    • Automorphisms are syntactical symmetries
    • Result: set Σ of generator symmetries for <Σ>
       
  • Very efficient! 

Static symmetry breaking in SAT

 

  • lex-leader symmetry breaking formula sbf(σ) for σ
    • based on variable order

    •  
  • Shatter preprocessor [2]
    • Given Σ, construct sbf(σ) for each σ∈Σ
    • Linear encoding of sbf(σ) into clauses; add to T 

avoid symmetrical parts of the search space

\forall i\colon (\forall j < i \colon x_j \Leftrightarrow \sigma(x_j)) \Rightarrow \neg x_i \vee \sigma(x_i)
i:(j<i:xjσ(xj))¬xiσ(xi)\forall i\colon (\forall j < i \colon x_j \Leftrightarrow \sigma(x_j)) \Rightarrow \neg x_i \vee \sigma(x_i)
x_0 < x_1 < \ldots < x_n
x0<x1<<xnx_0 < x_1 < \ldots < x_n

What's up with symmetry in SAT?

Let's try Shatter on the pigeonhole problem...

Only 2 more instances solved?

What's up with symmetry in SAT?

Let's try Shatter on the pigeonhole problem...

Only 2 more instances solved?

  • Problem lies with generator symmetries Σ
  • <Σ> is not completely broken by conjunction of sbf(σ), σ∈Σ
  • For pigeonhole, there does exist some small Σ' for which sbf(σ), σ∈ Σ' breaks < Σ> completely

In general, Σ lacks information on structure of the group < Σ>

BreakID tries to exploit symmetry group structure

  • Detect row interchangeability symmetry subgroups of < Σ>
  • Symmetry breaking based on stabilizer chain of < Σ>
  • Small performance optimizations

BreakID: symmetry breaking preprocessor similar to Shatter

Detecting row interchangeability

  • Row interchangeability: common form of symmetry
    • Stems from (locally) interchangeable objects
    • Variables can be ordered as rows in matrix
    • All permutations of rows are symmetries
  • Can be broken completely by constructing sbf only for consecutive row swaps [3]
    • Assuming appropriate variable ordering

Detecting row interchangeability

Occupies(p1,h1) Occupies(p1,h2) Occupies(p1,h3)
Occupies(p2,h1) Occupies(p2,h2) Occupies(p2,h3)
Occupies(p3,h1) Occupies(p3,h2) Occupies(p3,h3)
Occupies(p4,h1) Occupies(p4,h2) Occupies(p4,h3)

Variable rows for 4 pigeons, 3 holes:

Symmetry due to interchangeable pigeons completely broken by

sbf(swap(p1 ,p2 )) sbf(swap(p2 ,p3 )) sbf(swap(p3 ,p4 ))

(Greedily) detecting row interchangeability

  • Input: CNF theory T, Σ detected by Saucy
  • Output: variable matrix M such that rows are interchangeable and <M>⊆<Σ>
  1. extract σ 1 , σ 2 ∈Σ that form 2 subsequent row swaps
    • forms initial 3-rowed variable matrix M
  2. apply every σ∈Σ to all detected rows r∈M so far
    • images σ(r) disjoint of M are candidates to extend M
    • test if swap r ↔ σ(r) is a symmetry by syntactical check on T
    • if success, extend M with σ(r)
  3. use Saucy to extend Σ with new symmetry generators by fixing all variable nodes with variable in M, first row excepted

Let's try BreakID on the pigeonhole problem...

  • Detects full pigeon subsymmetry
  • Poly performance
  • 100+ holes are no problem
     

Detecting row interchangeability

Stabilizer chain symmetry breaking

\forall i\colon (\forall j < i \colon x_j \Leftrightarrow \sigma(x_j)) \Rightarrow \neg x_i \vee \sigma(x_i)
i:(j<i:xjσ(xj))¬xiσ(xi)\forall i\colon (\forall j < i \colon x_j \Leftrightarrow \sigma(x_j)) \Rightarrow \neg x_i \vee \sigma(x_i)
  • Recall sbf(σ):
    • for i=0:
    • Binary symmetry breaking clause
       
  • x is stabilized by <Σ> iff σ(x)=x for all σ∈< Σ>
  • x' ∈ orbit(x) under <Σ> iff there exists σ∈<Σ> s.t. σ(x)=x'
     
  • Given symmetry group < Σ> with smallest non-stabilized variable x, with x' ∈ orbit(x) under < Σ>,
                   is logical consequence of sbf(σ) for all σ∈<Σ>
\neg x_0 \vee \sigma(x_0)
¬x0σ(x0)\neg x_0 \vee \sigma(x_0)
\neg x \vee x'
¬xx\neg x \vee x'

Stabilizer chain symmetry breaking

  • < Σ> has subgroups that have other smallest non-stabilized variables, depending on variable order
  • Create stabilizer chain of < Σ> along variable ordering:
     
  • next subgroup Gi is the stabilizer subgroup stabilizing the next non-stabilized variable in ordering
    • Gi have different smallest non-stabilized variables x
  • for each i: Orbit(x) under Gi  leads to binary symmetry breaking clauses
    • Derives all binary symmetry breaking clauses of <Σ> under variable ordering [4]
<\Sigma> = G_0 \supset G_1 \supset \ldots \supset 1
<Σ>=G0G11<\Sigma> = G_0 \supset G_1 \supset \ldots \supset 1

Stabilizer chain symmetry breaking

Occupies(p1,h1) Occupies(p1,h2) Occupies(p1,h3)
Occupies(p2,h1) Occupies(p2,h2) Occupies(p2,h3)
Occupies(p3,h1) Occupies(p3,h2) Occupies(p3,h3)
Occupies(p4,h1) Occupies(p4,h2) Occupies(p4,h3)

Occupies(p1 ,h1 ) < Occupies(p2 ,h1 ) < Occupies(p3 ,h1 ) < Occupies(p4 ,h1 ) < ...

Stabilizer chain for pigeon symmetry consists of 4 subgroups

G0 permutes rows 1-2-3-4,
G1 permutes rows 2-3-4,
G2 swaps rows 3-4,
G3 =1

Binary symmetry breaking clauses:

¬Occupies(p1 ,h1 ) v Occupies(p2 ,h1 )
...
¬Occupies(p3 ,h1 ) v Occupies(p4 ,h1 )

Further improvements on Shatter

  • More compact conversion of sbf to CNF [5]
    • 3 clauses of size 3 instead of
      4 clauses of sizes 3-4
  • Limit the size of sbf 
  • Limit symmetry detection time of Saucy

BreakID combines row interchangeability detection with stabilizer chain symmetry breaking (approximative)

Symmetry breaking in SAT competitions?

  • BreakID overhead is low enough to be competitive, even on asymmetric instances
  • Participated in SAT13, SAT15, SAT16 competitions
  • Experimental results on SAT14 instances
    (Glucose as base solver)

Summary

  • BreakID follows in Shatter's footsteps (after ~10 years!)
  • Derive group structure information
    • Row interchangeability detection
    • Stabilizer chain based symmetry breaking
      • Both are approximative algorithms
  • Efficiency optimizations ensure competitiveness

Much work remains...

  • Combination with computational algebra should improve group structure detection
    • Intertwine with graph automorphism detection?
  • BreakID is easy to fool with redundant clauses... Insight in properties of real-world problems?
     
  • Extend these structure ideas to dynamic symmetry exploitation?
    • Completeness result on row interchangeability for dynamic symmetry breaking?

[1] Symmetry and Satisfiability: An Update - 2010 - Katebi e.a.
[2] Efficient Symmetry-Breaking for Boolean Satisfiability - 2006 - Aloul e.a.
[3] On the importance of row symmetry - 2014 - Devriendt e.a.
[4] Automatic generation of constraints for partial symmetry breaking - 2011 - Jefferson & Petrie
[5] Symmetry and satisfiability - 2009 - Sakallah
[6] On Local Domain Symmetry for Model Expansion - 2016 - Devriendt e.a. (accepted for ICLP)

Part II

Symmetric explanation learning: Effective dynamic symmetry handling for SAT

Symmetry exploitation in SAT

Two classes of techniques

  • Dynamic symmetry exploitation
  • Static symmetry exploitation/breaking

avoid symmetrical parts of the search space

Symmetry exploitation in SAT

  • Takes symmetries into account during search
  • Requires dedicated solvers
    • SymChaff [3]
    • Symmetry propagation for SAT (SP) [4]
    • Symmetric Learning Scheme (SLS) [5]
    • ...

dynamic symmetry exploitation

Symmetric learning

  • Modern SAT solvers learn new clauses during search
  • Typically by resolution, but the idea works for any proof system
  • Learnt clauses are logical consequences of the theory
  • Whenever c is a consequence of T, so is σ(c) 
  • Why not also learn σ(c)?  

 

Symmetric learning

  • Learning σ(c)  for all   σ ∈<Σ>:
    1. infeasible [6] (simpy too many   σ)
    2. How to ensure that no duplicate/subsumed clauses are learnt?
  • Various algorithms exist that make some choice on which symmetries to apply (SP, SEL)
  • None of them performs as well as state-of-the art static symmetry breaking
  • We present a novel algorithm in this family: symmetric explanation learning

Symmetric explanation learning

  • Idea: we prefer to learn clauses with a high chance of propagation
    • Even better: we prefer to only learn clauses that propagate at least once
  • Idea: symmetries typically permute only a fraction of the literals
  • Hence, if c is unit, σ(c) has a good chance of being unit as well
    • This notion of "interesting" candidate learned clause is dynamic

Symmetric explanation learning

  • Proposal: whenever c propagates, store σ(c)  in a separate clause store θ  for each σ ∈Σ
  • Propagation on θ happens with low priority: only if standard unit propagation is at a fixpoint
  • Whenever a clause in θ propagates, we learn it (we upgrade it to a "normal" learned clause)
  • Whenever we backtrack over the level where c propagated, we remove  σ(c) from θ

SEL: Properties

  • Duplicate/subsumed clauses are never learnt
  • Combinatorial explosion of learnt clauses is contained by:
    • Only considering candidates σ(c) for σ∈Σ (not σ∈<Σ>) and c a learnt clause
    • Removing candidates upon backtracking
  • Still allows for transitive effects (learning σ(σ'(c)) if σ'(c) propagates at least once)

SEL: Optimizations

  • Standard two-watched literal scheme applied to θ
  • Do not store σ(c) if it is satisfied at the time of c's propagation
  • Store a simplification of σ(c) in   θ :
    • Remove all false literals at the time of c's propagation
    • Restore original σ(c) if needed

SEL vs SLS

  • SLS: learn σ(c) for each learned clause c and each σ∈Σ
  • SLS: possibly learns useless clauses (shows up in experiments)
  • SEL: possibly misses useful clauses
  • SLS: no (intrinsic) mechanism to avoid duplicates
  • SLS: no transitive effects (possibly misses useful clauses)

SEL vs SP

  • SP: similar idea to SEL: heuristic to guarantee that symmetric learnt clause propagates at least once
  • SP: based on weak activity
  • SP vs SEL: whenever SP learns σ(c), so does SEL (not vice versa)

Notation

  • State:
  •   : assignment (list of literals)
  •   : decision literals
  •   : learned clause store
  •   : explanation (maps propagated literals to the clause that propagated them)
  •   : symmetrical learned clause store


\langle\alpha,\gamma,\Delta,\mathcal{E},\Theta\rangle
α,γ,Δ,E,Θ\langle\alpha,\gamma,\Delta,\mathcal{E},\Theta\rangle
\alpha
α\alpha
\gamma
γ\gamma
\Delta
Δ\Delta
\mathcal{E}
E\mathcal{E}
\Theta
Θ\Theta

SEL: Experiments

  • Limits:
    • 5000s 
    • 16GB
  • Benchmarks
    • app14, hard14
    • app16, hard16
    • highly symmetric instances
  • Solvers:
    • Glucose 4.0
    • Glucose 4.0 + BreakID
    • Glucose 4.0 + SEL
    • Glucose 4.0 + SP
    • Glucose 4.0 + SLS

If SEL does not outperform static symmetry breaking... Why bother?

Symmetry exploitation in SAT

Static symmetry breaking:

  • Compatible with any solver (simple preprocessing!) 
  • Discards models a priori

Dynamic symmetry exploitation (SEL):

  • Simple idea
  • Requires dedicated solvers
  • Does not discard models a priori
    • Compatible with other inference tasks, such as:
      • Model counting
      • (partial) MAXSAT
      • Optimization
  • "Lazy"

static vs dynamic

Summary

  • We presented the first general-purpose dynamic symmetry exploitation technique +/- on par with static symmetry breaking
  • Based on symmetric learning; propagation-directed selection of learnt clauses
  • No completeness results yet. Under which conditions on Σ is SEL guaranteed to never visit two symmetrical assignments? 

[1] Symmetry and Satisfiability: An Update - 2010 - Katebi e.a.
[2] Efficient Symmetry-Breaking for Boolean Satisfiability - 2006 - Aloul e.a.
[3] On the importance of row symmetry - 2014 - Devriendt e.a.
[4] Automatic generation of constraints for partial symmetry breaking - 2011 - Jefferson & Petrie
[5] Symmetry and satisfiability - 2009 - Sakallah
[6] On Local Domain Symmetry for Model Expansion - 2016 - Devriendt e.a.
[7] Improved Static Symmetry Breaking for SAT  - 2016 - Devriendt e.a.

Part III:
On Local Domain Symmetry for Model Expansion

 

An open research problem:

Do 7825 pigeons fit in 7824 holes? 

Do 7825 pigeons fit in 7824 holes? 

  • impossible for SAT solvers
  • trivial by first-order reasoning
\forall p \colon \exists h \colon Occupies(p,h)
p:h:Occupies(p,h)\forall p \colon \exists h \colon Occupies(p,h)
\forall p~p'~h\colon Occupies(p,h) \land Occupies(p',h) \Rightarrow p=p'
p p h:Occupies(p,h)Occupies(p,h)p=p\forall p~p'~h\colon Occupies(p,h) \land Occupies(p',h) \Rightarrow p=p'

Background: IDP

  • A knowledge base system 
    • Language: extensions first-order logic
      • Types
      • Inductive definitions
      • Aggregates
      • Partial functions
    • Multiple types of inference
      • Model expansion
      • Model checking
      • Propagation (various forms)
      • Entailment
      • Querying
      • Definition evaluation

Model expansion in IDP

  • Given: theory T, partial structure S (interprets some symbols) with finite domain
  • Ground and solve:
    1. Eliminate quantifiers in T (using S)
    2. Solve the resulting problem using MinisatID
          (an extension of Minisat with inductive
          definitions, integer variables, ...)
  • ​Without special treatment: symmetries pose the same problem as for SAT. 

Symmetry exploitation

  • We know how to exploit symmetry in the underlying solver (let's assume we break them statically)
    • (more could, and should, happen here, e.g., exploit them during grounding; that is not our focus)
  • How to detect symmetry?
    • Efficiently: preferably using syntactic properties of the first-order representation
    • get the "right" symmetries
      • yields row interchangeability of the ground theory

Symmetry exploitation

  • We know how to exploit symmetry in the underlying solver (let's assume we break them statically)
    • (more could happen here, e.g., exploit them during grounding; that is not our focus)
  • How to detect symmetry?
    • Efficiently: preferably using syntactic properties of the first-order representation
    • get the "right" symmetries
      • yields row interchangeability of the ground theory

Global domain symmetry

  • A domain permutation is a bijection
  • Each domain permutation induces a structure transformation (renaming)      such that

     
  • Each such structure transformation is a symmetry of any theory
  • If furthermore,      preserves S, then it is also a symmetry of MX(T,S)
\pi: D\to D
π:DD\pi: D\to D
\sigma_\pi
σπ\sigma_\pi
(\pi(d_1),\cdots,\pi(d_n)) \in P^{\sigma_\pi(I)} \text{ iff } (d_1,\cdots,d_n) \in P^I
(π(d1),,π(dn))Pσπ(I) iff (d1,,dn)PI(\pi(d_1),\cdots,\pi(d_n)) \in P^{\sigma_\pi(I)} \text{ iff } (d_1,\cdots,d_n) \in P^I
\sigma_\pi
σπ\sigma_\pi

Local domain symmetry

  • Sometimes, we can exchange domain elements locally
  • Given a set A of argument positions 
  •    and  A induce a structure transformation      that performs    only on those positions in A
  • When is       a symmetry of T?
  • When is       a symmetry of MX(T,S)?

  •  
P|i
PiP|i
\sigma_\pi^A
σπA\sigma_\pi^A
\pi
π\pi
\pi
π\pi
\sigma_\pi^A
σπA\sigma_\pi^A
\sigma_\pi^A
σπA\sigma_\pi^A

Local domain symmetry

  • If for each occurrence of an expression

    in T,
    then      is a symmetry of T.
  • If furthermore      preserves S, it is a symmetry of MX(T,S). 
P(t_1,\cdots, t_n) \text{ with } t_k = f(s)
P(t1,,tn) with tk=f(s)P(t_1,\cdots, t_n) \text{ with } t_k = f(s)
P|k\in A\text{ iff } f|0 \in A
PkA iff f0AP|k\in A\text{ iff } f|0 \in A
\sigma_\pi^A
σπA\sigma_\pi^A
\sigma_\pi^A
σπA\sigma_\pi^A

Local domain symmetry

  • If for each occurrence of an expression

    in T, where neither P nor f are interpreted by S

    and      preserves S, then      is a symmetry of MX(T,S).
P(t_1,\cdots, t_n) \text{ with } t_k = f(s)
P(t1,,tn) with tk=f(s)P(t_1,\cdots, t_n) \text{ with } t_k = f(s)
P|k\in A\text{ iff } f|0 \in A
PkA iff f0AP|k\in A\text{ iff } f|0 \in A
\sigma_\pi^A
σπA\sigma_\pi^A
\sigma_\pi^A
σπA\sigma_\pi^A

Efficiently breakable?

  • Given a set A of argument positions such that A contains at most one argument position per predicate
  • Assume D' is a subset of the domain such that for each permutation    of D',      is a symmetry of MX(T,S)
  • Then, the group of all such symmetries can be broken by lex-leader constraints for swaps of two domain elements in D'
  • Essentially, boils down to row interchangeability for the grounding     
\sigma_\pi^A
σπA\sigma_\pi^A
\pi
π\pi

Experimental results

  • Much more efficient than symmetry detection on the ground level.
  • However, restricted to certain kinds of symmetries. 

Conclusion

  • Presented three recent developments in symmetry exploitation:
    • SAT: detection & static symmetry breaking
    • SAT: dynamic symmetry exploitation (symmetric learning)
    • FO: symmetry detection
  • Work well to speed up search
  • Open questions:
    • More robust static symmetry breaking 
    • Completeness guarantees for dynamic approaches
    • More extensive FO detection (different types of symmetries)

Symmetry exploitation for combinatorial problems

By krr

Symmetry exploitation for combinatorial problems

KTH invited talk

  • 2,397