Symmetry exploitation for combinatorial problems

November 6, 2017, KTH, Stockholm

Bart Bogaerts, KU Leuven

(joint work with Jo Devriendt, Maurice Bruynooghe, Marc Denecker)

Setting

• Given:
  • A logical representation of knowledge
  • (possibly) some instance (a partial structure)
  • An inference task to solve 
• Search:
  • A solution to your problem
• E.g.,
  • SAT: determine satisfiability of a propositional theory in CNF
  • FO-MX: find a model of a first-order theory that expands some input structure
  • CP: find a(n) (optimal) solution of a set of constraints
  • MIP, SMT, ASP, ... 

Problem

• If your specification is highly symmetric, complete solvers tend to get lost 
• How to exploit this symmetry?

Overview

• Improved Static Symmetry Breaking for SAT (and ASP and QBF) SAT 2016
• Symmetric explanation learning: Effective dynamic symmetry handling for SAT SAT 2017 
• On Local Domain Symmetry for Model Expansion ICLP 2016

Part I

Improved static symmetry breaking for SAT

Or: the story of BreakID

Outline

• What's up with symmetry in SAT?
• Our improvements
  • Row interchangeability detection
  • Stabilizer chain symmetry breaking
  • Efficiency optimizations
• Future ideas for symmetry in SAT

What's up with symmetry in SAT?

• CNF theory T, literal l, variable x, assignment α
   
• Symmetry σ: permutation of literals
   
  • commutes with negation        σ(l) = ¬σ(¬l)
  • preserves satisfaction    σ(α) ⊨ T iff α ⊨ T
    • syntactically fixes T                   σ(T) = T
       
• Set of symmetries Σ generate a mathematical group <Σ> under ◦ (composition)

What's up with symmetry in SAT?

• E.g. pigeonhole problem
  
  
   
• Set of holes {h₁ ,h₂ ,...,h_n }
• Set of pigeons {p₁ ,p₂ ,...,p_n+1 }
• Symmetries σ on pigeons and holes
  e.g., swapping h₁ and h₂
  
   
• Symmetry group <Σ> of all permutations on pigeons and/or holes

Symmetry detection in SAT

• Saucy [1]
  • Convert CNF to colored graph
  • Automorphisms are syntactical symmetries
  • Result: set Σ of generator symmetries for <Σ>
     
• Very efficient! 

Static symmetry breaking in SAT

• lex-leader symmetry breaking formula sbf(σ) for σ
  • based on variable order
  • 
     
• Shatter preprocessor [2]
  • Given Σ, construct sbf(σ) for each σ∈Σ
  • Linear encoding of sbf(σ) into clauses; add to T 

avoid symmetrical parts of the search space

What's up with symmetry in SAT?

Let's try Shatter on the pigeonhole problem...

What's up with symmetry in SAT?

Let's try Shatter on the pigeonhole problem...

Only 2 more instances solved?

• Problem lies with generator symmetries Σ
• <Σ> is not completely broken by conjunction of sbf(σ), σ∈Σ
• For pigeonhole, there does exist some small Σ' for which sbf(σ), σ∈ Σ' breaks < Σ> completely

BreakID tries to exploit symmetry group structure

• Detect row interchangeability symmetry subgroups of < Σ>
• Symmetry breaking based on stabilizer chain of < Σ>
• Small performance optimizations

BreakID: symmetry breaking preprocessor similar to Shatter

Detecting row interchangeability

• Row interchangeability: common form of symmetry
  • Stems from (locally) interchangeable objects
  • Variables can be ordered as rows in matrix
  • All permutations of rows are symmetries
• Can be broken completely by constructing sbf only for consecutive row swaps [3]
  • Assuming appropriate variable ordering

Detecting row interchangeability

Variable rows for 4 pigeons, 3 holes:

(Greedily) detecting row interchangeability

• Input: CNF theory T, Σ detected by Saucy
• Output: variable matrix M such that rows are interchangeable and <M>⊆<Σ>

1. extract σ ₁ , σ ₂ ∈Σ that form 2 subsequent row swaps
   • forms initial 3-rowed variable matrix M
2. apply every σ∈Σ to all detected rows r∈M so far
   • images σ(r) disjoint of M are candidates to extend M
   • test if swap r ↔ σ(r) is a symmetry by syntactical check on T
   • if success, extend M with σ(r)
3. use Saucy to extend Σ with new symmetry generators by fixing all variable nodes with variable in M, first row excepted

Let's try BreakID on the pigeonhole problem...

• Detects full pigeon subsymmetry
• Poly performance
• 100+ holes are no problem
   

Detecting row interchangeability

Stabilizer chain symmetry breaking

• Recall sbf(σ):
  • for i=0:
  • Binary symmetry breaking clause
     
• x is stabilized by <Σ> iff σ(x)=x for all σ∈< Σ>
• x' ∈ orbit(x) under <Σ> iff there exists σ∈<Σ> s.t. σ(x)=x'
   
• Given symmetry group < Σ> with smallest non-stabilized variable x, with x' ∈ orbit(x) under < Σ>,
                 is logical consequence of sbf(σ) for all σ∈<Σ>

Stabilizer chain symmetry breaking

• < Σ> has subgroups that have other smallest non-stabilized variables, depending on variable order
• Create stabilizer chain of < Σ> along variable ordering:
   
• next subgroup G_i is the stabilizer subgroup stabilizing the next non-stabilized variable in ordering
  • G_i have different smallest non-stabilized variables x
• for each i: Orbit(x) under G_i  leads to binary symmetry breaking clauses
  • Derives all binary symmetry breaking clauses of <Σ> under variable ordering [4]

Stabilizer chain symmetry breaking

Occupies(p₁ ,h₁ ) < Occupies(p₂ ,h₁ ) < Occupies(p₃ ,h₁ ) < Occupies(p₄ ,h₁ ) < ...

Further improvements on Shatter

• More compact conversion of sbf to CNF [5]
  • 3 clauses of size 3 instead of
    4 clauses of sizes 3-4
• Limit the size of sbf 
• Limit symmetry detection time of Saucy

BreakID combines row interchangeability detection with stabilizer chain symmetry breaking (approximative)

Symmetry breaking in SAT competitions?

• BreakID overhead is low enough to be competitive, even on asymmetric instances
• Participated in SAT13, SAT15, SAT16 competitions
• Experimental results on SAT14 instances
  (Glucose as base solver)

Summary

• BreakID follows in Shatter's footsteps (after ~10 years!)
• Derive group structure information
  • Row interchangeability detection
  • Stabilizer chain based symmetry breaking
    • Both are approximative algorithms
• Efficiency optimizations ensure competitiveness

Much work remains...

• Combination with computational algebra should improve group structure detection
  • Intertwine with graph automorphism detection?
• BreakID is easy to fool with redundant clauses... Insight in properties of real-world problems?
   
• Extend these structure ideas to dynamic symmetry exploitation?
  • Completeness result on row interchangeability for dynamic symmetry breaking?

[1] Symmetry and Satisfiability: An Update - 2010 - Katebi e.a.
[2] Efficient Symmetry-Breaking for Boolean Satisfiability - 2006 - Aloul e.a.
[3] On the importance of row symmetry - 2014 - Devriendt e.a.
[4] Automatic generation of constraints for partial symmetry breaking - 2011 - Jefferson & Petrie
[5] Symmetry and satisfiability - 2009 - Sakallah
[6] On Local Domain Symmetry for Model Expansion - 2016 - Devriendt e.a. (accepted for ICLP)

Part II

Symmetric explanation learning: Effective dynamic symmetry handling for SAT

Symmetry exploitation in SAT

Two classes of techniques

• Dynamic symmetry exploitation
• Static symmetry exploitation/breaking

avoid symmetrical parts of the search space

Symmetry exploitation in SAT

• Takes symmetries into account during search
• Requires dedicated solvers
  • SymChaff [3]
  • Symmetry propagation for SAT (SP) [4]
  • Symmetric Learning Scheme (SLS) [5]
  • ...

dynamic symmetry exploitation

Symmetric learning

• Modern SAT solvers learn new clauses during search
• Typically by resolution, but the idea works for any proof system
• Learnt clauses are logical consequences of the theory
• Whenever c is a consequence of T, so is σ(c) 
• Why not also learn σ(c)?  

Symmetric learning

• Learning σ(c)  for all   σ ∈<Σ>:
  1. infeasible [6] (simpy too many   σ)
  2. How to ensure that no duplicate/subsumed clauses are learnt?
• Various algorithms exist that make some choice on which symmetries to apply (SP, SEL)
• None of them performs as well as state-of-the art static symmetry breaking
• We present a novel algorithm in this family: symmetric explanation learning

Symmetric explanation learning

• Idea: we prefer to learn clauses with a high chance of propagation
  • Even better: we prefer to only learn clauses that propagate at least once
• Idea: symmetries typically permute only a fraction of the literals
• Hence, if c is unit, σ(c) has a good chance of being unit as well
  • This notion of "interesting" candidate learned clause is dynamic

Symmetric explanation learning

• Proposal: whenever c propagates, store σ(c)  in a separate clause store θ  for each σ ∈Σ
• Propagation on θ happens with low priority: only if standard unit propagation is at a fixpoint
• Whenever a clause in θ propagates, we learn it (we upgrade it to a "normal" learned clause)
• Whenever we backtrack over the level where c propagated, we remove  σ(c) from θ

SEL: Properties

• Duplicate/subsumed clauses are never learnt
• Combinatorial explosion of learnt clauses is contained by:
  • Only considering candidates σ(c) for σ∈Σ (not σ∈<Σ>) and c a learnt clause
  • Removing candidates upon backtracking
• Still allows for transitive effects (learning σ(σ'(c)) if σ'(c) propagates at least once)

SEL: Optimizations

• Standard two-watched literal scheme applied to θ
• Do not store σ(c) if it is satisfied at the time of c's propagation
• Store a simplification of σ(c) in   θ :
  • Remove all false literals at the time of c's propagation
  • Restore original σ(c) if needed

SEL vs SLS

• SLS: learn σ(c) for each learned clause c and each σ∈Σ
• SLS: possibly learns useless clauses (shows up in experiments)
• SEL: possibly misses useful clauses
• SLS: no (intrinsic) mechanism to avoid duplicates
• SLS: no transitive effects (possibly misses useful clauses)

SEL vs SP

• SP: similar idea to SEL: heuristic to guarantee that symmetric learnt clause propagates at least once
• SP: based on weak activity
• SP vs SEL: whenever SP learns σ(c), so does SEL (not vice versa)

Notation

• State:
•   : assignment (list of literals)
•   : decision literals
•   : learned clause store
•   : explanation (maps propagated literals to the clause that propagated them)
•   : symmetrical learned clause store

SEL: Experiments

• Limits:
  • 5000s 
  • 16GB
• Benchmarks
  • app14, hard14
  • app16, hard16
  • highly symmetric instances
• Solvers:
  • Glucose 4.0
  • Glucose 4.0 + BreakID
  • Glucose 4.0 + SEL
  • Glucose 4.0 + SP
  • Glucose 4.0 + SLS

If SEL does not outperform static symmetry breaking... Why bother?

Symmetry exploitation in SAT

Static symmetry breaking:

• Compatible with any solver (simple preprocessing!) 
• Discards models a priori

Dynamic symmetry exploitation (SEL):

• Simple idea
• Requires dedicated solvers
• Does not discard models a priori
  • Compatible with other inference tasks, such as:
    • Model counting
    • (partial) MAXSAT
    • Optimization
• "Lazy"

static vs dynamic

Summary

• We presented the first general-purpose dynamic symmetry exploitation technique +/- on par with static symmetry breaking
• Based on symmetric learning; propagation-directed selection of learnt clauses
• No completeness results yet. Under which conditions on Σ is SEL guaranteed to never visit two symmetrical assignments? 

[1] Symmetry and Satisfiability: An Update - 2010 - Katebi e.a.
[2] Efficient Symmetry-Breaking for Boolean Satisfiability - 2006 - Aloul e.a.
[3] On the importance of row symmetry - 2014 - Devriendt e.a.
[4] Automatic generation of constraints for partial symmetry breaking - 2011 - Jefferson & Petrie
[5] Symmetry and satisfiability - 2009 - Sakallah
[6] On Local Domain Symmetry for Model Expansion - 2016 - Devriendt e.a.
[7] Improved Static Symmetry Breaking for SAT  - 2016 - Devriendt e.a.

Part III:
On Local Domain Symmetry for Model Expansion

An open research problem:

Do 7825 pigeons fit in 7824 holes? 

• impossible for SAT solvers
• trivial by first-order reasoning

Background: IDP

• A knowledge base system 
  • Language: extensions first-order logic
    • Types
    • Inductive definitions
    • Aggregates
    • Partial functions
  • Multiple types of inference
    • Model expansion
    • Model checking
    • Propagation (various forms)
    • Entailment
    • Querying
    • Definition evaluation

Model expansion in IDP

• Given: theory T, partial structure S (interprets some symbols) with finite domain
• Ground and solve:
  1. Eliminate quantifiers in T (using S)
  2. Solve the resulting problem using MinisatID
         (an extension of Minisat with inductive
         definitions, integer variables, ...)
• ​Without special treatment: symmetries pose the same problem as for SAT. 

dtai.cs.kuleuven.be/krr/idp-ide/

Symmetry exploitation

• We know how to exploit symmetry in the underlying solver (let's assume we break them statically)
  • (more could, and should, happen here, e.g., exploit them during grounding; that is not our focus)
• How to detect symmetry?
  • Efficiently: preferably using syntactic properties of the first-order representation
  • get the "right" symmetries
    • yields row interchangeability of the ground theory

Symmetry exploitation

• We know how to exploit symmetry in the underlying solver (let's assume we break them statically)
  • (more could happen here, e.g., exploit them during grounding; that is not our focus)
• How to detect symmetry?
  • Efficiently: preferably using syntactic properties of the first-order representation
  • get the "right" symmetries
    • yields row interchangeability of the ground theory

Global domain symmetry

• A domain permutation is a bijection
• Each domain permutation induces a structure transformation (renaming)      such that
  
   
• Each such structure transformation is a symmetry of any theory
• If furthermore,      preserves S, then it is also a symmetry of MX(T,S)

Local domain symmetry

• Sometimes, we can exchange domain elements locally
• Given a set A of argument positions 
•    and  A induce a structure transformation      that performs    only on those positions in A
• When is       a symmetry of T?
• When is       a symmetry of MX(T,S)?

 

Local domain symmetry

• If for each occurrence of an expression
  
  in T,
  then      is a symmetry of T.
• If furthermore      preserves S, it is a symmetry of MX(T,S). 

Local domain symmetry

• If for each occurrence of an expression
  
  in T, where neither P nor f are interpreted by S
  
  and      preserves S, then      is a symmetry of MX(T,S).

Efficiently breakable?

• Given a set A of argument positions such that A contains at most one argument position per predicate
• Assume D' is a subset of the domain such that for each permutation    of D',      is a symmetry of MX(T,S)
• Then, the group of all such symmetries can be broken by lex-leader constraints for swaps of two domain elements in D'
• Essentially, boils down to row interchangeability for the grounding     

Experimental results

• Much more efficient than symmetry detection on the ground level.
• However, restricted to certain kinds of symmetries. 

Conclusion

• Presented three recent developments in symmetry exploitation:
  • SAT: detection & static symmetry breaking
  • SAT: dynamic symmetry exploitation (symmetric learning)
  • FO: symmetry detection
• Work well to speed up search
• Open questions:
  • More robust static symmetry breaking 
  • Completeness guarantees for dynamic approaches
  • More extensive FO detection (different types of symmetries)