Symmetry exploitation for combinatorial problems
November 6, 2017, KTH, Stockholm
Bart Bogaerts, KU Leuven
(joint work with Jo Devriendt, Maurice Bruynooghe, Marc Denecker)
Setting
 Given:
 A logical representation of knowledge
 (possibly) some instance (a partial structure)
 An inference task to solve
 Search:
 A solution to your problem
 E.g.,
 SAT: determine satisfiability of a propositional theory in CNF
 FOMX: find a model of a firstorder theory that expands some input structure
 CP: find a(n) (optimal) solution of a set of constraints
 MIP, SMT, ASP, ...
Problem
 If your specification is highly symmetric, complete solvers tend to get lost
 How to exploit this symmetry?
Overview
 Improved Static Symmetry Breaking for SAT (and ASP and QBF) SAT 2016
 Symmetric explanation learning: Effective dynamic symmetry handling for SAT SAT 2017
 On Local Domain Symmetry for Model Expansion ICLP 2016
Part I
Improved static symmetry breaking for SAT
Or: the story of BreakID
Outline
 What's up with symmetry in SAT?
 Our improvements
 Row interchangeability detection
 Stabilizer chain symmetry breaking
 Efficiency optimizations
 Future ideas for symmetry in SAT
What's up with symmetry in SAT?
 CNF theory
T, literal
l, variable
x, assignment
α
 Symmetry
σ: permutation of literals
 commutes with negation σ(l) = ¬σ(¬l)
 preserves satisfaction
σ(α) ⊨ T iff
α ⊨ T
 syntactically fixes
T
σ(T) = T
 syntactically fixes
T
σ(T) = T
 Set of symmetries Σ generate a mathematical group <Σ> under ◦ (composition)
What's up with symmetry in SAT?
 E.g. pigeonhole problem
 Set of holes {h_{1} ,h_{2} ,...,h_{n} }
 Set of pigeons {p_{1} ,p_{2} ,...,p_{n+1} }
 Symmetries σ on pigeons and holes
e.g., swapping h_{1} and h_{2}
 Symmetry group <Σ> of all permutations on pigeons and/or holes
Symmetry detection in SAT

Saucy [1]
 Convert CNF to colored graph
 Automorphisms are syntactical symmetries
 Result: set Σ of generator symmetries for <Σ>
 Very efficient!
Static symmetry breaking in SAT

lexleader symmetry breaking formula sbf(σ) for σ
 based on variable order


Shatter preprocessor [2]
 Given Σ, construct sbf(σ) for each σ∈Σ
 Linear encoding of sbf(σ) into clauses; add to T
avoid symmetrical parts of the search space
What's up with symmetry in SAT?
Let's try Shatter on the pigeonhole problem...
Only 2 more instances solved?
What's up with symmetry in SAT?
Let's try Shatter on the pigeonhole problem...
Only 2 more instances solved?
 Problem lies with generator symmetries Σ
 <Σ> is not completely broken by conjunction of sbf(σ), σ∈Σ
 For pigeonhole, there does exist some small Σ' for which sbf(σ), σ∈ Σ' breaks < Σ> completely
In general, Σ lacks information on structure of the group < Σ>
BreakID tries to exploit symmetry group structure
 Detect row interchangeability symmetry subgroups of < Σ>
 Symmetry breaking based on stabilizer chain of < Σ>
 Small performance optimizations
BreakID: symmetry breaking preprocessor similar to Shatter
Detecting row interchangeability
 Row interchangeability: common form of symmetry
 Stems from (locally) interchangeable objects
 Variables can be ordered as rows in matrix
 All permutations of rows are symmetries
 Can be broken completely by constructing sbf only for consecutive row swaps [3]
 Assuming appropriate variable ordering
Detecting row interchangeability
Occupies(p1,h1)  Occupies(p1,h2)  Occupies(p1,h3) 
Occupies(p2,h1)  Occupies(p2,h2)  Occupies(p2,h3) 
Occupies(p3,h1)  Occupies(p3,h2)  Occupies(p3,h3) 
Occupies(p4,h1)  Occupies(p4,h2)  Occupies(p4,h3) 
Variable rows for 4 pigeons, 3 holes:
Symmetry due to interchangeable pigeons completely broken by
sbf(swap(p_{1} ,p_{2} )) ∧ sbf(swap(p_{2} ,p_{3} )) ∧ sbf(swap(p_{3} ,p_{4} ))
(Greedily) detecting row interchangeability
 Input: CNF theory T, Σ detected by Saucy
 Output: variable matrix M such that rows are interchangeable and <M>⊆<Σ>
 extract σ _{1} , σ _{2} ∈Σ that form 2 subsequent row swaps
 forms initial 3rowed variable matrix M
 apply every σ∈Σ to all detected rows r∈M so far
 images σ(r) disjoint of M are candidates to extend M
 test if swap r ↔ σ(r) is a symmetry by syntactical check on T
 if success, extend M with σ(r)
 use Saucy to extend Σ with new symmetry generators by fixing all variable nodes with variable in M, first row excepted
Let's try BreakID on the pigeonhole problem...
 Detects full pigeon subsymmetry
 Poly performance
 100+ holes are no problem
Detecting row interchangeability
Stabilizer chain symmetry breaking
 Recall sbf(σ):
 for i=0:

Binary symmetry breaking clause
 x is stabilized by <Σ> iff σ(x)=x for all σ∈< Σ>

x' ∈ orbit(x) under <Σ> iff there exists σ∈<Σ> s.t. σ(x)=x'
 Given symmetry group < Σ> with smallest nonstabilized variable x, with x' ∈ orbit(x) under < Σ>,
is logical consequence of sbf(σ) for all σ∈<Σ>
Stabilizer chain symmetry breaking
 < Σ> has subgroups that have other smallest nonstabilized variables, depending on variable order
 Create stabilizer chain of < Σ> along variable ordering:
 next subgroup G_{i} is the stabilizer subgroup stabilizing the next nonstabilized variable in ordering
 G_{i} have different smallest nonstabilized variables x
 for each i: Orbit(x) under G_{i} leads to binary symmetry breaking clauses
 Derives all binary symmetry breaking clauses of <Σ> under variable ordering [4]
Stabilizer chain symmetry breaking
Occupies(p1,h1)  Occupies(p1,h2)  Occupies(p1,h3) 
Occupies(p2,h1)  Occupies(p2,h2)  Occupies(p2,h3) 
Occupies(p3,h1)  Occupies(p3,h2)  Occupies(p3,h3) 
Occupies(p4,h1)  Occupies(p4,h2)  Occupies(p4,h3) 
Occupies(p_{1} ,h_{1} ) < Occupies(p_{2 } ,h_{1} ) < Occupies(p_{3 } ,h_{1} ) < Occupies(p_{4} ,h_{1} ) < ...
Stabilizer chain for pigeon symmetry consists of 4 subgroups
G_{0} permutes rows 1234,
G_{1} permutes rows 234,
G_{2} swaps rows 34,
G_{3} =1
Binary symmetry breaking clauses:
¬Occupies(p_{1 } ,h_{1} ) v Occupies(p_{2} ,h_{1} )
...
¬Occupies(p_{3 } ,h_{1} ) v Occupies(p_{4 } ,h_{1} )
Further improvements on Shatter
 More compact conversion of sbf to CNF [5]
 3 clauses of size 3 instead of
4 clauses of sizes 34
 3 clauses of size 3 instead of
 Limit the size of sbf
 Limit symmetry detection time of Saucy
BreakID combines row interchangeability detection with stabilizer chain symmetry breaking (approximative)
Symmetry breaking in SAT competitions?
 BreakID overhead is low enough to be competitive, even on asymmetric instances
 Participated in SAT13, SAT15, SAT16 competitions
 Experimental results on SAT14 instances
(Glucose as base solver)
Summary
 BreakID follows in Shatter's footsteps (after ~10 years!)
 Derive group structure information
 Row interchangeability detection
 Stabilizer chain based symmetry breaking
 Both are approximative algorithms
 Efficiency optimizations ensure competitiveness
Much work remains...
 Combination with computational algebra should improve group structure detection
 Intertwine with graph automorphism detection?
 BreakID is easy to fool with redundant clauses... Insight in properties of realworld problems?
 Extend these structure ideas to dynamic symmetry exploitation?
 Completeness result on row interchangeability for dynamic symmetry breaking?
[1] Symmetry and Satisfiability: An Update  2010  Katebi e.a.
[2] Efficient SymmetryBreaking for Boolean Satisfiability  2006  Aloul e.a.
[3] On the importance of row symmetry  2014  Devriendt e.a.
[4] Automatic generation of constraints for partial symmetry breaking  2011  Jefferson & Petrie
[5] Symmetry and satisfiability  2009  Sakallah
[6] On Local Domain Symmetry for Model Expansion  2016  Devriendt e.a. (accepted for ICLP)
Part II
Symmetric explanation learning: Effective dynamic symmetry handling for SAT
Symmetry exploitation in SAT
Two classes of techniques
 Dynamic symmetry exploitation
 Static symmetry exploitation/breaking
avoid symmetrical parts of the search space
Symmetry exploitation in SAT
 Takes symmetries into account during search
 Requires dedicated solvers
 SymChaff [3]
 Symmetry propagation for SAT (SP) [4]
 Symmetric Learning Scheme (SLS) [5]
 ...
dynamic symmetry exploitation
Symmetric learning
 Modern SAT solvers learn new clauses during search
 Typically by resolution, but the idea works for any proof system
 Learnt clauses are logical consequences of the theory
 Whenever c is a consequence of T, so is σ(c)
 Why not also learn σ(c)?
Symmetric learning
 Learning
σ(c)
for
all
σ
∈<Σ>:
 infeasible [6] (simpy too many σ)
 How to ensure that no duplicate/subsumed clauses are learnt?
 Various algorithms exist that make some choice on which symmetries to apply (SP, SEL)
 None of them performs as well as stateofthe art static symmetry breaking
 We present a novel algorithm in this family: symmetric explanation learning
Symmetric explanation learning
 Idea: we prefer to learn clauses with a
high chance of propagation
 Even better: we prefer to only learn clauses that propagate at least once
 Idea: symmetries typically permute only a fraction of the literals
 Hence, if
c is unit,
σ(c) has a good chance of being unit as well
 This notion of "interesting" candidate learned clause is dynamic
Symmetric explanation learning
 Proposal: whenever c propagates, store σ(c) in a separate clause store θ for each σ ∈Σ
 Propagation on θ happens with low priority: only if standard unit propagation is at a fixpoint
 Whenever a clause in θ propagates, we learn it (we upgrade it to a "normal" learned clause)
 Whenever we backtrack over the level where c propagated, we remove σ(c) from θ
SEL: Properties
 Duplicate/subsumed clauses are never learnt
 Combinatorial
explosion of learnt clauses is
contained by:
 Only considering candidates σ(c) for σ∈Σ (not σ∈<Σ>) and c a learnt clause
 Removing candidates upon backtracking
 Still allows for transitive effects (learning σ(σ'(c)) if σ'(c) propagates at least once)
SEL: Optimizations
 Standard twowatched literal scheme applied to θ
 Do not store σ(c) if it is satisfied at the time of c's propagation
 Store a simplification of
σ(c)
in
θ
:
 Remove all false literals at the time of c's propagation
 Restore original σ(c) if needed
SEL vs SLS
 SLS: learn σ(c) for each learned clause c and each σ∈Σ
 SLS: possibly learns useless clauses (shows up in experiments)
 SEL: possibly misses useful clauses
 SLS: no (intrinsic) mechanism to avoid duplicates
 SLS: no transitive effects (possibly misses useful clauses)
SEL vs SP
 SP: similar idea to SEL: heuristic to guarantee that symmetric learnt clause propagates at least once
 SP: based on weak activity
 SP vs SEL: whenever SP learns σ(c), so does SEL (not vice versa)
Notation
 State:
 : assignment (list of literals)
 : decision literals
 : learned clause store
 : explanation (maps propagated literals to the clause that propagated them)
 : symmetrical learned clause store
SEL: Experiments
 Limits:
 5000s
 16GB
 Benchmarks
 app14, hard14
 app16, hard16
 highly symmetric instances
 Solvers:
 Glucose 4.0
 Glucose 4.0 + BreakID
 Glucose 4.0 + SEL
 Glucose 4.0 + SP
 Glucose 4.0 + SLS
If SEL does not outperform static symmetry breaking... Why bother?
Symmetry exploitation in SAT
Static symmetry breaking:
 Compatible with any solver (simple preprocessing!)
 Discards models a priori
Dynamic symmetry exploitation (SEL):
 Simple idea
 Requires dedicated solvers
 Does not discard models a priori

Compatible with other inference tasks, such as:
 Model counting
 (partial) MAXSAT
 Optimization

Compatible with other inference tasks, such as:
 "Lazy"
static vs dynamic
Summary
 We presented the first generalpurpose dynamic symmetry exploitation technique +/ on par with static symmetry breaking
 Based on symmetric learning; propagationdirected selection of learnt clauses
 No completeness results yet. Under which conditions on Σ is SEL guaranteed to never visit two symmetrical assignments?
[1] Symmetry and Satisfiability: An Update  2010  Katebi e.a.
[2] Efficient SymmetryBreaking for Boolean Satisfiability  2006  Aloul e.a.
[3] On the importance of row symmetry  2014  Devriendt e.a.
[4] Automatic generation of constraints for partial symmetry breaking  2011  Jefferson & Petrie
[5] Symmetry and satisfiability  2009  Sakallah
[6] On Local Domain Symmetry for Model Expansion  2016  Devriendt e.a.
[7] Improved Static Symmetry Breaking for SAT
 2016  Devriendt e.a.
Part III:
On Local Domain Symmetry for Model Expansion
An open research problem:
Do 7825 pigeons fit in 7824 holes?
Do 7825 pigeons fit in 7824 holes?
 impossible for SAT solvers
 trivial by firstorder reasoning
Background: IDP
 A knowledge base system

Language: extensions firstorder logic
 Types
 Inductive definitions
 Aggregates
 Partial functions
 Multiple types of inference
 Model expansion
 Model checking
 Propagation (various forms)
 Entailment
 Querying
 Definition evaluation

Language: extensions firstorder logic
Model expansion in IDP
 Given: theory T, partial structure S (interprets some symbols) with finite domain

Ground and solve:
 Eliminate quantifiers in T (using S)
 Solve the resulting problem using MinisatID
(an extension of Minisat with inductive
definitions, integer variables, ...)
 Without special treatment: symmetries pose the same problem as for SAT.
Symmetry exploitation
 We know how to exploit symmetry in the underlying solver (let's assume we break them statically)
 (more could, and should, happen here, e.g., exploit them during grounding; that is not our focus)
 How to detect symmetry?
 Efficiently: preferably using syntactic properties of the firstorder representation
 get the "right" symmetries
 yields row interchangeability of the ground theory
Symmetry exploitation
 We know how to exploit symmetry in the underlying solver (let's assume we break them statically)
 (more could happen here, e.g., exploit them during grounding; that is not our focus)
 How to detect symmetry?
 Efficiently: preferably using syntactic properties of the firstorder representation
 get the "right" symmetries
 yields row interchangeability of the ground theory
Global domain symmetry
 A domain permutation is a bijection
 Each domain permutation induces a structure transformation (renaming) such that
 Each such structure transformation is a symmetry of any theory
 If furthermore, preserves S, then it is also a symmetry of MX(T,S)
Local domain symmetry
 Sometimes, we can exchange domain elements locally
 Given a set A of argument positions
 and A induce a structure transformation that performs only on those positions in A
 When is a symmetry of T?
 When is a symmetry of MX(T,S)?
Local domain symmetry
 If for each occurrence of an expression
in T,
then is a symmetry of T.  If furthermore preserves S, it is a symmetry of MX(T,S).
Local domain symmetry
 If for each occurrence of an expression
in T, where neither P nor f are interpreted by S
and preserves S, then is a symmetry of MX(T,S).
Efficiently breakable?
 Given a set A of argument positions such that A contains at most one argument position per predicate
 Assume D' is a subset of the domain such that for each permutation of D', is a symmetry of MX(T,S)
 Then, the group of all such symmetries can be broken by lexleader constraints for swaps of two domain elements in D'
 Essentially, boils down to row interchangeability for the grounding
Experimental results
 Much more efficient than symmetry detection on the ground level.
 However, restricted to certain kinds of symmetries.
Conclusion
 Presented three recent developments in symmetry exploitation:
 SAT: detection & static symmetry breaking
 SAT: dynamic symmetry exploitation (symmetric learning)
 FO: symmetry detection
 Work well to speed up search
 Open questions:
 More robust static symmetry breaking
 Completeness guarantees for dynamic approaches
 More extensive FO detection (different types of symmetries)
Symmetry exploitation for combinatorial problems
By krr
Symmetry exploitation for combinatorial problems
KTH invited talk
 966