## SOGrounder: modelling and solving with Second-Order Logic

Matthias van der Hallen

28/10/2019

## First-Order Logic

• A domain of non-logical objects: $$alice, bob, charly ...$$
• Predicates: $$Person(alice), Person(bob), Dog(charly)$$
• Functions: $$Owner(charly)=alice$$

Introduce variables representing non-logical objects:

$$\forall x : Person(x) \lor Dog(x).$$

"Everything is either a dog or a person."

## Second-Order Logic

Also introduce variables representing predicates & functions:

$$\exists P : P(alice) \land P(bob).$$

"There is a set that contains both $$alice$$ and $$bob$$."

## Graphs

$$Edge(a,b), Edge(c,b), Edge(c,d),$$

$$Edge(d,c), Edge(d,e), Edge(e,e).$$

Predicate representing its $$Edge$$ relation:

## Colorings

$$color(a)=red, color(b)=blue, color(c)=green$$

$$color(d)=blue, color(e)=red.$$

Coloring represented by the function:

## Homomorphisms

$$h(f) = c, h(g) = d$$

Homomorphism represented by the function:

## Schedules

$$shift(monday)=alice, shift(tuesday)=bob,$$

$$\ldots$$

Schedule represented by the function:

## Why?

• Graphs
• Colorings
• Homomorphisms
• Schedules
• ...

### Some concepts are naturally expressed using predicates or functions:

Many interesting problems use these concepts!

## Graph Mining

Find a graph $$\mathcal{G}$$ such that:

### Core problem

• homomorphisms exist with the + examples
• no homomorphisms exist with the - examples

## Graph Mining

Find a graph $$\mathcal{G}$$ such that:

### Core problem

• homomorphisms exist with the + examples
• no homomorphisms exist with the - examples

+ example

Candidate for $$\mathcal{G}$$

- example

## Graph Mining

$$\lnot \exists f : \forall x, y : \mathcal{G}(x,y) \Rightarrow NegEx(x,y).$$

Find a graph $$\mathcal{G}$$ such that:

• homomorphisms exist with the + examples
• no homomorphisms exist with the - examples

$$\exists f : \forall x, y : \mathcal{G}(x,y) \Rightarrow PosEx(x,y).$$

Theory

This requires support for Second-Order Logic!

## Critical Friendship

Divide $$n$$ people over $$k$$ cars such that

nobody has to sit with someone they dislike.

Critical friendship pair:

Two distinct people s.t. if they would dislike each other, no proper division exists.

## Critical Friendship

Observations:

• Model 'dislikes' as a graph
• Proper division is a $$k$$-coloring
• Two people $$p1$$ and $$p2$$ form a critical pair iff.
Any valid k-colouring maps $$p1$$ and $$p2$$ to the same car

Alice

Bob

David

Charles

Alice

Bob

David

Charles

## Critical Friendship

Observations:

• Model 'dislikes' as a graph
• Proper division is a $$k$$-coloring
• Two people $$p1$$ and $$p2$$ form a critical pair iff.
Any valid k-colouring maps $$p1$$ and $$p2$$ to the same car
type Person. % The Person type, corresponding to nodes
type Car.    % The Car type, corresponding to colours

dislike :: (Person, Person). % A predicate encoding dislikes
p1 :: Person.                % A person constant
p2 :: Person.                % A second person constant

p1 ≠ p2.
∃f :: (Person)→Car:  ∀ a,b :: Person: dislike(a,b) ⇒ f(a)≠f(b).
∀f :: (Person)→Car: (∀ a,b :: Person: dislike(a,b) ⇒ f(a)≠f(b)) ⇒ f(p1)=f(p2).

Model:

## Critical Friendship

type Person. % The Person type, corresponding to nodes
type Car.    % The Car type, corresponding to colours

dislike :: (Person, Person). % A predicate encoding dislikes
p1 :: Person.                % A person constant
p2 :: Person.                % A second person constant

p1 ≠ p2.
∃f :: (Person)→Car:  ∀ a,b :: Person: dislike(a,b) ⇒ f(a)≠f(b).
∀f :: (Person)→Car: (∀ a,b :: Person: dislike(a,b) ⇒ f(a)≠f(b)) ⇒ f(p1)=f(p2).

Model:

Person = {Alice; Bob; Charles; David}.
Car = {Astra; Berlingo}.
dislike = {Alice, Bob; Bob, David; Charles, David}.
p1 = Bob, p2 = Charles

Model expansion

## Ground-and-solve

Step 1: Translate (grounding) the high-level language to a simpeler low-level language.

## Ground-and-solve

Step 2: Use a general-purpose solver to find interpretations for this translation.

## Ground-and-solve

Step 3: Back-translate to an interpretation for the high-level model.

## First-Order Logic

Translation to SAT:

SAT

Formulas over propositional variables

$$x \lor \lnot (y \land z)$$

## First-Order Logic

Translation to SAT:

• Push negations: $$\lnot \exists x : \phi \rightsquigarrow \forall x : \lnot \phi$$,
$$\lnot (x \land y) \rightsquigarrow \lnot x \lor \lnot y$$, ...
• Unnest: $$f(g(x)) = z \rightsquigarrow \exists y : g(x) = y \land f(y)=z$$
• Replace functions $$f/n$$ by predicates $$f'/(n+1)$$
• Ground first-order quantifications:

• Replace ground predicate applications $$P(a)$$ by propositional variables $$p_a$$

$$\forall x : \phi \rightsquigarrow \bigwedge_{c \in type(x)} \phi[x/c]$$.

$$\exist x : \phi \rightsquigarrow \bigvee_{c \in type(x)} \phi[x/c]$$.

## First-Order Logic

### Example

type Person = {Alice; Bob}.
type Car = {Astra; Berlingo}.

dislike :: (Person, Person). % A predicate encoding dislikes
sitsIn  :: (Person)→Car

∀ a, b :: Person : dislike(a,b) ⇒ sitsIn(a) ≠ sitsIn(b).
∀ a, b :: Person : ¬dislike(a,b) ∨ sitsIn(a) ≠ sitsIn(b).
∀ a, b :: Person : ¬dislike(a,b) ∨
(∃ sa, sb :: Car : sitsIn(a) = sa ∧ sitsIn(b) = sb ∧ ≠(sa,sb)).
∀ a, b :: Person : ¬dislike(a,b) ∨
(∃ sa, sb :: Car : sitsIn(a,sa) ∧ sitsIn(b,sb) ∧ ≠(sa,sb)).
(¬dislike(Alice,Bob) ∨
((sitsIn(Alice,Astra)    ∧ sitsIn(Bob,Astra)    ∧ ≠(Astra,    Astra   )) ∨
(sitsIn(Alice,Astra)    ∧ sitsIn(Bob,Berlingo) ∧ ≠(Astra,    Berlingo)) ∨
(sitsIn(Alice,Berlingo) ∧ sitsIn(Bob,Astra)    ∧ ≠(Berlingo, Astra   )) ∨
(sitsIn(Alice,Berlingo) ∧ sitsIn(Bob,Berlingo) ∧ ≠(Berlingo, Berlingo)))) ∧

...

$$(\lnot d_{Alice,Bob} \lor (s_{Alice,Astra} ∧ s_{Bob,Berlingo}) ∨ (s_{Alice,Berlingo} ∧ s_{Bob,Astra})) \land ...$$

## Second-Order Logic

Translation to Quantified Boolean Formulas:

Introduces quantifiers ($$\forall, \exists$$) for propositional variables

$$\forall x\exists y\forall z . x \lor \lnot (y \land z)$$

## Second-Order Logic

Translation to Quantified Boolean Formulas:

Ground second-order quantifications $$\forall P : \phi$$, $$\exist P : \phi$$:

• Ensure all predicate and function names are unique (renaming)
• Ground $$\phi$$
• Based on the type $$T_1 \times \ldots \times T_n$$ of $$P$$, introduce propositions $$p_{t_1,\ldots,t_n}$$ for $$t_1 \in T_1$$, etc. and quantify them correctly.
• For functions, introduce constraints ensuring existence and uniqueness

## Second-Order Logic

### Example

type Person = {Alice; Bob}.
type Car = {Astra; Berlingo}.

dislike :: (Person, Person). % A predicate encoding dislikes

∀s :: (Person)→Car: (∀ a,b :: Person: dislike(a,b) ⇒ s(a)≠s(b)) ⇒ s(p1)=s(p2).

$$\forall s_{Alice,Astra}, s_{Alice,Berlingo}, s_{Bob,Astra}, s_{Bob,Berlingo} :$$

$$(\Psi) \Rightarrow (\lnot d_{Alice,Bob} \lor (s_{Alice,Astra} ∧ s_{Bob,Berlingo}) ∨ (s_{Alice,Berlingo} ∧ s_{Bob,Astra})) \land ...$$

$$(s_{Alice, Astra} \lor s_{Alice,Berlingo} \lor s_{Bob,Astra} \lor s_{Bob,Berlingo}) \land$$

$$(\lnot s_{Alice,Astra} \lor \lnot s_{Alice,Berlingo}) \land$$

$$(\lnot s_{Bob,Astra} \lor \lnot s_{Bob,Berlingo})$$

$$\Psi$$:

## Caveats

• Some propositions introduced by quantifications represent irrelevant predicate applications

• Some solvers require input in conjunctive normal form:

Standard technique: Tseitin transformation

But

$$\bigwedge_i C_i$$  where $$C_i = \bigvee_j p_{i,j}$$

Naive introduction of auxiliary propositions can introduce unwanted dependencies between propositions, hurting search

$$(p_1 \land p_2) \lor p_3 \rightsquigarrow \exists t . (t \Leftrightarrow p_1 \land p_2) \land (t \lor p_3)$$

• Reduced Grounding: RED
• Lifted Unit Propagation
• Ground with Bounds

• Reduced Grounding: RED

Use knowledge about the truth value of sub-formulas to simplify formulas.

$\forall a,b . dislike(a,b) \Rightarrow f(a) \neq f(b).$

While grounding for $$a,b$$, if $$dislike(a,b)$$ can be evaluated, replace it and simplify the formula.

• Reduced Grounding: RED
• Lifted Unit Propagation

Derive Certainly-True and Certainly-False values of unknown predicates using lifted (symbolic) reasoning.

$$\forall x . P(x) \Rightarrow Q(x).$$

If we know e.g. $$P(1)$$, derive $$Q(1)$$

When deriving CT / CF value for second-order variable

$$\exist$$: Process normally

$$\forall$$: derive UNSAT

• Reduced Grounding: RED
• Lifted Unit Propagation
• Ground-with-Bounds

Extend CT/CF concept to (sub-)formulas and manipulate them to reduce grounding.

$$\phi \lor (\forall x . P(x) \Rightarrow Q(x)).$$

$$Type(x) = \{1;2;3\}, P_{ct} = \{1;2\}, Q_{ct} = \{1\}$$ $$\Rightarrow \forall_{ct} = \{1\}$$

Ground $$\forall$$ for $$x \not\in \forall_{ct}$$

## Solving

Consider a Quantified Boolean Formula in Prenex-CNF form:

$$Q_1 X_1, \ldots, Q_n X_n . \phi$$ with $$Q_i \in \{\exist, \forall\}$$

## Solving QBF

Consecutive variables $$x$$ with the same quantifier $$Q$$ are gathered in quantifier blocks $$X$$

$$Q_1 X_, \ldots, Q_n X_n$$ often abbreviated to $$\Pi$$

$$\Pi$$ introduces ordering: $$x \prec_{\Pi} x'$$

## Solving QBF: QCDCL

Conflict / Solution driven clause/cube learning

Generalization of well-known SAT solving technique CDCL: QCDCL

Clause:

Cube:

• Disjunction of literals $$\bigvee l_i$$
• $$\Pi . \phi \Rightarrow C$$
• Conjunction of literals $$\bigvee l_i$$
• $$\Pi . C \Rightarrow \phi$$

## Solving QBF: QCDCL

PCNF $$\phi$$

Extend Assignment $$A$$

$$\phi[A]$$ = T/F

Propagation

T: Learn Cube

F: Learn Clause

No

UNSAT

$$\empty$$ Clause

$$\empty$$ Cube

SAT

Backtrack

Cube/Clause $$\neq\empty$$

## Learn Cube/Clause: QRES

Resolution-Clause/Cube: From $$C_1 \cup \{p\}$$ and $$C_2 \cup \{\overline{p}\}$$, learn $$C_1 \cup C_2$$ if $$p$$ is $$\exist$$ / $$\forall$$

Reduce Clause: Drop $$\forall$$-quantified $$l$$ from $$C \cup \{l\}$$ to learn $$C$$ iff. $$C$$ contains no $$\exist$$-quantified variable $$l'$$ s.t. $$l \prec_{\Pi} l'$$

Reduce Cube: Drop $$\exist$$-quantified $$l$$ from $$C \cup \{l\}$$ to learn $$C$$ iff. $$C$$ contains no $$\forall$$-quantified variable $$l'$$ s.t. $$l \prec_{\Pi} l'$$

## QRES: Example

Consider prefix: $$\exist x, y \forall a\exist z$$

$$\overline{x} \land \overline{y} \land \overline{a} \land \overline{z}$$

$$\overline{x} \land \overline{y} \land a \land z$$

red

$$\overline{x} \land \overline{y} \land \overline{a}$$

$$\overline{x} \land \overline{y} \land a$$

res

$$\overline{x} \land \overline{y}$$

red

red

$$\empty$$

## Conclusion

• If we want to derive 'critical pairs', we can specify it in second-order logic
• To reason over these specifications, we can use the SOGrounder system
• It will ground the specification, resulting in a quantified boolean formula
• Generic, off-the-shelve solvers can determine satisfiability, with assignments or proofs
• We translate the solution back to the problem domain

## Future work

Extend SOGrounder with:

• Support for aggregates
• Higher-Order predicates (Templating)

## Critical Friendship

Observations:

• Model 'dislikes' as a graph
• Proper division is a $$k$$-coloring
• Two people $$p1$$ and $$p2$$ form a critical pair iff.
Any valid k-colouring maps $$p1$$ and $$p2$$ to the same car
type Person. % The Person type, corresponding to nodes
type Car.    % The Car type, corresponding to colours

dislike :: (Person, Person). % A predicate encoding dislikes
p1 :: Person.                % A person constant
p2 :: Person.                % A second person constant

p1 ≠ p2.
∃f :: (Person)→Car:  ∀ a,b :: Person: dislike(a,b) ⇒ f(a)≠f(b).
∀f :: (Person)→Car: (∀ a,b :: Person: dislike(a,b) ⇒ f(a)≠f(b)) ⇒ f(p1)=f(p2).

Model:

type Person. % The Person type, corresponding to nodes
type Car.    % The Car type, corresponding to colours

dislike :: (Person, Person). % A predicate encoding dislikes
p1 :: Person.                % A person constant
p2 :: Person.                % A second person constant

p1 ≠ p2.
∃f :: (Person)→Car:  ∀ a,b :: Person: dislike(a,b) ⇒ f(a)≠f(b).
∀f :: (Person)→Car: (∀ a,b :: Person: dislike(a,b) ⇒ f(a)≠f(b)) ⇒ f(p1)=f(p2).
type Person. % The Person type, corresponding to nodes
type Car.    % The Car type, corresponding to colours

dislike :: (Person, Person). % A predicate encoding dislikes
p1 :: Person.                % A person constant
p2 :: Person.                % A second person constant

p1 ≠ p2.
∃f :: (Person)→Car:  ∀ a,b :: Person: dislike(a,b) ⇒ f(a)≠f(b).
∀f :: (Person)→Car: (∀ a,b :: Person: dislike(a,b) ⇒ f(a)≠f(b)) ⇒ f(p1)=f(p2).

## Ground-and-solve

Step 1: Translate the high-level language to a simpeler low-level language.

Step 2: Use a general-purpose solver to find interpretations for this translation.

Step 3: Back-translate to an interpretation for the high-level model.

## First-Order Logic

Translation to SAT:

Formulas over propositional variables

$$x \lor \lnot y \land z$$

• Push negations: $$\lnot \exists x : \phi \rightsquigarrow \forall x : \lnot \phi$$,
$$\lnot x \land y \rightsquigarrow \lnot x \lor \lnot y$$, ...
• Unnest: $$f(g(x)) = z \rightsquigarrow \exists y : g(x) = y \land f(y)=z$$
• Replace functions $$f/n$$ by predicates $$f'/(n+1)$$
• Ground first-order quantifications:

• Replace ground predicate applications $$P(a)$$ by propositional variables $$p_a$$

$$\forall x : \phi \rightsquigarrow \bigwedge_{c \in type(x)} \phi[x/c]$$.

$$\exist x : \phi \rightsquigarrow \bigvee_{c \in type(x)} \phi[x/c]$$.

## Solving QBF: QCDCL

PCNF $$\phi$$

Extend Assignment

$$\phi[A]$$ = T/F

Propagation

T: Learn Cube

F: Learn Clause

No

UNSAT

$$\empty$$ Clause

$$\empty$$ Cube

SAT

Backtrack

Cube/Clause $$\neq\empty$$

By krr

• 733