Chris Liu
Math gradudate student at Colorado State University
QuickSylver
Chris Liu
Fast solutions to Simultaneous Sylvester Systems
Joint work with Josh Maglione, James Wilson
Simultaneous Sylvester System
A \in \mathbb{R}^{s \times b \times c}\\
B \in \mathbb{R}^{a \times t \times c}\\
C \in \mathbb{R}^{a \times b \times c}
X \in \mathbb{R}^{a \times s}\\
Y \in \mathbb{R}^{t \times b}
Given
Find
Such that
(\forall i) \; XA_i + B_iY = C_i
Applications
Module endomorphisms
$$\operatorname{End}({}_AM) = \{X \in \operatorname{End}(M) \mid (\forall i) \; XA_i = A_iX\}$$
Adjoint Algebra of a bilinear map
$$\operatorname{Adj}(*) = \{(X,Y) \mid (\forall u,v) \; Xu*v = u*Yv\}$$
Centroid of a bilinear map
$$\operatorname{Cen}(*) = \{(X,Y,Z) \mid (\forall u,v) \; Xu*v = u*Yv = Z(u*v)\}$$
\( abc \) equations
\( as + bt \) constraints
\( O(n^6) \) to find a solution as a flattened system
Interwoven striding in augmented matrix with \( O(n^4) \) non-zero entries.
\begin{bmatrix} A_1 \\ \vdots \\ A_b \end{bmatrix} \otimes I_a
\begin{bmatrix}
B_1 \otimes I_b\\
\vdots\\
B_a \otimes I_b
\end{bmatrix}
> t := Random(KTensorSpace(GF(997), [20,20,20])); > Nucleus(t,1,2); Constructing a 800 by 8000 matrix over Finite field of size 997. Computing the nullspace of a 800 by 8000 matrix. Matrix Algebra of degree 40 with 1 generator over GF(997)
(Row vector convention)
(\( 20 \times 20 + 20 \times 20 \)) by (\( 20 \times 20 \times 20 \))
Multilinear algebra library in Magma
Observation: suggestive structure after block row/column permutations
Dense system after clearing
X backsub
Y backsub
Tensors
\text{A tensor is an element of a tensor space}
A tensor space has the data of
- Axes \(A\): Set
- Frame \(V: A \rightarrow \text{Vect}(K) \)
- Multi-linear interpreation $$ \langle - \mid - \rangle: T \times \prod_{a \in A} V_a \rightarrowtail \text{Vect}(K)$$
A cube of numbers as a tensor
\Gamma \in \mathbb{R}^{a \times b \times c}
\langle \Gamma \mid - \rangle: \mathbb{R}^a \times \mathbb{R}^b \times \mathbb{R}^c \rightarrowtail \mathbb{R}
\langle \Gamma \mid e_i,e_j,e_k\rangle = \Gamma_{ijk}
\langle \Gamma \mid u,v,w\rangle = \sum_{i,j,k} \Gamma_{ijk}u_iv_jw_k
Combining tensors via contractions
\langle M \mid_b N \rangle \text{ denotes the resulting tensor.}
M \in \mathbb{R}^{a \times \textcolor{red}{b}} \quad N \in \mathbb{R}^{\textcolor{red}{b} \times c}
(M\cdot N)_{ij} = \sum_{k=1}^{b} M_{i\textcolor{red}{k}}N_{\textcolor{red}{k}j}
Contract on multiple axes
\langle M \mid_{\textcolor{red}{a}\textcolor{blue}{b}} N \rangle \in \mathbb{R}^{c} \times \mathbb{R}^{d} \rightarrowtail \mathbb{R}
M \in \mathbb{R}^{\textcolor{red}{a} \times \textcolor{blue}{b} \times c}, N \in \mathbb{R}^{\textcolor{red}{a} \times \textcolor{blue}{b} \times d}
\langle M \mid_{\textcolor{red}{a}\textcolor{blue}{b}} N \rangle(e_i, e_j) = \sum_{\textcolor{red}{k},\textcolor{blue}{l}}
M_{\textcolor{red}{k}\textcolor{blue}{l}i}N_{\textcolor{red}{k}\textcolor{blue}{l}j}
Tensors and their contractions as diagrams
Simultaneous Sylvester System
A \in \mathbb{R}^{s \times b \times c}\\
B \in \mathbb{R}^{a \times t \times c}\\
C \in \mathbb{R}^{a \times b \times c}
X \in \mathbb{R}^{a \times s}\\
Y \in \mathbb{R}^{t \times b}
Given
Find
Such that
(\forall i) \; XA_i + B_iY = C_i
\text{Find } X,Y \text{ such that } \langle X \mid_s \Lambda \rangle + \langle \Psi \mid_t Y \rangle = \Upsilon
Row Reduction
Mx = b
R \coloneqq (R_n \cdots R_1) \text{ where } RM = \operatorname{RREF}(M)
Given
Find
\text{Then } RMx = Rb \text{ is easy to solve for $x$.}
Idea: act opposite to variables
Face Reduction
\langle \Gamma \mid_a x \rangle = \Upsilon \qquad \Gamma \in \mathbb{R}^{a \times b \times c}
\Delta \text{ where } \langle \Delta \mid_{bc} \Gamma \rangle \text{ face reduced }
Given
Find
\text{Meaning } \langle \Delta \mid_{bc} \Gamma \mid_a x \rangle = \langle \Delta \mid_{bc} \Upsilon \rangle \\ \text{ is easy to solve for $x$.}
Idea: avoid the \( a \) axis
Flattened Matrix Perspective
\Gamma \in \mathbb{R}^{a\times b \times c} \\
(\Gamma^{(i)})_{jk} \coloneqq \Gamma_{kji}
\Gamma^{\flat} \coloneqq \begin{bmatrix}
\Gamma^{(1)}
\\
\vdots
\\
\Gamma^{(c)}
\end{bmatrix}
ab\text{-face reduce } \Gamma \coloneqq \text{Row reduce } \Gamma^{\flat}
Module Perspective
M \text{ a left } \mathbb{M}_{b}(\mathbb{R})\text{-mod generated by } \Gamma^{(1)}, \ldots, \Gamma^{(c)}
\text{Row reduce $\Gamma^{\flat}$} = \text{Find aligned generators for } M
Lemma: Size of minimum generating sets is well-defined
ab\text{-face reduce } \Gamma \text{ is to }\\ \text{find aligned generators for } M
Analogy: row rank
Shaded means face reduced
Example with data in matrix perspective
\Gamma \in \mathbb{R}^{a \times b \times c} \qquad \Gamma_{**1} = \begin{bmatrix}
1 & 0 & 2 & 0 & 6\\
0 & 1 & 3 & 7 & 0\\
0 & 0 & 0 & 0 & 0
\end{bmatrix}
\qquad
\Gamma_{**2} = \begin{bmatrix}
0 & 0 & 1 & 0 & 3\\
0 & 0 & 0 & 0 & 0\\
0 & 0 & 0 & 0 & 0
\end{bmatrix}
\Delta = \begin{bmatrix}
1 & 0 & 0 & -2 & 0 & 0\\
0 & 1 & 0 & -3 & 0 & 0\\
0 & 0 & 0 & 1 & 0 & 0\\
0 & 0 & 1 & 0 & 0 & 0\\
0 & 0 & 0 & 0 & 1 & 0\\
0 & 0 & 0 & 0 & 0 & 1
\end{bmatrix} \in \mathbb{M}_2(\mathbb{M}_3(\mathbb{R}))
\langle \Gamma \mid_{ac} \Delta \rangle \text{ is matrix multiplication, resulting in a face reduced tensor}
\( a=3,b=5,c=2 \)
Challenge: face reducing tensors overlap
E in the way of F
Orthogonal idempotents of matricies
M = \begin{bmatrix}M_1 \\ M_2 \end{bmatrix}, \; \text{rowspan}(M) = \text{rowspan}(M_1)
(\exists e, 1-e), eM = \begin{bmatrix}M_1 \\ 0 \end{bmatrix},\; (1-e)M = \begin{bmatrix} 0 \\ M_2 \end{bmatrix}
(\exists T_1, T_2) \begin{bmatrix}
T_1 & 0\\
T_2 & I
\end{bmatrix} \begin{bmatrix}M_1\\ M_2 \end{bmatrix} = \operatorname{RREF}(M) \quad (\ast)
(\exists T_1, T_2) \begin{bmatrix}
T_1 & 0\\
T_2 & I
\end{bmatrix} \begin{bmatrix}M_1\\ M_2 \end{bmatrix} = \operatorname{RREF}(M) \quad (\ast)
eTeM = \operatorname{RREF}(M)\\
(1-e)TeM + (1-e)M = 0 \\
T(1-e) = (1-e)
T \coloneqq \begin{bmatrix}
T_1 & 0\\
T_2 & I
\end{bmatrix}
eT =
\begin{bmatrix}
T_1 & 0 \\
0 & 0
\end{bmatrix}, \quad
(1-e)T =
\begin{bmatrix}
0 & 0\\
T_2 & I
\end{bmatrix}
(1-e)TeM + (1-e)M = 0
T(1-e) = (1-e)
Matricies
Tensors
eTeM = \operatorname{RREF}(M)
Flattened perspective on \( \Delta \)
\( e \)
\( e \)
\( 1-e \)
\( 1-e \)
E in the way
Problem
Craft face reducing tensors so they commute
Key Insight
Controlled tensors
Notation
Orthogonal Idempotents
M \otimes
\begin{bmatrix}
1 & 0 \\
0 & 0
\end{bmatrix}
+
I_2 \otimes
\begin{bmatrix}
0 & 0\\
0 & 1
\end{bmatrix}
M = M
\begin{bmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0 & 0 & 0
\end{bmatrix}
+
M
\begin{bmatrix}
0 & 0 & 0\\
0 & 0 & 0\\
0 & 0 & 1
\end{bmatrix}
Because both tensors equals to
Proof idea
Proof idea: expand out both definitions
Proof idea continued: no overlap of E and F on (1,1)
Proof idea continued
Proof idea continued: no overlap of E and F on (1,1)
QuickSylver
Simultaneous face reduction
What's it like with the flattened matrix
Dense system after clearing
X backsub
Y backsub
E clearing A
F clearing B
After \(E\) and \(F\)
Dense system
X backsub
Y backsub
Analyze result of contraction by region
Region with small number of variables
Key Lemma
\text{Matrix analogy: } (\forall x), 0 = (1-e)TeMx + (1-e)Mx
\text{Backsubstitution for rest of } X \text{ and } Y
\text{Theorem: $O(n^3)$ time to find a solution to Simultaneous Sylvester System} \\ {\scriptstyle n = a+b+c+s+t}
\text{cubic in } sr_{\Psi} + tr_{\Lambda} \text{ number of variables}\\
% {\scriptstyle \text{cubic factor}
% }
Solve linear system
Backsubstitution
\text{linear in } s(a-r_{\Psi}) + t(b-r_{\Lambda}) \text{ contractions}\\
% {\scriptstyle \text{linear factor}}
Benefit of reasoning with tensor network diagram: avoid describing echelonizing this monstrosity
Implementing QuickSylver
Future: Derivations
A \in \mathbb{R}^{s \times b \times c}\\
B \in \mathbb{R}^{a \times t \times c}\\
C \in \mathbb{R}^{a \times b \times r}
X \in \mathbb{R}^{a \times s}\\
Y \in \mathbb{R}^{t \times b}\\
Z \in \mathbb{R}^{r \times c}
Given
Find
Such that
\langle X \mid_s A \rangle + \langle Y \mid_t B \rangle + \langle Z \mid_r C \rangle = 0
Challenge
Matrix flattening is even worse to describe echelonizing strategy
> t := Random(KTensorSpace(GF(997), [20,20,20])); > DerivationAlgebra(t); Construting a 1200 by 8000 matrix over Finite field of size 997. Adding in possible fusion data. Computing the nullspace of a 1200 by 8000 matrix.
Idea: face reducing tensors with the following properties to get commuting behavior for all 3 face reducing tensors
References
Fast Solutions to Simultaneous Sylvester Systems
By Chris Liu
Fast Solutions to Simultaneous Sylvester Systems
This presentation details ideas for solving Simultaneous Sylvester Systems quickly.
