# A Tensor Playground:

### a demonstration of $$\textsf{TensorSpace}$$

#### Joshua Maglione

jmaglione@math.uni-bielefeld.de

Universität Bielefeld

Began in 2015 and is still evolving.

Joint with Pete Brooksbank and James Wilson.

Funded in part by NSF: DMS-1620454.

$$\textsf{TensorSpace}$$ refers to a collection of Magma packages:

• TensorSpace: data structures for tensors,
• Sylver: algorithms for simultaneous Sylvester equations,
• Densor: algorithms to construct linear closures of tensor spaces.

# Constructing Tensors in $$\textsf{TensorSpace}$$

## Flexible data structure for tensors

Goal: Want flexible definition for tensors.

Questions: Is a tensor more than just...

1. a high-dimensional grid?
2. a multilinear map?

How do the questions we want to answer motivate a data structure?

## Examples require context

1. $$b$$ bilinear form of $$V$$. Interpret:

$$\langle b | : V \times V \rightarrowtail K$$

3. $$V$$ an $$A$$-module. Interpret:

$$\langle \mu | : V \times A \rightarrowtail V$$

2. $$\psi$$ an entangled system of qubits. Interpret:

$$\langle \psi | : \mathbb{C}^2\times \cdots \times \mathbb{C}^2 \rightarrowtail \mathbb{C}$$

## Tensors in $$\textsf{TensorSpace}$$ are framed

We require tensors to have three things:

1. frame:  $$(U_0, \ldots, U_n)$$,

2. function:  $$U_n\times \cdots \times U_1\rightarrowtail U_0$$,

3. category*.

*We do not specify now, but we will come back to this.

A "default" category is assigned when not given.

## Constructing a tensor from scratch

We construct the tensor

\begin{aligned} t : \text{M}_2(\mathbb{Q}) \times \text{M}_2(\mathbb{Q}) &\rightarrowtail \text{M}_2(\mathbb{Q}),\\ (A, B) &\mapsto AB \end{aligned}

We have two structures we define:

1. frame:  $$(\text{M}_2(\mathbb{Q}),\; \text{M}_2(\mathbb{Q}),\; \text{M}_2(\mathbb{Q}))$$,
2. function:  $$(A, B)\mapsto AB$$.
Q := Rationals();
A := MatrixAlgebra(Q, 2);              // 2 x 2 matrices over Q
frame := [A, A, A];
frame;
[
Full Matrix Algebra of degree 2 over Rational Field,
Full Matrix Algebra of degree 2 over Rational Field,
Full Matrix Algebra of degree 2 over Rational Field
]

First we define our frame.

Require a function to map a tuple of matrices to their product.

mult := func< x | x[1]*x[2] >;         // <A, B> |-> A*B
t := Tensor(frame, mult);
t;
Tensor of valence 3, U2 x U1 >-> U0
U2 : Full Vector space of degree 4 over Rational Field
U1 : Full Vector space of degree 4 over Rational Field
U0 : Full Vector space of degree 4 over Rational Field

## Tensors as multilinear functions

Let's do a brief sanity check, and verify $$t$$ does what we expect.

X := A![2, 2/5, 0, -1];               // define matrices
Y := A![0, -3, 5, 7];                 // from sequences
X, Y;
[  2 2/5]
[  0  -1]

[ 0 -3]
[ 5  7]
X*Y, <X, Y> @ t;
[    2 -16/5]
[   -5    -7]

(    2 -16/5    -5    -7)

Recall, the print statement for $$t$$: no matrix algebras

t;
Tensor of valence 3, U2 x U1 >-> U0
U2 : Full Vector space of degree 4 over Rational Field
U1 : Full Vector space of degree 4 over Rational Field
U0 : Full Vector space of degree 4 over Rational Field

It states vector spaces, but we gave it matrices: $$X$$ and $$Y$$.

The frame gives context, so that we leave the typing to the machines.

<X, [0, -3, 5, 7]> @ t;
<0, [1, 1, 1, 1]> @ t;
(    2 -16/5    -5    -7)
(0 0 0 0)

## Structure constants

We define a tensor of the form

$$\langle t| : K^{d_n} \times \cdots \times K^{d_1} \rightarrowtail K^{d_0}$$

by $$d_0\cdots d_n$$ elements in $$K$$: a $$(d_n\times \cdots \times d_0)$$-grid

$$[t_{i_n\cdots i_1}^{i_0}].$$

Notational challenge to do this for general $$n$$. We do $$n=2$$:

$$(u, v)\mapsto \left( \sum_{i=1}^{d_2}\sum_{j=1}^{d_1} u_{i}t_{ij}^{1} v_{j}, \ldots, \sum_{i=1}^{d_2}\sum_{j=1}^{d_1} u_{i}t_{ij}^{d_0} v_{j}\right)\in K^{d_0}.$$

Let's construct an example.

d := [3, 2, 5, 4];                 // dimensions of frame
grid := [1..3*2*5*4];              // ints 1, ..., 120
t := Tensor(Q, d, grid);
t;
Tensor of valence 4, U3 x U2 x U1 >-> U0
U3 : Full Vector space of degree 3 over Rational Field
U2 : Full Vector space of degree 2 over Rational Field
U1 : Full Vector space of degree 5 over Rational Field
U0 : Full Vector space of degree 4 over Rational Field

We verify that the structure constants of $$t$$ are what we expect.

StructureConstants(t)[1..10];       // Only first 10 entries
[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]

Notice: structure constants are a flat sequence.

The grid model provides easier understanding of data manipulation.

We describe 2 operations that potentially change the frame.

## Slicing

Recall, our current tensor has the frame:

$$\langle t | : \mathbb{Q}^3 \times \mathbb{Q}^2 \times \mathbb{Q}^5 \rightarrow \mathbb{Q}^4$$

We grab any entry of the grid $$[t_{i_n\cdots i_1}^{i_0}]$$ and consider the corresponding sub-grid.

This can be interpreted as a tensor itself.

We query for the $$t_{3,2,5}^{4}$$ constant:

ind := [{3}, {2}, {5}, {4}];                  // last entry
Slice(t, ind);
[ 120 ]

We can query any sequence of indices.

indices := [{1, 3}, {2}, {1, 3, 4}, {4}];     // 2*1*3*1 entries
Slice(t, indices);
[ 24, 32, 36, 104, 112, 116 ]

The above example can be interpreted as a new tensor

$$\langle s | : \mathbb{Q}^2 \times \mathbb{Q} \times \mathbb{Q}^3 \rightarrowtail \mathbb{Q}.$$

So it might be helpful to interpret the output as a matrix.

SliceAsMatrices(t, indices, 3, 1);
[
[ 24  32  36]
[104 112 116]
]

## Shuffling

We apply permutations to the frame. Suppose we are framed by vector spaces

$$\langle t | : U_n \times \cdots \times U_1\rightarrowtail U_0.$$

For a permutation $$\sigma$$ of $$\{0, \ldots, n\}$$, a $$\sigma$$-shuffle of $$t$$ is a tensor

$$\langle \sigma(t) | : U_{\sigma(n)} \times \cdots \times U_{\sigma(1)} \rightarrowtail U_{\sigma(0)}.$$

Note: shuffling requires dual-spaces. $$\textsf{TensorSpace}$$ handles this.

We continue with current example:

$$\langle t | : \mathbb{Q}^3 \times \mathbb{Q}^2 \times \mathbb{Q}^5 \rightarrowtail \mathbb{Q}^4.$$

We apply the permutation $$\sigma = (0, 2, 3, 1)$$.

sigma := [2, 0, 3, 1];                // cycle: (0, 2, 3, 1)
s := Shuffle(t, sigma);
s;
Tensor of valence 4, U3 x U2 x U1 >-> U0
U3 : Full Vector space of degree 5 over Rational Field
U2 : Full Vector space of degree 3 over Rational Field
U1 : Full Vector space of degree 4 over Rational Field
U0 : Full Vector space of degree 2 over Rational Field

We can take a look at the structure constants.

StructureConstants(s)[1..10];        // first 10 entries
[ 1, 3, 5, 7, 25, 27, 29, 31, 49, 51 ]

Recall: the structure constants were $$[1, \ldots, 120]$$.

# Algebras associated to tensors

These definitions work in greater generality, but fix a $$K$$-bilinear map

$$\langle t | : U\times V\rightarrowtail W.$$

Set $$\Omega = \text{End}(U)\times \text{End}(V)\times \text{End}(W)$$. Our algebras are subsets of $$\Omega$$.

The first algebra we define is the centroid, and the second is the derivation algebra.

\begin{aligned} \mathcal{C}_t &= \{ (X, Y, Z) \in\Omega \mid \forall u, v,\; \langle t | Xu, v\rangle = \langle t | u, Yv\rangle = Z\langle t | u, v\rangle \} \\ \mathcal{D}_t &= \{ (X, Y, Z) \in \Omega \mid \forall u, v,\; \langle t | Xu, v\rangle + \langle t | u, Yv\rangle = Z\langle t | u, v\rangle \} \end{aligned}

Fact. $$\mathcal{C}_t$$ is a $$K$$-algebra with $$1$$, and $$\mathcal{D}_t$$ is a Lie algebra.

# Example from quantum info. theory

## The GHZ and W states

The Greenberger-Horne-Zeilinger (GHZ) state is

\begin{aligned} \sqrt{2} \cdot GHZ &= |000\rangle + |111\rangle \\ &\equiv (e_1\otimes e_1\otimes e_1) + (e_2\otimes e_2\otimes e_2) \\ &\equiv \begin{bmatrix} (1,0) & (0,0) \\ (0,0) & (0,1) \end{bmatrix}. \end{aligned}

The W state is

\begin{aligned} \sqrt{3} \cdot W &= |001\rangle + |010\rangle + |100\rangle \\ &\equiv (e_1\otimes e_1\otimes e_2) + (e_1\otimes e_2\otimes e_1) + (e_2\otimes e_1\otimes e_1) \\ &\equiv \begin{bmatrix} (0,1) & (1,0) \\ (1,0) & (0,0) \end{bmatrix}. \end{aligned}

Both states are $$(2\times 2\times 2)$$-grids. We interpret these states as

\begin{aligned} \langle GHZ| &: \mathbb{C}^2\times \mathbb{C}^2 \rightarrowtail \mathbb{C}^2 , \\ \langle W| &: \mathbb{C}^2\times \mathbb{C}^2 \rightarrowtail \mathbb{C}^2 \end{aligned}

(Typically interpreted as $$\mathbb{C}^2 \times \mathbb{C}^2 \times \mathbb{C}^2 \rightarrowtail \mathbb{C}$$.)

Question: $$GHZ$$ and $$W$$ are contained in the same space. Maybe there is a change of basis (of each $$\mathbb{C}^2$$) that makes GHZ and W equivalent?

(There isn't!)

Let's construct the tensors (over $$\mathbb{Q}$$).

Q := Rationals();
d := [2, 2, 2];                                 // dimensions of frame
GHZ := Tensor(Q, d, [1, 0, 0, 0, 0, 0, 0, 1]);
W := Tensor(Q, d, [0, 1, 1, 0, 1, 0, 0, 0]);

Quick sanity check.

AsMatrices(GHZ, 2, 1);
[
[1 0]
[0 0],

[0 0]
[0 1]
]

Now we compute their centroids.

C_GHZ := Centroid(GHZ);
C_W := Centroid(W);
C_GHZ, C_W;
Matrix Algebra of degree 6 with 2 generators over Rational Field
Matrix Algebra of degree 6 with 2 generators over Rational Field
[
[1 0 0 0 0 0]
[0 1 0 0 0 0]
[0 0 1 0 0 0]
[0 0 0 1 0 0]
[0 0 0 0 1 0]
[0 0 0 0 0 1],

[0 1 0 0 0 0]
[0 0 0 0 0 0]
[0 0 0 1 0 0]
[0 0 0 0 0 0]
[0 0 0 0 0 0]
[0 0 0 0 1 0]
]
Basis(C_W);

Let's play around with $$\mathcal{C}_W$$.

M := C_W.1 + C_W.2;                   // an element of the ring
M;
[1 1 0 0 0 0]
[0 1 0 0 0 0]
[0 0 1 1 0 0]
[0 0 0 1 0 0]
[0 0 0 0 1 0]
[0 0 0 0 1 1]

We can get the individual blocks.

X := M @ Induce(C_W, 2);                     // build map
Y := M @ Induce(C_W, 1);                     // for each
Z := M @ Induce(C_W, 0);                     // coordinate
Z;

[1 0]
[1 1]
Verify the centroid satisfies the two equations:
$$\langle W | Xu, v \rangle = \langle W | u, Yv \rangle = Z \langle W | u, v \rangle.$$

Build two vectors from our $$2$$-dimensional vector space

V := VectorSpace(Q, 2);
u := V![-1, 22/7];
v := V![1/2, 4/27];
u, v;
(  -1 22/7)
( 1/2 4/27)
<u*X, v> @ W eq <u, v*Y> @ W;            // 1st eqn
<u, v*Y> @ W eq (<u, v> @ W)*Z;          // 2nd eqn
true
true

We distinguish $$W$$ and $$GHZ$$ by the nilpotent elements of their centroids.

J_GHZ := JacobsonRadical(C_GHZ);
Dimension(J_GHZ), Dimension(J_W);

Because the dimension of $$J(GHZ)$$ is different from $$J(W)$$ these states are inequivalent.

With a little work, one shows that

\begin{aligned} \mathcal{C}_{GHZ} &\cong \mathbb{C}^2, & \mathcal{C}_{W} &\cong \mathbb{C}[x]/\left(x^2\right). \end{aligned}

0 1

Oh, and just by wiggling your pinky...

The derivation algebras bring their differences to the surface.

D_GHZ := DerivationAlgebra(GHZ);
D_W := DerivationAlgebra(W);
Dimension(D_GHZ), Dimension(D_W);

$$\dim_{\mathbb{C}}(\mathcal{D}_{GHZ}) = 4 \ne 5 = \dim_{\mathbb{C}}(\mathcal{D}_{W}).$$

4 5

Because the dimensions of the derivation algebras are different, $$GHZ$$ and $$W$$ states are inequivalent

# What is really happening?

The equation $$\langle t | Xu, v\rangle = \langle t | u, Yv\rangle$$ becomes

$$XT^{(k)}_{**} - T^{(k)}_{**}Y = 0,$$

where $$T^{(k)}_{**}$$ is the $$k$$th slice in the $$0$$-coordinate as a matrix:

$$T_{**}^{(k)} = \begin{bmatrix} t_{11}^{k} & t_{12}^{k} & \cdots \\ t_{21}^{k} & t_{22}^{k} & \cdots \\ \vdots & \vdots & \ddots \end{bmatrix}$$

The equation $$\langle t | Xu, v\rangle = Z\langle t | u, v\rangle$$ becomes

$$XT^{*}_{*j} - T^{*}_{*j}Z = 0,$$

where $$T^{*}_{*j}$$ is the $$j$$th slice in the $$1$$-coordinate as a matrix:

$$T_{*j}^{*} = \begin{bmatrix} t_{1j}^{1} & t_{1j}^{2} & \cdots \\ t_{2j}^{1} & t_{2j}^{2} & \cdots \\ \vdots & \vdots & \ddots \end{bmatrix}$$

## Solving simultaneous Sylvester systems: $$\textsf{Sylver}$$

These algorithms are part of a general family of Sylvester-like equations:

$$XA + BY = C.$$

We need to solve a simultaneous system of Sylvester-like equations.

Naively, this requires $$O(d^7)$$ field operations for a $$(d\times d\times d)$$-grid.

In general, construct a basis of the kernel of a

$$(d_0\cdots d_n) \times (d_0^2 +\cdots + d_n^2)$$-matrix.

Theorem (M.-Wilson 2018).
There is an algorithm to solve simultaneous Sylvester equations of a nondegenerate $$(d\times d\times d)$$-grid using $$O(d^4)$$ field operations.

# Tensor spaces & tensor categories

Tensor spaces are the spaces containing tensors. Therefore, defined similarly to tensors.

A tensor space $$T$$ is a $$K$$-vector space with

1. a frame $$(U_0, \ldots, U_n)$$,

2. an interpreter map $$\langle \cdot | : T\rightarrow \left( U_n\times \cdots \times U_1\rightarrowtail U_0\right)$$,

3. a category,

4. a basis.

The map $$\langle \cdot |$$ gives every $$t\in T$$ a multilinear map interpretation

$$\langle t | : U_n\times \cdots \times U_1\rightarrowtail U_0.$$

## Examples that hint towards tensor categories

Equivalence up to a change of bases. The states $$GHZ$$ and $$W$$ are inequivalent.

For all $$X, Y, Z\in \text{GL}_2(\mathbb{C})$$, there exists $$u, v\in\mathbb{C}^2$$ such that

\begin{aligned} Z\langle GHZ | Xu, Yv\rangle \ne \langle W | u, v \rangle. \end{aligned}

Equivalence of algebras: an isomorphism.

There exists $$\varphi\in\text{GL}(A)$$ such that for all $$a,b\in A$$

\begin{aligned} \varphi(ab) &= \varphi(a)\varphi(b). \end{aligned}

Adjoint operators of a bilinear form $$\langle\,,\,\rangle$$.

For $$u,v\in V$$, and $$A\in \text{GL}(V)$$,

\begin{aligned} \langle Au, v \rangle &= \langle u, A^*v\rangle. \end{aligned}

## A data structure for tensor categories

For us a tensor category is

1. a function $$\textsf{A}: [n] \rightarrow \{-1, 0, 1\}$$,

2. a partition of $$[n]$$.

The function $$\textsf{A}$$ tells us which way the arrows go: $$\downarrow$$, $$\parallel$$, or $$\uparrow$$.

The partition tells us which coordinates are treated as equal.

## Examples in $$\textsf{TensorSpace}$$

Let's look at a tensor created earlier: the multiplication in an algebra

$$\langle t | : \text{M}_2(\mathbb{Q}) \times \text{M}_2(\mathbb{Q}) \rightarrowtail \text{M}_2(\mathbb{Q}).$$

A := MatrixAlgebra(Q, 2);
t := Tensor(A);
TensorCategory(t);
Tensor category of valence 3 (->,->,->) ({ 0, 1, 2 })
• all arrows are $$\downarrow$$, and
• an operator acts the same way in all three coordinates.

Let's look at an example as a high-dimensional grid.

t := Tensor(GF(2), [2, 3, 2, 3], [1..36]);
t;
Tensor of valence 4, U3 x U2 x U1 >-> U0
U3 : Full Vector space of degree 2 over GF(2)
U2 : Full Vector space of degree 3 over GF(2)
U1 : Full Vector space of degree 2 over GF(2)
U0 : Full Vector space of degree 3 over GF(2)

The default tensor category is given.

TensorCategory(t);          // default category
Tensor category of valence 4 (->,->,->,->) ({ 1 },{ 2 },{ 0 },{ 3 })

# Relating tensor spaces & operators

## Densor subspaces

The $$GHZ$$ and $$W$$-state are not equivalent because derivations were not isomorphic.

For these kinds of questions, we consider spaces of all tensors sharing common operators.

Recall the derivation algebra of $$t\in T$$,

\begin{aligned} \mathcal{D}_t &= \{ (X, Y, Z) \in \Omega \mid \forall u, v,\; \langle t | Xu, v\rangle + \langle t | u, Yv\rangle = Z\langle t | u, v\rangle\} \end{aligned}

For $$t\in T$$, the densor subsapce of $$t$$ is the tensor subspace

$$\textsf{Dens}(t) = \{ s\in T \mid \mathcal{D}_t \subseteq \mathcal{D}_s \}.$$

### Example from Lie algebras

Lie algebra of $$3\times 3$$ trace zero matrices over a field $$K$$, denoted $$\mathfrak{sl}_3$$.

The tensor given by the matrix commutator is the following.

K := GF(7);
sl3 := LieAlgebra("A2", K);
t := Tensor(sl3);
t; 
Tensor of valence 3, U2 x U1 >-> U0
U2 : Full Vector space of degree 8 over GF(7)
U1 : Full Vector space of degree 8 over GF(7)
U0 : Full Vector space of degree 8 over GF(7)
Few tensors contain an $$8$$-dimensional Lie algebra.

Note the tensor category.

TensorCategory(t);
Tensor category of valence 3 (->,->,->) ({ 0, 1, 2 })

Each operator acts the same on each of the three coordinates.

From standard results, $$\mathcal{D}_t \cong \mathfrak{sl}_3$$.

D := DerivationAlgebra(t);
Dimension(D);
8

The densor subspace of $$t$$ is computed using $$\textsf{Densor}$$.

Dens := UniversalDensorSubspace(t);
Dens;
Tensor space of dimension 2 over GF(7) with valence 3
U2 : Full Vector space of degree 8 over GF(7)
U1 : Full Vector space of degree 8 over GF(7)
U0 : Full Vector space of degree 8 over GF(7)

The dimension of the densor subspace is tiny compared to the ambient space.

T := Parent(t);                      // full tensor space of t
Dimension(T), Dimension(Dens);
512 2

What just happened?

## Example: matrix multiplication

The tensor space $$T$$ framed by $$(K^{12}, K^8, K^6)$$ contains

$$\langle t | : \text{M}_{3\times 4} \times \text{M}_{4\times 2} \rightarrowtail \text{M}_{3\times 2}.$$
t := MatrixTensor(Q, [3, 4, 2]);
t;
Tensor of valence 3, U2 x U1 >-> U0
U2 : Full Vector space of degree 12 over Rational Field
U1 : Full Vector space of degree 8 over Rational Field
U0 : Full Vector space of degree 6 over Rational Field

The derivation algebra of $$t$$ is large because matrix multiplication is distributive.

Recall the derivation equation:

$$\langle t | Xu, v\rangle + \langle t | u, Yv \rangle = Z\langle t | u, v \rangle$$

If $$X\in \text{M}_{3\times 3}$$, then we re-interpret

$$(XU)*V + U*(0V) = X(U*V).$$

Similar equations hold for $$Y\in\text{M}_{4\times 4}$$ and $$Z\in\text{M}_{2\times 2}$$.

Therefore, $$\text{M}_{3\times 3} \cup \text{M}_{4\times 4} \cup \text{M}_{2\times 2} \subseteq \mathcal{D}_t$$.

We denote the algebra $$\text{M}_{n\times n}$$ with the matrix commutator by $$\mathfrak{gl}_n$$.

Fact. $$\mathcal{D}_t \cong \left(\mathfrak{gl}_3\oplus \mathfrak{gl}_4 \oplus\mathfrak{gl}_2\right)/K$$.

D := DerivationAlgebra(t);
Dimension(D) eq (3^2 + 4^2 + 2^2 - 1);           // compare dimensions

We can easily verify that the dimensions match.

true
Dens := UniversalDensorSubspace(t);
Dens;
Tensor space of dimension 1 over Rational Field with valence 3
U2 : Full Vector space of degree 12 over Rational Field
U1 : Full Vector space of degree 8 over Rational Field
U0 : Full Vector space of degree 6 over Rational Field

With derivaton algebra so large, the densor subspace must be small.

Therefore, in a $$576$$-dimensional space, $$T$$, matrix multiplication is set apart from the rest.

As a consequence, the symmetries of $$t$$ are completely determined by its derivations.

# How are densor spaces computed?

## A dual solve to derivations

Recall, we consider Sylvester-like systems of the form:

$$XT_{**}^{(k)} + T_{**}^{(k)}Y = 0$$

The same system is used to get the densor subspace$-$the roles are swapped.

We are given the derivation algebra, so we slice a symbolic tensor.

Simultaneously solve a "dual" version of the Sylvester-like equations.

Theorem. (Brooksbank-M.-Wilson)

There exists an algorithm to simultaneously solve a "dual" Sylvester-like system using $$O(d^9m^2)$$ field operations, given $$m$$ operators acting on a $$(d\times d\times d)$$-grid.

## Summary

• Still uncovering new algebraic data from tensors, and this is a snapshot of $$\textsf{TensorSpace}$$.

• Context through the frame and tensor category give flexibility in the data structures.

• These algebras provide general tools to different kinds of applications.