Your image is probably not a tensor;

neither is your neural net!

(...you have to put the work in to it.)

2021 James B. Wilson, Colorado State University

Here is a table.  Is it a matrix?

Is it meaningful too...

  • ... reorder along an row/column?
  • ...rescale along a row/column?
  • ...add one row to another, or do so with columns?

 

Bus tables are not matrices

"matrix" elevates data to a math;

thus, it must comply with rules.

 

A table which cannot be explored linearly is not a matrix.

Is a Grade Book a matrix?

Rows?

  • You could reorder rows.
  • No value in rescaling or combinations.

Columns?

  • Reorder,
  • Rescale,
  • Combine.

Are graphs matrices?

Adjacency: \(A_{ij}\) if \((i,j)\) is an edge.

Node-arc: rows=nodes, columns=edges,

+1 if start of edge, -1 if end of edge.

Are graphs matrices?

No, you cannot do linear algebra on the rows and columns without destroying the graph.

...and yet

  • If \(A\) is the matrix then \(A^n\) records the paths of length \(n\)
  • If \(A\) is a node-arc matrix of a graph, then row/column elimination is essential for linear programming, max flows, min-cuts, etc..
  • Spectral graph theory is another deep subject.

 

These are linear connection; but require nuance.

Is an image a matrix (2-tensor)?

An image after some operations has nothing meaningful to do with the original information.

  • It is a rows and columns with scalar entries
  • Folks do linear algebra on images (I was told in class years ago)

...and yet...

  • If \(M\) is an image matrix then the eigenvectors of \(M^* M\) tell us close approximations to the image.  (Singular value decomposition more generally).
  • Shifting \(M\) to a phase space with a cosine transform lets us compress the image close to faithfully.

 

Once more, linear algebra is lurking, but you have to work for it.

Is a video a 3-tensor?

Neither time nor rows/columns are meaningfully linear on video data.

  • It is a 3-axis grid of scalars.
  • Folks do linear algebra with videos, but its not directly on the axes.

Is a neural network a tensor?

Certainly not across the non-linear part.

Linear algebra is scalpel not a hammer.

 

If you attack a grid as a tensor,

you better show reason to expect the results to be meaningful.

Yes, but...

(the rebuttal)

If your advisor, boss, or mentor calls every array a "tensor" then by all means go with it.

 

(You might just call actual tensors "formal tensors")