This thesis addresses this question:

This thesis addresses this question:

The end of Moore's law?

The QML toolkit

• Query access to matrices

• Quantum linear algebra

• Distance estimations

• Singular Value Estimation

• Tomography (of pure states)

• Hamiltonian simulation

• Amplitude estimation and amplification

• Singular value transformations

• Polynomial approximations

• ...

Quantum access to a matrix

Quantum query:

$|i\rangle |0 \rangle \mapsto |i\rangle |x_i\rangle$

Where  $|x_i\rangle=\frac{1}{\|x_i\|_2} x_i = \frac{1}{\|x_i\|_2} \sum_i (x_i)_j |j\rangle$

$X \in \mathbb{R}^{\textcolor{red}{n \times d}} = [x_1, \dots, x_n]^T$    $x_i \in \mathbb{R}^d$

• Classical preprocessing time: $O(\textcolor{red}{nd} \log nd)$

• Classical space: $O\left( \textcolor{red}{nd} \log nd \right)$

• Query time: $O(\textcolor{red}{\log nd})$

• Quantum space: $O \left(\textcolor{red}{ \log nd }\right)$

Where  $|x_i\rangle=\frac{1}{\|x_i\|_2} x_i = \frac{1}{\|x_i\|_2} \sum_i (x_i)_j |j\rangle$

Quantum query:

$\frac{1}{\sqrt{n}}\sum_i |i\rangle |0 \rangle \mapsto \frac{1}{\sqrt{n}}\sum_i |i\rangle |x_i\rangle$

$X \in \mathbb{R}^{\textcolor{red}{n \times d}} = [x_1, \dots, x_n]^T$    $x_i \in \mathbb{R}^d$

Quantum access to a matrix

Quantum linear systems of equations

Given:

• quantum sparse access $A \in \R^{n \times n}$,

• and a vector $x \in \R^n$

The HHL algorithm produces a state $|z\rangle$ such that

$\kappa(A) = \frac{\sigma_1(A)}{\sigma_n(A)}$ and $s = \text{row's sparsity}$

Harrow, Aram, Avinatan Hassidim, Seth Lloyd - Physical review letters - 2009 

Quantum linear systems of equations

The HHL algorithm produces a state $|z\rangle$ such that

$\kappa(A) = \frac{\sigma_1(A)}{\sigma_n(A)}$

Harrow, Aram, Avinatan Hassidim, Seth Lloyd - Physical review letters - 2009 

Given:

• quantum access $A \in \R^{n \times n}$,

• and a vector $x \in \R^n$

Quantum singular value transformations

Given matrix $A \in \R^{n \times m}$,  and a vector $x \in \R^m$

It is possible to produce a state $|z\rangle$ s.t.

Iordanis Kerenidis, Anupam Prakash - 8th Innovations in Theoretical Computer Science Conference - 2017.

András Gilyén, et al.  - Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing -  2019.

Guang Hao Low, Isaac L. Chuang -  Physical review letters  - 2017.

$\mu(A) = \min\left(\|A\|_F, \sqrt{\max_{i \in [n]} \|a_i\|_{2p}^{2p} \max_{i \in [d]} \|a_{*i}\|_{2(1-p)}^{(1-p)} }\right)$

Distance estimation subroutines

• Euclidean distance [QMEANS]

• Quadratic forms [Thesis]

• Distance induced by $A$ [QEM]

$|i,j\rangle \mapsto |i,j ,\|x_i- x_j\|_2\rangle$

$|i,j\rangle \mapsto |i,j ,d_A(x_i,  x_j) \rangle$

\(|i,j\rangle \mapsto |i,j ,x_i^TA^\textcolor{orange}{-1}x_j \rangle  \)

$\widetilde{O}(\frac{1 }{\epsilon} )$

$\widetilde{O}(\frac{\mu(A)}{\epsilon} )$

$\widetilde{O}(\frac{\mu(A)\textcolor{orange}{\kappa(A)}}{\epsilon} )$

$X \in \mathbb{R}^{n \times d} = [x_1, \dots, x_n]^T$    $x_i \in \mathbb{R}^d$

We can produce an estimate $\overline{x}$ of $|x\rangle$ such that $\|\overline{x}\| = 1$ and

Tomography of pure states

 Iordanis Kerenidis, Anupam Prakash -  ACM Transactions on Quantum Computing, 2020.

Iordanis Kerenidis, Anupam Prakash, Jonas Landman -  International Conference on Learning Representations, 2019.

samples

samples

SVE of product of matrices

Theorem: Assume to have quantum access to $A, B \in \mathbb{R}^{n \times n}$.

There is an algorithm that performs the mapping

$\sum_i \alpha_i |i\rangle \mapsto \sum_i \alpha_i|i, \sigma_i \rangle$

where $\sigma_i$ is the i-th singular value of $AB$ in time

$O\left(\frac{(\kappa(A)+\kappa(B))( \mu(A)+\mu(B))}{\epsilon}\right)$

[QSFA]

Supervised

Learning

• $X \in \mathbb{R}^{n \times d}$ dataset

• $L \in [K]^{n}$ labels

(classification)

Quantum supervised learning

• Quantum slow feature analysis for dimensionality reduction

• Quantum Frobenius distance classifier: a simple NISQ classifier

• Simulation on MNIST dataset of handwritten digits

• Extension to other generalized eigenvalue problems in ML

[QSFA, Thesis]

https://cmp.felk.cvut.cz/cmp/software/stprtool

Slow Feature Analysis

$Y \in \mathbb{R}^{n \times \textcolor{orange}{K}}$

$y_i =\left[ w_1^Tx_i, \dots, w_K^Tx_i\right]$

$L \in [K]^{n}$ (labels)

Slow Feature Analysis

$Y \in \mathbb{R}^{n \times \textcolor{orange}{K}}$

$y_i =\left[ w_1^Tx_i, \dots, w_K^Tx_i\right]$

$L \in [K]^{n}$ (labels)

Quantum Slow Feature Analysis

Theorem:

• Quantum access to $X$, derivative matrix $\dot{X}$

• Let $\epsilon, \theta, \delta, \eta >0$

     There are quantum algorithms to get:

• Map dataset in the slow feature space $|\overline{Y}\rangle$  in time: $$\tilde{O}\left(  \frac{ ( \kappa(X) + \kappa(\dot{X})) ( \mu({X})+ \mu(\dot{X}) ) }{\delta\theta}  \gamma_{K-1} \right)$$

• Find $\{ w_j\}_{j=0}^{K}$ in time:    $$O\left(d^{1.5}\sqrt{K}\frac{\kappa(X)\kappa(\dot X X)(\mu(X) + \mu(\dot{X}))}{\epsilon^2}\right)$$

[QSFA]

QFDC: classification in slow feature space

[QSFA]

We get high accuracy with a fast quantum classifier

Classification of handwritten digits of MNIST dataset

Ex: polynomial expansion

of degree 2:

$[x_1, x_2, x_3] \mapsto [x_1^2, x_1x_2 \dots x_3^3 ]$

We get high accuracy with a fast quantum classifier

Malware detection via DGA classification

[Thesis]

• Using SFA in a classification problem improves its accuracy

• QSFA can process old datasets in new ways!

SFA as instance of a more general problem

The GEP (Genrealized Eigenvalue Problem) is defined as:

• ICA Independent Component Analysis

• G-IBM Gaussian Information Bottleneck Method

• CCA Canonical Correlation Analysis

• SC (some) Spectral Clustering  [1]

• PLS Partial Least Squares

• LE Laplacian Eigenmaps

• FLD Fisheral Linear Discriminant

• SFA Slow Feature Analysis

• KPCA Kernel Principal Component Analysis

$AW = BW\Lambda$

[1] Iordanis Kerenidis, Jonas Landman - Quantum spectral clustering. arXiv2007.00280. (2020)

Quantum supervised learning

• Quantum slow feature analysis for dimensionality reduction

• Simulation on MNIST dataset of handwritten digits

• Quantum Frobenius distance classifier: a simple NISQ classifier

• Extension to other generalized eigenvalue problems in ML

[QSFA, Thesis]

Unsupervised

Learning

• $X \in \mathbb{R}^{n \times d}$

(clustering)

Quantum unsupervised

learning

• q-means for clustering (quantum version of k-means)

• Quantum Expectation-Maximization

• Simulation on VoxForge dataset for speaker recognition

The k-means algorithm:

$t \leftarrow 0$

Step 1:

• Compute distance for all points $x_i$ and centroid $\mu_j^{t},$  $$d(x_i, \mu_j^{t})$$

•  Assign points to closest cluster: $$l(x_i) = \argmin_{c \in [K]}{ d(x_i, \mu_j^{t})}$$

 Step 2:

• Compute the barycenter:  $$\mu_j^{t+1} = \frac{1}{|C_j|}\sum_{i \in C_j}^{} x_i$$

t $\leftarrow$t+1

q-means

$t \leftarrow 0$

Step 1:

• Compute distance for all points $v_i$ and centroid $\mu_j^{t},$  $$|i,j\rangle \mapsto  |i,j,d(v_i, \mu_j^{t})\rangle$$

•  Generate characteristic vector of a cluster: $$|\chi_j \rangle = \frac{1}{|C_j|}\sum_{i \in C_j} |i \rangle$$

Step 2:

• Use quantum linear algebra to build $$|\mu_j^{t+1}\rangle = \frac{1}{|C_j|}\sum_{i \in C_j}^{} |v_i\rangle$$

• Perform tomograph on $|\mu_j^{t+1}\rangle$

• Build quantum access to $\mu_j$.

$t \leftarrow t+1$

q-means

$t \leftarrow 0$

Expectation:

• Compute distance for all points $v_i$ and centroid $\mu_j^{t},$  $$|i,j\rangle \mapsto  |i,j,d(v_i, \mu_j^{t})\rangle$$

•  Generate characteristic vector of a cluster: $$|\chi_j \rangle = \frac{1}{|C_j|}\sum_{i \in C_j} |i \rangle$$

Maximization:

• Use quantum linear algebra to build $$|\mu_j^{t+1}\rangle = \frac{1}{|C_j|}\sum_{i \in C_j}^{} |v_i\rangle$$

• Perform tomograph on $|\mu_j^{t+1}\rangle$

• Build quantum access to $\mu_j$.

$t \leftarrow t+1$

Gaussian mixture models

$k$ labels

Multinomial distribution

$[\theta, \mu_1, \dots, \mu_k, \Sigma_1, \dots, \Sigma_k ]$

$\gamma^* =   \text{argmax} \prod_i \sum_{j \in [k]} \theta_j p(x_i|\mu_j,\Sigma_j)$

Gaussian distribution

$_\gamma$

•     $\|\theta-\overline{\theta}\|<\delta_\theta$

• $\|\mu_j - \overline{\mu_j}\| < \delta_\mu$

• $\|\Sigma_j - \overline{\Sigma_j}\| < \delta_\mu\sqrt{\eta}$

Maximum Likelihood Estimation

Expectation-Maximization

                      Repeat

                      $t = 0$

• Expectation   $$r_{ij}^t \leftarrow \frac{\theta^t_j N(v_i; \mu^t_j, \Sigma^t_j )}{\sum_{l=1}^k \theta^t_l N(v_i; \mu^t_l, \Sigma^t_l)}$$

                         $t \leftarrow t+1$

                      Until  $| \ell(\gamma^{t-1};V) - \ell(\gamma^t;V) | < \tau$

• Maximization

• Update the parameters $\theta, \mu, \Sigma$ using the responsibilities $r_{ij}$

Quantum Expectation-Maximization for

Gaussian mixture models

           Theorem: Given quantum access to a matrix $X \in \mathbb{R}^{n \times d}$ there is a quantum EM algorithm that fits a GMM in time:

[QEM]

Classical: $O(d^2 k n)$

We get high accuracy with a fast quantum classifier

• Classical ML accuracy: 169/170

• Quantum ML accuracy: 167/170

• Max element of $\Sigma_j^{-1}$ set to $5$  via $\kappa = \frac{1}{\lambda_\tau}$

Speaker recognition problem on VoxForge dataset

Quantum unsupervised

learning

• q-means for clustering (quantum version of k-means)

• Quantum Expectation-Maximization

• Simulation on VoxForge dataset for speaker recognition

Quantum Spectral Sums

$S_f(A) = \sum_i^n f(\lambda_i)$

$A \in \mathbb{R}^{n \times n}$ SPD

$f : \mathbb{R} \mapsto \mathbb{R}$

Quantum algorithms for log-determinant

$S_{\log(x)}(A) =\log\det(A) = \sum_i^n \log(\lambda_i)$

[QSS]

Tyler's M-estimator

• Data from sub-Gaussian distributions.

• Robust to outliers

• Valid for data $X^{n \times d}$ with $n,d \mapsto \infty$

[Thesis, AQSS]

In many cases $C = X^TX$ is not a "good" sample covariance matrix

Tyler's M-estimator

[Thesis, AQSS]

Runtime:

$$\tilde O\left(\textcolor{red}{d^2}\frac{\mu(X)\kappa(\Sigma_k)\mu(\Sigma_k)}{\epsilon^3}\gamma\right)$$

$\Gamma_{k+1} = \sum_{i=1}^n \frac{x_ix_i^T}{ x_i^T \Gamma_k^{-1}x_i}/ Tr[\sum_{i=1}^n \frac{x_ix_i^T}{x_i^T \Gamma_k^{-1}x_i}]$

Stopping condition: log-likelihood with a log-determinant

Classical:

$O\left( d^2n \right)$

Other spectral sums and applications (not in thesis)

• Schatten p-norm             $O(2^{p/2}\mu(A)(p+\kappa(A))\sqrt{n}/\epsilon)$

• Von Neumann entropy       $O(\mu(A)\kappa(A)n /\epsilon )$

• Trace of Inverse              $O(\mu^2 \kappa(A)^2/\epsilon)$

[QSS]

Thanks

Elham Kashefi
Iordanis Kerenidis
Frédéric Magniez
Filippo Miatto
Simon Perdrix
Simone Severini

Conclusions and outlook

• We have a corpus of algorithms with provable speedups.

  • Simple to extend current algorithms to more powerful models.

• Quantum algorithms seems to work promisingly well in ML:

  • $\kappa(A)$, $\mu(A), s, \eta, \epsilon$,

• QML might allow solving new or  existing problems:

  • better, faster, cheaper, or a combination.

• Artificial Intelligence might be promising to explore

  • Smaller QRAM?

• We should work directly on state-of-the-art ML algorithm:

  • Interpretable, explainable, fair, robust, privacy-preserving ML.

Thanks for your time, there is never enough.

Evolution of Mutual Information between layers while training a QNN

The dropout technique for avoiding barren plateaus

Rebecca Erbanni - Master's thesis

Alexander Singh - IRIF internship

Counting triangles

$$\Delta(G) = \frac{1}{6} Tr[A^3]$$

• Create block encoding of $B=A^{1.5}$

• Estimate $Tr[B^TB]$

Van Apeldoorn, Joran, et al. - 58th Annual Symposium on Foundations of Computer Science - 2017.

Canonical Correlation Analysis

Input: $\{ (x_i, y_i) \}_{i=0}^n$ where $x_i \in \mathbb{R}^{d_1}$ and $y_i \in \mathbb{R}^{d_2}$ i.e. matrices $X,Y$

Classical:

• Step1: Solve GEP $$\Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX}w_x = \lambda^2\Sigma_{XX}w_x$$

• Step 2: Find $w_y$ by

  $w_y = \frac{\Sigma_{YY}^{-1}\Sigma_{YX}w_x}{\lambda}$

CCA model: find $w_x, w_y$ such that

${w_x, w_y} =\arg\max_{w_x, w_y} cos((Xw_x, Yw_y))$

Quantum:

$\Sigma_{XX}\Sigma_{XY}^{-1}\Sigma_{YY} = U\Sigma V^T$

$W_x = \Sigma_{XX}^{-1/2}U$

 $W_y = \Sigma_{YY}^{-1/2}V$.

Quantum algorithms for model checking

A state-space exploration approach

• Formalize our software as an automaton $A_P$.

• For a temporal property $f$ we build the automaton $A_{\neg f}$.

• Solve the emptiness problem of the language:  $$L(A_P\times A_{\neg f}) = \emptyset$$.

Software $\mapsto$ specification LTL

$\mapsto$ Büchi automata $\mapsto$  $\omega$-language

Idea: use quantum DFS!

Dürr, Christoph, et al. - SIAM Journal on Computing 35.6 (2006)

Theorem: The emptiness problem for $\omega$-languages is decidable!