Min-Hsiu Hsieh
Hon Hai (Foxconn) Quantum Computing Research Center
Progress Reports in Quantum Machine Learning
NTU Quantum Seminar
Type of Input
Type of Algorithms
CQ
CC
QC
QQ
CQ
QQ
QC
-
Linear Equation Solvers
-
Peceptron
-
Recommendation Systems
-
Many Others (such as non-Convex Optimization)
-
State Tomography
-
Entanglement Structure
-
Quantum Control
How could QML achieve better end-to-end runtime?
QML Process
1. Readin
2. Readout
Many Challenges!
3.
Learning Machines
4. Noise
3.
Learning
Machine
Expressivity
Trainability
Generalization
Learning
Model
"how the architectural properties of a neural network (depth, width, layer type) affect the resulting functions it can compute"
[1] On the Expressive Power of Deep Neural Networks. (ICML2017) arXiv:1606.05336
3.1
Expressivity
\(\geq\)
[1] Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, Dacheng Tao. The Expressive Power of Parameterized Quantum Circuits. Physical Review Research 2, 033125 (2020) [arXiv:1810.11922].
3.1
Expressivity
\(\leq\)
\(\geq\)
"How easy is it to find the appropriate weights of the neural networks that fit the given data?"
3.2
Trainability
3.2
Trainability
Barren Plateau problem:
\mathbb{E}_{\theta}\|\triangledown_{\mathbf{\theta}} f \|^ 2 =\epsilon \leq 2^{-\text{poly}(n)}
\text{where } f(\mathbf{\theta},\rho) =\text{Tr}[O U(\theta)\rho U(\theta)^\dagger].
[1] Jarrod R McClean, Sergio Boixo, Vadim N Smelyanskiy, Ryan Babbush, and Hartmut Neven. Barren plateaus in quantum neural network training landscapes. Nature communications, 9(1):1– 6, 2018.
3.2
Trainability
Known BP Results
3.2
Trainability
Bad News for QML
-
Flat loss landscape.
-
Extremely small toleration to noise.
3.2
Trainability
Contribution 1:
BP free architecture
\mathbb{E}_{\theta}\|\triangledown_{\mathbf{\theta}} f \|^ 2 \geq c(\rho_{in}) 2^{-3LS}
\text{where } {\bf{\theta}} \sim [0,2\pi].
[1] Kaining Zhang, Min-Hsiu Hsieh, Liu Liu, Dacheng Tao. Toward Trainability of Deep Quantum Neural Networks. [arXiv:2112.15002]
3.2
Trainability
Binary classification on the wine dataset (N=13)
[1] Kaining Zhang, Min-Hsiu Hsieh, Liu Liu, Dacheng Tao. Toward Trainability of Deep Quantum Neural Networks. [arXiv:2112.15002]
3.2
Trainability
Contribution 2:
Initialization Matters
\mathbb{E}_{\bf{\theta}}\|\triangledown_{\bf{\theta}} f \|^ 2 \geq c(\rho_{in}) \text{poly}(L)^{-1}
\text{where } {\bf{\theta}} \sim N(0, \frac{1}{4S(L+2)}).
[1] Kaining Zhang, Min-Hsiu Hsieh, Liu Liu, Dacheng Tao. NeurIPS 2022.3.2
Trainability
Finding the ground energy of the Ising model (N=15, L=10)
[1] Kaining Zhang, Min-Hsiu Hsieh, Liu Liu, Dacheng Tao. NIPS 2022.[1] Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, Shan You, Dacheng Tao. On the learnability of quantum neural networks. PRX-Quantum 2, 040337 (2021)[arXiv:2007.12369]
\bm{\theta}^*= \arg \min_{\bm{\theta}\in\mathcal{C}} \mathcal{L}(\bm{\theta},\bm{z})
\mathcal{L}(\bm{\theta}):= \frac{1}{n}\sum_{j=1}^n \ell(y_i, \hat{y}_i) + r(\bm{\theta})
R_1\left(\bm{\theta}^{(T)}\right) := \mathbb{E} \left\|\nabla \mathcal{L}(\bm{\theta}^{(T)})\right\|^2
R_2\left(\bm{\theta}^{(T)}\right) := \mathbb{E}[\mathcal{L}(\bm{\theta}^{(T)})] - \mathcal{L}(\bm{\theta}^*)
3.2
Trainability
Contribution 3:
Trainability in ERM
R_1\left(\bm{\theta}^{(T)}\right) := \mathbb{E} \left\|\nabla \mathcal{L}(\bm{\theta}^{(T)})\right\|^2
R_1 \leq \tilde{O}\left(poly\left(\frac{d}{T(1-p)^{L_Q}}, \frac{d}{BK(1-p)^{L_Q}} \right) \right)
\(d\)＝ \(|\bm{\theta}|\)
\(T\)＝ # of iteration
\(L_Q\)＝ circuit depth
\(p\)＝ error rate
\(K\)＝ # of measurements
3.2
Trainability
[1] Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, Shan You, Dacheng Tao. On the learnability of quantum neural networks. PRX-Quantum 2, 040337 (2021)[arXiv:2007.12369]
R_2\left(\bm{\theta}^{(T)}\right) := \mathbb{E}[\mathcal{L}(\bm{\theta}^{(T)})] - \mathcal{L}(\bm{\theta}^*)
R_2\leq \tilde{O}\left( poly\left(\frac{d}{K^2B (1-p)^{L_Q}} ,\frac{d}{(1-p)^{L_Q}}\right) \right)
\(d\)＝ \(|\bm{\theta}|\)
\(T\)＝ # of iteration
\(L_Q\)＝ circuit depth
\(p\)＝ error rate
\(K\)＝ # of measurements
3.2
Trainability
[1] Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, Shan You, Dacheng Tao. On the learnability of quantum neural networks. PRX-Quantum 2, 040337 (2021)[arXiv:2007.12369]
3.3
Generalization
"Generalization refers to the model's ability to adapt properly to new, previously unseen data, drawn from the same distribution as the one used to train the model."
[1] S. Arunachalam, A. B. Grilo, and H. Yuen, arXiv:2002.08240 (2020).
3.3
Generalization
Separation between Learning models
3.3
Generalization
Contribution:
"Noisy QNN can efficiently simulate QSQ oracle."
[1] Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, Shan You, Dacheng Tao. On the learnability of quantum neural networks. PRX-Quantum 2, 040337 (2021)[arXiv:2007.12369]
3.4
Quantum
Advantage
Any quantum advantage could be achieved using QML?
->No universal quantum advantage.
Qian, Y., Wang, X., Du, Y., Wu, X., & Tao, D. (2022). The dilemma of quantum neural networks. IEEE Transactions on Neural Networks and Learning Systems.
3.4
Quantum
Advantage
How to guarantee problem-dependent quantum advantage using QNN?
->We identify conditions when the expected risk in classification tasks is minimized.
Yuxuan Du, Yibo Yang, Dacheng Tao, Min-Hsiu Hsieh. "Demystify Problem-Dependent Power of Quantum Neural Networks on Multi-Class Classification." arXiv:2301.01597 (2023).
3.4
Quantum
Advantage
Contribution:
Yuxuan Du, Yibo Yang, Dacheng Tao, Min-Hsiu Hsieh. "Demystify Problem-Dependent Power of Quantum Neural Networks on Multi-Class Classification." arXiv:2301.01597 (2023).
經典: 1 error per 6 month in a 128MB PC100 SDRAM (2009)
量子: 1 error per second per qubit (2021)
4
Noise
(\bm{\theta}^*,\bm{a}^*)= \arg \min_{\bm{\theta}\in\mathcal{C},\bm{a}\in\mathcal{A}} \mathcal{L}(\bm{\theta},\bm{a}, \mathcal{E}_{\bm{a}})
\(\mathcal{C}\): The collection of all parameters
\(\mathcal{A}\): The collection of all possible circuits
\(\mathcal{E}_{\bm{a}}\): The error for the architecture \(\bm{a}\)
[1] Yuxuan Du, Tao Huang, Shan You, Min-Hsiu Hsieh, Dacheng Tao. Quantum circuit architecture search: error mitigation and trainability enhancement for variational quantum solvers. arXiv:2010.10217 (2020).
4.1
Error Mitigation
4.1
Error Mitigation
[1] Yuxuan Du, Tao Huang, Shan You, Min-Hsiu Hsieh, Dacheng Tao. Quantum circuit architecture search: error mitigation and trainability enhancement for variational quantum solvers. arXiv:2010.10217 (2020).
Hydrogen Simulation+EM
4.1
Error Mitigation
[1] Yuxuan Du, Tao Huang, Shan You, Min-Hsiu Hsieh, Dacheng Tao. Quantum circuit architecture search: error mitigation and trainability enhancement for variational quantum solvers. arXiv:2010.10217 (2020).
Could noise become useful in QML?
YES!
4.2
Harnessing Noise
4.2
Harnessing Noise
[Robustness]
[Privacy]
4.2.1
Providing Privacy
Differential Privacy (DP)
Classical DP is well studied; however, Quantum DP is not.
[1] Li Zhou and Mingsheng Ying. Differential privacy in quantum computation. In 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pages 249–262. IEEE, 2017.
[2] Scott Aaronson and Guy N Rothblum. Gentle measurement of quantum states and differential privacy. Proceedings of ACM STOC‘2019.
4.2.1
Providing Privacy
Regression+DP
[1] Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, Shan You, Dacheng Tao. Quantum differentially private sparse regression learning. arXiv:2007.11921 (2020)
1. The first quantum DP algorithm.
2. Have the same privacy guarantee with the best classical DP algorithm.
3. Huge runtime improvement.
Contribution:
[1] Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, Shan You, Dacheng Tao. Quantum differentially private sparse regression learning. arXiv:2007.11921 (2020)
4.2.1
Providing Privacy
Adversarial Attack
[1] Lu et.al, “Quantum Adversarial Machine Learning". [arXiv:2001.00030]
4.2.2
Robustness
Adversarial Robustness
4.2.2
Robustness
2. Depolarizing noise suffices.
Contribution:
1. Explicit relation between p and \(\tau\).
[1]Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, Dacheng Tao, Nana Liu. Quantum noise protects quantum classifiers against adversaries. Physical Review Research 3, 023153 (2021). [arXiv:2003.09416].
4.2.2
Robustness