OPTIMIZATION, NEURAL NETWORKS AND INVERSE DESIGN
FROM NANOPHOTONICS TO COMPUTATIONAL IMAGING
Giovanni Pellegrini @
WHO AM I?
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
Giovanni Pellegrini
Associate Professor - Physics Department - University of Pavia
Research Interests
- Computational Nanophotonics
- Machine Learning and Optimization for Nanophotonics
- Machine Learning for Image Reconstruction
- Machine Learning for Signal Processing
Also interested in
- Industrial Automation and Robotics
- Deep Learning for Machine Vision
- Both @ Sinteco Robotics
OPTIMIZATION, NEURAL NETWORKS AND INVERSE DESIGN
FROM NANOPHOTONICS TO COMPUTATIONAL IMAGING
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
GENETIC MULTI-OBJECTIVE OPTIMIZATION FOR INVERSE DESIGN OF 1DPC
SUPERVISED NEURAL NETWORKS FOR DIRECT AND INVERSE DESIGN OF NANOHOLE ARRAYS
SUPERVISED, PHYSICS INFORMED NEURAL NETWORKS FOR PHASE RETRIEVAL
UNSUPERVISED, PHYSICS INFORMED NEURAL NETWORKS FOR PTYCHOGRAPHY
GENETIC MULTI-OBJECTIVE OPTIMIZATION FOR INVERSE DESIGN OF 1D PHOTONIC CRYSTALS
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
Giovanni Pellegrini, Jonathan J. Barolak, Marco Finazzi, Michele Celebrano, F. Michelotti, A. Occhicone and Paolo Biagioni
CHIRALITY (IN NATURE)
Molecules
DNA
Proteins
L Enantiomer
R Enantiomer
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
PROBING CHIRALITY WITH ELECTROMAGNETIC FIELDS
TE Polarization
TM Polarization
Left = TE + i*TM
Right = TE - i*TM
Circular Dichroism (CD)
Optical Rotation (\(\psi\))
\[\mathrm{CD = A_{R}-A_{L}}\]
\[\mathrm{\psi = \Delta \theta} \]
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
SUPERCHIRAL LIGHT
Local optical chirality
C = -\frac{\varepsilon_{0} \omega}{2} Im(\mathrm{\mathbf{E}}^{*} \cdot \mathrm{\mathbf{B}})
Optical chirality enhancement
\frac{C}{C_{\mathrm{CPL}}}
Finazzi, Marco, et al. "Quasistatic limit for plasmon-enhanced optical chirality." Physical Review B 91.19 (2015): 195427.
Schäferling, Martin. "Chiral nanophotonics." Springer Series in Optical Sciences 205 (2017): 159.
Mattioli, F. et al. Plasmonic Superchiral Lattice Resonances in the Mid-Infrared. ACS Photonics 7, 2676–2681 (2020)
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
SUPERCHIRAL LIGHT WITH A PLANAR GEOMETRY
-
Large optical chirality enhancement
-
Uniformity over large areas
-
Optical chirality switching
-
Broadband operation
-
Flat spectral response
Planar Geometries as an Ideal Candidate
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
A PLANAR GEOMETRY FOR SUPERCHIRAL LIGHT: TAKE 1
TE ± i*TM
TE ± i*TM
Surface Plasmon
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
A PLANAR GEOMETRY FOR SUPERCHIRAL LIGHT: TAKE 2
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
\[ \mathrm{ (\lambda_{TM},\theta_{TM})} \]
\[ \mathrm{ (\lambda_{TE},\theta_{TE})} \]
\[ \mathrm{ (\lambda_{TM},\theta_{TM})} \neq \mathrm{ (\lambda_{TE},\theta_{TE})} \]
Bloch Surface Waves (BSW)
TE ± i*TM
TE ± i*TM
ALIGNING THE DISPERSION RELATIONS
\[ \mathrm{ (\lambda_{TM},\theta_{TM})} \neq \mathrm{ (\lambda_{TE},\theta_{TE})} \]
\[ \mathrm{ (\lambda_{TM},\theta_{TM})} = \mathrm{ (\lambda_{TE},\theta_{TE})} \]
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
AN INVERSE DESIGN SOLUTION
GENETIC MULTI OBJECTIVE OPTIMIZATION
\omega_1 \, (\lambda_1)
\omega_2 \, (\lambda_2)
\begin{cases}
C(\omega_1) = -\frac{\varepsilon_{0} \omega_1}{2} Im(\mathrm{\mathbf{E}}^{*} \cdot \mathrm{\mathbf{B}}) \\
C(\omega_2) = -\frac{\varepsilon_{0} \omega_2}{2} Im(\mathrm{\mathbf{E}}^{*} \cdot \mathrm{\mathbf{B}})
\end{cases}
Maximize \(C \)
@ \(\omega_1\) and \(\omega_2\)
@ the 1DPC Surface
SIMULTANEOUSLY!!!
Band Alignement
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
GENETIC MULTI-OBJECTIVE OPTIMIZATION: NSGAII
Vilfredo Pareto
\mathbf{t}=
\begin{pmatrix}
t_1 \\
t_2 \\
t_3 \\
... \\
t_i \\
... \\
t_n \\
\end{pmatrix}
\in \mathbb{R}^{n} =
Mixing
Mutation
Selection
1st Gen.
2nd Gen.
n-th Gen.
Evolution
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
GENETICALLY EVOLVED (NSGAII) SiO₂-Ta₂O₅ 1PDC
SiO\(_{2}\)
Ta\(_{2}\)O\(_{5}\)
SiO\(_{2}\)
Ta\(_{2}\)O\(_{5}\)
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
GENETICALLY EVOLVED (NSGAII) SIO2-SIOX 1PDC
SiO\(_{2}\)
SiO\(_{x}\)
SiO\(_{2}\)
SiO\(_{x}\)
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
GENETICALLY EVOLVED (NSGAII) POLYMERIC 1PDC
Cellulose
Polystyrene
Cellulose
Polystyrene
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
GENETIC MULTI-OBJECTIVE INVERSE DESIGN
FINAL CONSIDERATIONS
-
Multiple Objectives
-
Global
-
Robust
-
Flexible
-
Physical Insight
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
SUPERVISED, PHYSICS INFORMED NEURAL NETWORKS FOR DIRECT AND INVERSE DESIGN OF NANOHOLE ARRAYS
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
| n.1 | n.2 | n.3 | |
|---|---|---|---|
| Lattice | Hexagonal | Square | Square |
| Material | Ag | Au | SiOx |
| Thickness (nm) | 100 | 115 | 90 |
| Radius (nm) |
60 | 110 | 140 |
| Pitch (nm) |
450 | 550 | 515 |
DIRECT
INVERSE
Jahan, T. et al. Deep learning-driven forward and inverse design of nanophotonic nanohole arrays: streamlining design for tailored optical functionalities and enhancing accessibility. Nanoscale (2024) doi:10.1039/D4NR03081H.
Structural parameters
spectra
INVERSE DESIGN WITH NEURAL NETWORK: A TOY PROBLEM
OPTICAL PROPERTIES OF NANOHOLE ARRAYS
Jahan, T. et al. Deep learning-driven forward and inverse design of nanophotonic nanohole arrays: streamlining design for tailored optical functionalities and enhancing accessibility. Nanoscale (2024) doi:10.1039/D4NR03081H.
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
THE NOTEBOOKS AND THE TUTORIAL SLIDES
Data analysis
Direct model
Inverse model
Slides for the full tutorial
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
A "TOY" PROBLEM FOR OUR TUTORIAL
OPTICAL PROPERTIES OF NANOHOLE ARRAYS
| n.1 | n.2 | n.3 | |
|---|---|---|---|
| Lattice | Hexagonal | Square | Square |
| Material | Ag | Au | SiOx |
| Thickness (nm) | 100 | 115 | 90 |
| Radius (nm) |
60 | 110 | 140 |
| Pitch (nm) |
450 | 550 | 515 |
DIRECT
INVERSE
Jahan, T. et al. Deep learning-driven forward and inverse design of nanophotonic nanohole arrays: streamlining design for tailored optical functionalities and enhancing accessibility. Nanoscale (2024) doi:10.1039/D4NR03081H.
Structural parameters
spectra
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
IN PRACTICE
| Lattice | Hexagonal |
| Material | Ag |
| Thickness (nm) | 100 |
| Radius (nm) |
60 |
| Pitch (nm) |
450 |
\Rightarrow x=
\begin{pmatrix}
p_1 \\
p_2 \\
p_3 \\
p_4 \\
p_5 \\
\end{pmatrix}
\in \mathbb{R}^{5}
\Rightarrow
\begin{cases}
p_1 \in \{0,1\}\\
p_2 \in \{0,1,2\}\\
p_3,p_4,p_5 \in \mathbb{R} \\
\end{cases}
\Rightarrow y=
\begin{pmatrix}
t_1 \\
t_2 \\
\vdots \\
t_n \\
\vdots \\
t_{200} \\
\end{pmatrix}
\in \mathbb{R}^{200}
\Rightarrow t_n \in [0,1]
Direct: \( g_{_{W}}:\mathbb{R}^{5} \to \mathbb{R}^{200} \)
Inverse: \( g_{_{W}}:\mathbb{R}^{200} \to \mathbb{R}^{5} \)
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
EXPLORING THE DATA: PLOT
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
THE IMPORTANCE OF THE DATA
Credits: Alberto De Giuli
AI GENERATED
REAL PICTURE
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
MODEL PERFORMANCE: DIRECT PROBLEM
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
MODEL PERFORMANCE: INVERSE PROBLEM
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
SUPERVISED, PHYSICS INFORMED NEURAL NETWORKS FOR PHASE RETRIEVAL
Giovanni Pellegrini and Jacopo Bertolotti
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
HOW THIS COLLABORATION WAS BORN
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
THE PROBLEM
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
Scattering
Autocorr
Autocorr
Inversion
Inversion
THE IDEA: NEURAL NETWORK VS ITERATIVE METHODS
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
ALSO: We want to erode the correlation information!!!
THE IDEA: RECONSTRUCTING INCOMPLETE INFORMATION
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
-
Current phase retrieval techniques adopt iterative approaches to recover phase information from the modulus of the Fourier transform
-
We apply a circular erosion mask to the Fourier autocorrelation, in order to assess the performance of traditional methods in presence of partial information
-
We observe that traditional approaches struggle in the presence of incomplete information. We employ a deep neural network to overcome these limitations, and show that such and approach vastly over-performs traditional methods in a large variety of scenarios
THE FUNDAMENTALS: DEFINING THE DATASET
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
-
We devise a synthetic dataset based on the MNIST handwritten digits as a benchmark for the reconstruction performance
DEFINING THE NEURAL NETWORK ARCHITECTURE
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
-
We adopt a deep convolutional autoencoder based on the DeepLabV3+ architecture with a L1 loss computed on the input and reconstructed autocorrelations
TAILORING THE TRAINING PROCESS
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
-
Phase 1: We train the network on full autocorrelation
-
Phase 2: We progressively erode the input autocorrelation, and train the network to reconstruct the missing information
-
Phase 3: We perform a final stabilization training
DEEP LEARNING: RESULTS
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
Deep Learning Vs Iterative
Deep Learning: exploration
UNSUPERVISED, PHYSICS INFORMED NEURAL NETWORKS FOR PTYCHOGRAPHY
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
Giovanni Pellegrini, Jonathan J. Barolak, Carmelo Grova, Charles S. Bevis, Daniel E. Adams and Giulia F. Mancini
PTYCHOGRAPHY FOR DUMMIES
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
Complex Image
Complex Probe
Probe Raster on a Grid
Free Space
Propagation
Sample Plane
Image Plane
AN UNTRAINED, PHYSICS INFORMED NETWORK APPROACH
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
-
The reconstruction process happens with an untrained network on a physics-informed closed loop. The Neural Network substitutes entirely the iterative deterministic algorithm.
AN UNTRAINED, PHYSICS INFORMED NETWORK APPROACH
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
-
We tested the reconstruction performance against different probe grids
-
The physics informed neural network approach apparently allows for reconstructions with very sparse gridding
-
We devise a strategy to speedup the reconstruction
-
We adopt a warmup strategy reconstructing an unrelated dummy image with a similar probe
-
Without warmup the reconstruction does not reach convergence
-
With warmup the reconstruction quickly reaches convergence
HAPPY INVERSE DESIGNING!
Dipartimento di Fisica - Università di Pavia
Polimi 17 March 2025, Milan, Italy
