Deep Convolutional Autoencoders for Image Reconstruction from Incomplete Fourier Amplitude Measurements

Giovanni Pellegrini and Jacopo Bertolotti

https://slides.com/giovannipellegrini/icton2024

HOW THIS COLLABORATION WAS BORN

THE PROBLEM

Scattering

Autocorr

Inversion

THE IDEA: NEURAL NETWORK VS ITERATIVE METHODS

ALSO: We want to erode the correlation information!!!

THE FUNDAMENTALS: DEFINING THE DATASET

We devise a synthetic dataset based on the MNIST handwritten digits as a benchmark for the reconstruction performance

A PRELIMINARY CHECK: DO ITERATIONS WORK?

Current phase retrieval techniques adopt iterative approaches to recover phase information from the modulus of the Fourier transform
We apply a circular erosion mask to the Fourier autocorrelation, in order to assess the performance of traditional methods in presence of partial information
We observe that traditional approaches struggle in the presence of incomplete information. We employ a deep neural network to overcome these limitations, and show that such and approach vastly over-performs traditional methods in a large variety of scenarios

DEFINING THE NEURAL NETWORK ARCHITECTURE

We adopt a deep convolutional autoencoder based on the DeepLabV3+ architecture with a L1 loss computed on the input and reconstructed autocorrelations

TAILORING THE TRAINING PROCESS

Phase 1: We train the network on full autocorrelation
Phase 2: We progressively erode the input autocorrelation, and train the network to reconstruct the missing information
Phase 3: We perform a final stabilization training

DEEP LEARNING: RESULTS

Deep Learning Vs Iterative

Deep Learning: exploration

CONCLUSIONS

Neural network reconstructions routinely outperform traditional ones, especially in scenarios where large chunks of the autocorrelation information are missing
The neural network can perform phase retrieval on out of distribution samples.
The neural network performance degrades once the information is not just incomplete, but completely missing from the input

1. Pellegrini, G. & Bertolotti, J. Phase-Retrieval with Incomplete Autocorrelations Using Deep Convolutional Autoencoders. Preprint at https://doi.org/10.48550/arXiv.2304.09303 (2023).

Deep Convolutional Autoencoders for Image Reconstruction from Incomplete Fourier Amplitude Measurements

Giovanni Pellegrini and Jacopo Bertolotti

https://slides.com/giovannipellegrini/icton2024

HOW THIS COLLABORATION WAS BORN

THE PROBLEM

Scattering

Autocorr

Autocorr

Inversion

Inversion

THE IDEA: NEURAL NETWORK VS ITERATIVE METHODS

ALSO: We want to erode the correlation information!!!

THE FUNDAMENTALS: DEFINING THE DATASET

We devise a synthetic dataset based on the MNIST handwritten digits as a benchmark for the reconstruction performance

A PRELIMINARY CHECK: DO ITERATIONS WORK?

Current phase retrieval techniques adopt iterative approaches to recover phase information from the modulus of the Fourier transform

We apply a circular erosion mask to the Fourier autocorrelation, in order to assess the performance of traditional methods in presence of partial information

We observe that traditional approaches struggle in the presence of incomplete information. We employ a deep neural network to overcome these limitations, and show that such and approach vastly over-performs traditional methods in a large variety of scenarios

DEFINING THE NEURAL NETWORK ARCHITECTURE

We adopt a deep convolutional autoencoder based on the DeepLabV3+ architecture with a L1 loss computed on the input and reconstructed autocorrelations

TAILORING THE TRAINING PROCESS

Phase 1: We train the network on full autocorrelation

Phase 2: We progressively erode the input autocorrelation, and train the network to reconstruct the missing information

Phase 3: We perform a final stabilization training

DEEP LEARNING: RESULTS

Deep Learning Vs Iterative

Deep Learning: exploration

CONCLUSIONS

Neural network reconstructions routinely outperform traditional ones, especially in scenarios where large chunks of the autocorrelation information are missing

The neural network can perform phase retrieval on out of distribution samples.

The neural network performance degrades once the information is not just incomplete, but completely missing from the input

1. Pellegrini, G. & Bertolotti, J. Phase-Retrieval with Incomplete Autocorrelations Using Deep Convolutional Autoencoders. Preprint at https://doi.org/10.48550/arXiv.2304.09303 (2023).