The growing role of machine learning in radio continuum surveys 

Dr. Kenneth Duncan

Leiden Observatory

Work by Rafael Mostert (Leiden) and the LOFAR surveys team

What drives the rise in machine learning in astronomy?

Make science with large datasets tractable


Automate previously manual repetitive tasks


Speed up or improve key measurements required for large samples


Enable exploration and discovery of massive and complicated datasets

The Low Frequency Array (LOFAR)

LOFAR Key Science Projects

Epoch of Reionization

Deep Extragalactic Surveys

Cosmic Magnetism


High-energy Cosmic Rays

Solar Science / Space Weather

The LOFAR Two-metre Sky Survey (LoTSS)

LoTSS DR1 HETDEX Spring Field

Shimwell et al. (2019) - Radio Images/Catalogs

Williams et al. (2019) - Optical IDs

Duncan et al. (2019) - Photo-z's and Rest-frame magnitudes

424 sq.deg - 70uJy/beam RMS : >320,000 radio sources
~10x deeper than FIRST VLA Survey

The LOFAR Two-metre Sky Survey (LoTSS)

Captain America's favourite survey!

Challenges - Part 1

LOFAR's exquisite survey depth and its sensitivity to extended emission means an unprecedented zoo of radio morphologies

How do we begin to explore, classify and understand this diversity of morphologies in a quick, unbiased way?

Dimensionality reduction and unsupervised learning

Self-organised (or Kohonen) maps 

Flipping and Rotational Invariance

Compare all possible rotations of the object with the prototypes 

'PINK' - Polsterer et al. (2015)

Different pre-processing choices can significantly affect the resulting SOMs:

e.g. linear vs asinh/log stretch
Sigma-clipping of noisy regions

Ditto for SOM hyper-parameters (shape/smoothing scale)

8x8 Trained SOM for 15k extended radio sources

Choice of SOM size and shape can effect prototypes

Explore the LOFAR morphologies:


SOMs for morphological outlier identification

aka "Find me interesting things in unseen data"

SOMs for morphological outlier identification

Challenges - Part 2

Source association & localisation

Lots of visual inspection still required


Radio Galaxy Zoo


LOFAR Galaxy Zoo

(soon to evolve into Radio Galaxy Zoo 2)

Automated source association & localisation

ClaRAN - Wu et al. (2018)

Faster Region-based Convolutional Neutral Net- works (Faster R-CNN)







Based on radio and optical or IR cutouts - identify sources

Where next?

Source assocation, localisation and cross-identification

Source identification

Source classification: e.g. FRI vs FRII (vs ...)

Visualization and Exploration

Deep Learning? Faster R-CNN etc. 

Intuitive and interactive VO services with more than just cutouts and catalogs

Full LoTSS sky and future SKA surveys will need robust automated pipelines in order to maximise their science value


SOMs/t-SNE ...?


As with all areas of astronomy - radio continuum surveys are ripe to be transformed by the development of machine learning


Unsupervised learning techniques offer a means of making big datasets easy to visualise and explore


Robust and accurate automatic identification, localisation and classification of sources represents the key next step


With massive and complex datasets - the way we access and explore this data will need to become more advanced - machine learning offers many exciting options for this 

Machine Learning in Radio Continuum Surveys

By Kenneth Duncan

Machine Learning in Radio Continuum Surveys

A short talk for the RAS Specialist Meeting "AI and ML Applied to Astronomy" - 8th March 2019

  • 111
Loading comments...

More from Kenneth Duncan