Interactive QA Dashboard for

LSST Pipelines

Demo #1

  • Panel/Holoviews/Bokeh
    • Dashboard framework and interactions
  • Datashader
    • Renders aggregated scatter-plot data as image
  • Parquet
    • Allows loading only desired columns
  • Dask
    • Parallel reading from disk
  • Kartothek
    • Smart on-disk partitioning for targeted reads

The Tools

Demo #2

  • prepareQADashboard.py
  • clone and install dashboard repo & deps
  • lsst_data_repartition
  • lsst_data_explorer

How to DIY

  • Custom/on-demand columns
  • Live metric calculations
  • Extract selected source IDs
  • Drill down to image cutouts
  • Save/share state
  • Requests?

Future?

(related: any other stack uses of parquet/kartothek partitioning?)

Made with Slides.com