PFHub REFACTor

Daniel Wheeler

https://pages.nist.gov/pfhub

11/02/2021

Current Difficulties

• Difficult for me to maintain and update
  • Difficult to implement BM8 and BM9 due to data aggregation
  • Frontend needs updating (JS / HTML dependency versions)
  • Apps need maintenance (Staticman, data app)
• Difficult for collaborators to develop
  • Data views impossible to amend
  • Wiki is unreliable

Solutions

• Use Jupyter Notebooks pervasively
• Better upload procedure
  • Fork example repository
• Use GitHub Actions / CI to update data views
  • Stop using in-browser updates
  • Remove apps / no more additional apps

Radical idea: no custom Website

USE Jupyter Notebooks

• Move comparison pages to Jupyter notebooks (in progress, implementing BM8)
• Auto-generated individual landing pages via notebooks
• Check notebooks on the CI
  • instead of JS/HTML frontend views

Jupyter Example

Jupyter Example

PFHUb PYTHON MODule

• Python package that deals with all data transformations and aggregations (implemented)
• All data exported as Pandas dataframes
• Easy for others to augment, develop, change
• Everything in Python
  • Plotly has improved Python support
• Everything working outside of website setting

UPLOAD Procedure

• Fork example repository

  • Edit pfhub.yaml in repository

  • Store data in forked repository
  • Git LFS for large data
  • PFHub extracts metadata from forks
  • GitHub Actions rebuilds website and notebooks after fork

github Actions / Git LFS

• Use GitHub Actions / CI
  • Stop using Staticman, Data app, Box app
• Use Git LFS
  • Very low barrier solution both for maintenance and for users

No custom website

• Notebooks for everything
• Nbviewer for display
• Binder / Jupyter Hub instance for all notebooks
• GitHub wiki for content
• GitHub Actions + CI for all automation
• Git LFS for larger data
• No more apps to maintain
• ...maybe keep small Jekyll front page?

Reduced maintenance, Easier to collaborate and Better Interactive user experience