PFHub Overview

Daniel Wheeler

https://pages.nist.gov/pfhub

$ git log 7752dec8
commit 7752dec8b659657af62796c0d85d021538cd764f
Author: Daniel Wheeler <daniel.wheeler2@gmail.com>
Date:   Fri Feb 6 10:57:03 2015 -0500

    Initial commit

Overview

What's new
Headaches / problems / issues
- How do we broaden the community?
- Improve participation
What next?

What'S new

Commits

What's new

Spec for each BM 1-7
Many UX/design changes
- Responsiveness
Display page for each individual upload
BM spec table
BM 1, 2, 3 & 7 display of comparison results
Upload form improvements
Better testing
Surge

Headaches

Too many variations in each benchmark
```
usnistgov/pfhub is 931 MB
```
Can't get large datasets on frontend
Need an app
- Backend data manipulation
- CORS
- Replace Staticman
- Run Dockerfiles
How to get more uploads?
How to get more participation on the website?

PFHub DEsIGN

What NEXT?

Dockerfile / NIX / Vagrant for website dev
Comparison pages for all benchmarks
Code refactor
- Vega → Plotly
- Remove backend data processing
- React.js
Annotated upload examples using #491
- How to archive data and implement benchmarks
- Lint data on submission
Implement backend app
Improve benchmark comparisons
- more, bigger data
- tags
Upload and run containers

MILESTONES

v0.1: code metadata, May 2016
v0.2: upload results with meta.yaml, May 2016
v0.3: notebooks, benchmark specs, Dec 2016
v0.3.1: result pages, charts, upload form, Sep 2017
v0.4: reorg the site, Oct 2017
v0.4.1: all BM specs, Dec 2017
v0.4.2: refactor upload form, Feb 2018
v0.4.3: all results comparison pages
v0.5: developer uploads, big code refactor, improve frontend UX, make comparison pages better
v1.0: uploads from wider community, data app/server
v1.1: improve benchmark comparisons
v2.0: upload and run simulations

Past

Future

HelP

Uploads / data for BM 3, 4, 5, 6, 7
- be prepared to iterate, maybe need to rerun simulations multiple times
- help refine spec upload instructions
- dockerize and annotate simulations as examples
Design of comparison pages
- input on charts
- ideas for better data display
Pull request reviews
Help with detailed project planning