Active Learning in Performance Analysis
Dmitry Duplyakin
Jed Brown
Robert Ricci
09/14/2016
dmitry.duplyakin@colorado.edu
Outline
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Motivation
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Motivation: Example 1
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Each point represents a run of HPGMG-FE benchmark on a 4-node cluster provisioned on CloudLab testbed
Motivation: Example 2
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Each point represents a run of HPGMG-FE benchmark on a 4-node cluster provisioned on CloudLab testbed
Approach: Active Learning
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Approach: Gaussian Process Regression
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Approach: Putting it Together
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Upper: AL
Lower: GPR
Upper: Choose "best" experiment
Lower: Choose "best" hyperparameters
Approach: Details
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Upper: Choose "best" experiment
Lower: Choose "best" hyperparameters
Consider strategies:
Variance Reduction (VR):
Cost Efficiency (CE):
Use: Bayesian Model Selection
(Marginal Likelihood Maximization)
with 3 hyperparameters:
noise level, length scale, and amplitude
Implementation
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
* Note: offline refers to the fact that the prototype queries a database with collected data. Future work: in online mode, run AL alongside the computation
Analyzed Datasets
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Active Learning: 10 Iterations
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Shown points represent a subset of measurements in the Performance dataset; runtimes are log-transformed
Active Learning: 100 Iterations
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Shown points represent a subset of measurements in the Performance dataset; runtimes are log-transformed
Evaluation: Convergence Analysis
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Evaluation: Cost Analysis
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Summary and Future Work
Dmitry Duplyakin, University of Colorado
Active Learning in Performance Analysis
09/14/2016
Summary:
Future Work:
Dmitry Duplyakin, University of Colorado
Thank you!
Questions?
dmitry.duplyakin@colorado.edu
Active Learning in Performance Analysis
09/14/2016