Benefits of Using Chef

Dmitry Duplyakin, University of Colorado

• Nice structure: roles, cookbooks, recipes, nodes, etc.
• Idempotence
• All artifacts are code: reusable and extendable
• Over 2,000 community-developed cookbooks at Supermarket
• Vision: single codebase for all sites within CloudLab​

Characterizing Performance and Power Efficiency on CloudLab

11/18/2015

Configuration Management with Chef: Architecture

Dmitry Duplyakin, University of Colorado

Characterizing Performance and Power Efficiency on CloudLab

11/18/2015

Chef Terminology

Dmitry Duplyakin, University of Colorado

• Client -- node that is being configured
• Server -- node that performs configuration
• Workstation (knife) -- management utility
  • Can run on the server; multiple workstations are allowed     

• Recipe = script
• Cookbook -- collection of recipes
• Role -- collection of cookbooks, typically assigned to clients
• Environment -- cluster, aggregation, region, etc.
• Attributes - variables set for specific nodes, environments, etc.

Characterizing Performance and Power Efficiency on CloudLab

11/18/2015

Chef: Closer Look

Dmitry Duplyakin, University of Colorado

Characterizing Performance and Power Efficiency on CloudLab

11/18/2015

Demo:

• Emulab chef-repo
• emulab-nfs cookbook and related roles
• Reporting and WUI

Chef: Push Jobs

Dmitry Duplyakin, University of Colorado

Characterizing Performance and Power Efficiency on CloudLab

11/18/2015

name “get_power"
description "Role applied to nodes that need to download power data"
override_attributes(
  "push_jobs" => {

    "whitelist" => {
      “get_power" => "cd /tmp ; git clone https://github.com/dmdu/power-client.git ;\
                      /bin/bash -x power-client/power-client.sh -s clemson -l 12h"
    }

  }
)
run_list [ "push-jobs" ]

• Push job wrapped in a role:

• Submit role:

# knife role from file get_power.rb

• Assign to a node:

# knife node run_list add head "role[get_power]"

• Run the job:

# knife job start get_power head

Started.  Job ID: 3f0ae42b88ea60365f7d07c64e30ff54

Running (1/1 in progress) ...

Running (0/1 in progress) ...

Power Analysis

Dmitry Duplyakin, University of Colorado

Characterizing Performance and Power Efficiency on CloudLab

11/18/2015

• Developed supporting cookbooks:
  • emulab-R, emulab-shiny, emulab-powervis 

Sources of Power Data

Dmitry Duplyakin, University of Colorado

Characterizing Performance and Power Efficiency on CloudLab

11/18/2015

IOUT_hex = ipmi-raw --no-probing --driver-type=SSIF \
--driver-address=0x10 --driver-device=/dev/i2c-0 \
0 6 0x52 0x05 0x40 0x02 0x8C

VIN_hex = ipmi-raw --no-probing --driver-type=SSIF \
--driver-address=0x10 --driver-device=/dev/i2c-0 \
0 6 0x52 0x05 0x40 0x02 0x88

IOUT = int(IOUT_hex,16) * 0.01239 - 25.3717
VIN = int(VIN_hex,16) * 0.005208

POWER = VIN*IOUT

• On-node power (available on ARM at Utah only):

• Chassis Manager (CM) power data (available for all 3 sites)
  • Collectors and database managed by Emmanuel Cecchet  

  • power-client.sh -s clemson -l 12h

Power Analysis on ARM - Workload 1

Dmitry Duplyakin, University of Colorado

Workload 1 - Gradual Load

Benchmark: HPGMG-FE

Scaling: idle to 8 cores incrementally

Blue: on-node power

Green: CM power

Observations:

• Time lag
• Consistent estimates of the total energy 
• >10x difference between samping rates

Characterizing Performance and Power Efficiency on CloudLab

11/18/2015

Power Analysis on ARM - Workload 2

Dmitry Duplyakin, University of Colorado

Characterizing Performance and Power Efficiency on CloudLab

11/18/2015

Workload 2 - Abrupt Load

Benchmark: HPGMG-FE

Scaling: idle to 8 cores

Blue: on-node power

Green: CM power

Observations:

• The time lag can be comparable to periods of high load  
• At small time scales, the difference between energy estimates is noticable

Sampling On-Node Power Draw at Different Rates 

Dmitry Duplyakin, University of Colorado

Workload 1

Workload 2

• Sampling at high rates: estimates of total energy are consistent
• Sampling at low rates: estimates blow up
• Can find this threshold programmatically

Characterizing Performance and Power Efficiency on CloudLab

11/18/2015

Comparing Different Energy Estimates

Dmitry Duplyakin, University of Colorado

• Mean and variance decrease as the interval grows
• "Elbow" is at ~1000s; above that, the difference is at ~2%

Characterizing Performance and Power Efficiency on CloudLab

11/18/2015

Workload 1

Workload 2

Interactive Analysis of Power Data

Dmitry Duplyakin, University of Colorado

powervis

https://github.com/emulab/shiny-server

• Interactive dashboard for analysis and visualization of experiment power data
• Uses R and Shiny (by RStudio)
• Demo​

Characterizing Performance and Power Efficiency on CloudLab

11/18/2015

Power: Summary

• Gained confidence with using power samples
• Investigated granularity and accuracy of total energy estimates
• Can experiment with energy optimization
  • For instance, using Periscope Tuning Framework (PTF)
• Can make reasonable predictions

Dmitry Duplyakin, University of Colorado

Characterizing Performance and Power Efficiency on CloudLab

11/18/2015

Performance Analysis

Dmitry Duplyakin, University of Colorado

Performance of CloudLab Resources

09/30/2015

• Developed supporting cookbooks:
  • emulab-gcc, emulab-powervis, emulab-blis 

BLIS DGEMM: Single Core

Dmitry Duplyakin, University of Colorado

Performance of CloudLab Resources

09/30/2015

41.70 GF

16.23 GF

3.17 GF

Single-core theoretical peak in DP:

• 3.2 GHz (AVX, Turbo)
• 2 FMAs per cycle
• 2 flops in FMA
• 4 doubles in vector units

​Total: 51.2 GF

Single-core theoretical peak in DP:

• 3.0 GHz (Turbo)
• 1 FMAs per cycle
• 2 flops in FMA
• 4 doubles in vector units

​Total: 24.0 GF

Single-core theoretical peak in DP:

• 2.4 GHz (Normal)
• 2 cycles per FMA
• 2 flops in FMA
• 2 doubles in vector units

​Total: 4.8 GF

BLIS DGEMM: CPU and Node

Dmitry Duplyakin, University of Colorado

Performance of CloudLab Resources

09/30/2015

HPGMG-FE

Dmitry Duplyakin, University of Colorado

Performance of CloudLab Resources

09/30/2015

Measuring Performance of BLIS DGEMM on ARMv8

Dmitry Duplyakin, University of Colorado

Performance of CloudLab Resources

09/30/2015

• Can use PAPI hardware counters on ARMv8 to accurately measure application performance without code instrumentation

Running HPGMG-FE at Different Sites

Dmitry Duplyakin, University of Colorado

Performance of CloudLab Resources

09/30/2015

• Consistent performance at each site

Performance: Summary

Dmitry Duplyakin, University of Colorado

Performance of CloudLab Resources

09/30/2015

• Evaluated single-core, CPU, and node performance at every site
  • Theoretical peak, BLIS DGEMM, and HPGMG-FE
• Estimated power efficiency (GF/W)
• Used PAPI to estimate performance of HPGMG-FE on ARMv8
• Proposal 1: use Periscope Tuning Framework (PTF) for compiler flag optimization
• Proposal 2: use Extra-P for performance modeling of real applications