iRODS 4.1 - Build and Test

2015 iRODS User Meeting, Jun 10-11 -- Renaissance Computing Institute (RENCI), Chapel Hill, NC

Terrell Russell, Ph.D.

Renaissance Computing Institute (RENCI)

University of North Carolina at Chapel Hill

 

 

 

 

Ben Keller

Renaissance Computing Institute (RENCI)

University of North Carolina at Chapel Hill

 

Motivation and Goals

Build and Test serves multiple purposes. The target audiences include core developers, external developers, users, users’ managers, and grid administrators.

 

  • Transparency (in both process and product)

  • Use of existing industry best practices

  • Coverage →  Confidence in Refactoring

  • Packaging Ease of installation and upgrade

  • Test framework idempotency

  • Test independence

  • Topology awareness

  • Automation, Automation, Automation 

The zone_bundle.json schema

Configuration Management

  • iRODS 4.1 contains integrated building blocks for configuration management.  The configuration files are all schema-backed and can be validated at any time.
  • https://github.com/irods/irods_schema_configuration
  • One of these JSON schemas defines the "zone_bundle".

 

The izonereport iCommand produces a zone_bundle about the entire local Zone.  This representation serves two primary, complementary use cases:

  • Debugging - We can see the actual settings.  Better than admins telling us.
  • Deployment - We can deploy a particular self-describing topology.

 

Combining these two use cases provides a very powerful effect.

iRODS Build and Test - Yesterday

July 2011

  • Python → Node.js RabbitMQ Celery Eucalyptus

 

October 2012

  • Python Node.js ssh OpenStack

 

January 2013

  • Hudson Python OpenStack

 

October 2013

  • Hudson Python vSphere long-running VMs

iRODS Build and Test - Today

Spring 2015

  • Jenkins → Python → Ansible → vSphere dynamic VMs

 

This version stands out for two reasons:

  • The iRODS Development team is now in charge of the full pipeline and only dependent on RENCI infrastructure for production-level vSphere.
  • Ansible-driven deployments of dynamic VMs guarantees a "clean slate".
  • More modular, standardized - lays the groundwork for the next steps.

 

iRODS Build and Test - Today

Coverage

 

 

 

 

 

 

 

 

Topology Tests

  • 1 iCAT + 3 Resource servers

 

 

 

  Additional Cumulative  
Standalone single server 59.2% 59.2% observed
Topology 2.3% 61.5% observed
Error Checking ~14% 75% estimated
Untested features ~25% 100% estimated

iRODS Build and Test - Tomorrow

Fall 2015 (expected)

  • Jenkins → Python → Ansible → zone_bundles → vSphere dynamic VMs

 

Features

  • Make tests "zone_bundle aware" 
  • Move to CMake
  • Separately versioned external/
  • Testing unpushed branches
  • Enforced code review
  • Federation testing

 

 

Thank you

Questions?

 

Demo!

 

Terrell Russell, Ph.D

RENCI

 

Ben Keller

RENCI

Links

UGM2015 - Build and Test

By iRODS Consortium

UGM2015 - Build and Test

  • 133