UGM 2018 - Technology Update

Technology Update

June 5-7, 2018

iRODS User Group Meeting 2018

Durham, NC

Terrell Russell, Ph.D.

@terrellrussell

Chief Technologist, iRODS Consortium

Technology Update

In The Last Year

iRODS Release	Issues Closed
4.1.11	92
4.2.2	39
4.2.3	76 + 18

Plugins

Python Rule Engine Plugin
Storage Tiering Rule Engine Plugin
Auditing (AMQP) Rule Engine Plugin
S3 Resource Plugin
GSI Authentication Plugin
Kerberos Authentication Plugin
Curl Microservice Plugin

Clients

Python iRODS Client
Automated Ingest Framework
Cloud Browser

~/irods $ git shortlog --summary --numbered 4.1.10..4-1-stable
    31  Ben Keller
    20  Alan King
    20  Terrell Russell
     7  Hao Xu
     7  Rick Skarbez
     7  jkgill
     6  Zoey Greer
     5  Jason Coposky
     2  Matthew Vernon
     1  Brian N. Makin


~/irods $ git shortlog --summary --numbered 4.2.1..4-2-stable
    51  Alan King
    27  Jason Coposky
    22  Hao Xu
    18  Terrell Russell
    17  Ben Keller
    17  Zoey Greer
    15  Kory Draughn
     9  jkgill
     8  Justin James
     5  Andrew Kelly
     5  Kyle Ferriter
     2  d-w-moore
     2  xu-hao
     1  Felix A. Croes
     1  Marco Grossi
     1  Sam Morris

In The Last Year

Ongoing Development Work

iRODS 4.1.12
iRODS 4.2.4
iRODS 4.3.0
Python iRODS Client (PRC)
Indexing Framework
Lustre Integration
Metalnx
CockroachDB Database Plugin
Ceph RADOS Resource Plugin
QueryArrow Database Plugin
R Client Library (R-iRODS)
Multipart Transfer, v5 API
NFS4J Integration
Metadata Templates
Testing Infrastructure

4.1.12 Update

Replicas in a complex tree (across 2 zones)...

Not enough replicas! #3746

Problem: 1 copy == BAD

Cause(s): Network, storage, OS...

Solution: Try harder

Replication retry (tested with mungefs)

Too many replicas! #3665

Problem: Filling up storage

Cause: Surprise concurrency? Repl resc?

Solution: ???

"Enhanced logging" with PID/user info

root:passthru
└── replicate:replication
    └── green-rand:random
    │   └── green0:random
    │   │   ├── green0ufs0
    │   │   └── green0ufs1
    │   └── green1:random
    │       ├── green1ufs0
    │       └── green1ufs1
    └── red-rand:random
        └── red0:random
        │   ├── red0ufs0
        │   └── red0ufs1
        └── red1:random
            ├── red1ufs0
            └── red1ufs1

To be continued...

4.3.0 Update

irodsMonitor process
Checksums moving down into resource plugins
Containerize installation
Logging overhaul

4.3.0 Logging Update

Today

Inconsistent
Incomplete (syslog support)
Not very helpful in tracking what caused an error
Not very helpful when multiple servers are involved

Design Goals

Leverage existing logging library (spdlog, etc.)
Enable admins to easily capture, process, and analyze activity
Consistent
Easily track errors across multiple servers (hostname, timestamp, PID, plugin, etc.)
Tie in existing infrastructure
Provide more options for controlling output

Join Us!

What would you like to see accomplished with the logging refactor?
RFC: https://github.com/irods/irods_rfcs/blob/master/0001_logging.md
Have questions/suggestions? Let's talk!

iRODS Build and Test - Yesterday

July 2011

Python → Node.js → RabbitMQ → Celery → Eucalyptus

October 2012

Python → Node.js → ssh → OpenStack

January 2013

Hudson → Python → OpenStack

October 2013

Hudson → Python → vSphere long-running VMs

Spring 2015

Jenkins → Python → Ansible → vSphere dynamic VMs

iRODS Build and Test - Today

Spring 2015 - onwards

Jenkins → Python → Ansible → zone_bundles → vSphere dynamic VMs

Changes Since 2017

Ended support for CentOS 6 and Ubuntu 12
Moved iRODS build/test logic from Ansible to per-repository python modules
Unified Jenkins workflow to test all supported plugins
Consolidated to two parameterized Jenkins jobs
- run-script-on-vms
- run-script-on-irods-zone