4.1

iRODS User Group Meeting

June, 2015

Chapel Hill, NC

Jason Coposky, Chief Technologist

Last Year

Enterprise iRODS 3.0.1

Community iRODS 3.3.1

iRODS Consortium Release 4.0

iRODS Consortium Release 4.1

Focus on Hardening

 

 

Focus on Enterprise Readiness

 

 

Membership and Community Requests

Hardening

Static Analysis


Membership and Community Input


General Improvements

Hardening - static analysis

  • Fixed nearly 1200 Coverity issues in iRODS and external libraries
    • Now down to zero issues in iRODS and libraries
    • Includes Buffer Overruns, Null pointer dereferences, use of uninitialized variables 
  • Continuously keep Coverity, cppcheck and STACK clean

Hardening - community input

192 Bugs closed

 

  • Many permission inconsistencies ironed out

  • Parallel transfer works in multi-homed networks

  • irsync sending only updated files

  • Zip files available via ibun

  • Zero-length file behavior is consistent

  • Delayed rules running correctly

 

Hardening - general improvements

  • Push the plugin architecture farther into the API
  • Move magic numbers to run-time configuration
  • Removed built-in PostgreSQL DB Vacuum functionality

  • Removed boot user from install script

  • Removed "run_server_as_root" option

  • Removed roles for storageadmin, domainadmin, and rodscurators

  • Removed obfuscation (SIDKey and DBKey)

Enterprise Readiness

Move configuration to validated JSON

 

Zone Introspection

 

Grid-wide control plane

 

New Client-Hints API

Enterprise Readiness - configuration

New schema-driven validated JSON configuration files include

  • server_config.json
  • database_config.json
  • irods_environment.json
  • host_access_control_config.json
  • hosts_config.json

 

Automatically generated on upgrade from 4.0.3

Enterprise Readiness - zone report

A step towards configuration management for iRODS deployments

 

Generates a validated JSON document describing every server in the Zone

  • Sanitized snapshots of all json configuration files
  • Listing of all installed plugins with checksums
  • Listing of all configured resources
  • Contents of rule files in base64 encoding

 

Leveraged by Consortium CI for deployment and testing of topologies

Enterprise Readiness - control plane

Side channel for communication directly with the iRODS Server

 

Per-server or grid-wide operations

  • Status information - validated JSON containing connections, pids, server state
  • Gracefully Pause and Resume
  • Gracefully Shut Down

Enterprise Readiness - client hints

New API call - communicate Server configuration information to the Client

 

Validated JSON document containing

  • Strict ACL setting
  • Desired hash-scheme
  • Match hash-scheme policy
  • Listing of installed specific queries

 

Will track iRODS schema versions and grow over time

Membership and Community Requests

Put with atomic metadata

 

Key-Value Passthrough for iput and iget

 

Zero Length files now trigger resource hierarchy behavior

 

Weighted Passthru Resource Plugin

Requests - zero length files

Zero Length files now trigger Resource Composition behavior

 

  • replication - Compound & Replication Resources

Requests - iput and iget

iput now supports two new options

 

  • --metadata - atomically assign metadata after data is at rest and registered
    • Encoded in the form "attr1;val1;unit1;attr2;val2;unit2;"
  • --acl - atomically assign ACLs after data is at rest and registered
    • ​Encoded as "<perm> <user_or_group>;..."

 

iput and iget now support a new option

 

  • --kv_pass - pass key-value strings through to the resource hierarchy
    • ​Encoded in the form "key1=value1;key2=value2"

Requests - weighted passthru plugin

Passthru Coordinating Resource now supports weights for read and write operations

 

  • Multiplier for votes of the child resource
  • Use to influence the behavior of Resource Composition
    • ​Create a read-only resource
    • Create a write-only resource
    • Programmatically redirect reads or writes based on system behavior
  • Floating point values
  • Set in the context string of the form ​read=1.1;write=0.5

Questions?

iRODS 4.1 Overview

By jason coposky

iRODS 4.1 Overview

This is a highlight of the changes from iRODS 4.0.3 to iRODS 4.1

  • 1,304