Introduction

Dan Bedard

Director, iRODS Consortium

danb@renci.org

+1-919-445-0632

 

iRODS is free, open source software owned by a foundation called the iRODS Consortium.

• Goal is to sustain iRODS as free open source software by:

  ▹ Building good software.  ​▹ Growing the iRODS community.  ▹ Demonstrating value.

 

• Funds a team of 10+ developers, application engineers, documentation, support staff

The iRODS Consortium and Sustainability

Meet the Team

          Policy-Based Data Management

The Integrated Rule-Oriented Data System:

• Open Source

• Manages data for indefinite lifespan

• Metadata and event-triggered rules

 

Designed to work with big, important, and/or complex data

 

Example applications:

   • Combining data from multiple sources

   • Annotation and search

   • Tiered storage

   • Preservation

   • Sharing in place

 

 

 

Data

Virtualization

Data

Discovery

Workflow

Automation

Secure

Collaboration

  Data Virtualization

 

 

 

  Data Discovery

 

 

 

  Workflow Automation

 

 

 

  Secure Collaboration

          The Four Pillars

iRODS Clients

• Web-based and Standalone GUIs

  - iRODS Cloud Browser, MetaLnx, Kanki, Cyberduck

 

• Portals, External Systems

  - iPlant Discovery Environment, Islandora, Fedora Commons

 

• WebDAV for drag-and-drop access built in to the OS

• APIs: Python, REST, Qt, Java, C++

• Command Line Interface

User Group Meeting

June 7-9, 2016

Chapel Hill, NC

ugm2016.irods.org

Initial Trial

• Google Group
• Blog posts, social media
• Cloud images
• Documentation
• Training workshops
• iRODS Hub: The iRODS App Store

Proof of Concept to Pilot

• Occasional 1-on-1 Support
• iRODS Consortium Members
• iRODS Partners
• iRODS Consortium Service Contracts

Production

• iRODS Consortium Membership

Building Community and Demonstrating Value

https://irods.org/documentation/

Getting Plugged In with iRODS...

User Profile: NASA Atmospheric Science Data Center

• 2 PB of archived satellite data

• Publicly available, subsetting on demand

• In-house ingest and archiving software: ANGe (Archive Next Generation)

Virtual Collections

 

ls –l

/CER_100100.2012053100
/CER_100100.2012053100.met
/CER_100100.2012053101
/CER_100100.2012053101.met
/CER_100100.2012053102
/CER_100100.2012053102.met

Visibility determined by "visibility attribute"

Logical collection of files spread across physical storage resources.

Single Interface to Multiple Clients

 

WebDAV, FUSE, Web UI, Cyberduck

REST, Python, R, Java C++

(And more!)

Federation

 

 

 

 

 

 

 

 

User Profile:

National Institute of Environmental Health Sciences

• Viral Vector Core creates designer viruses:

    request⟶transfection and amplification⟶sample delivery⟶reports

 

• Uses iRODS to combine, organize, and analyze sets of requests and instrument results

   • Produces packaged results in response to researcher requests

   • Quarterly cost reports for chargeback and trend analysis for quality control

User Profile: University College London

• Repository for research data that spans social science, physics, and genomics

• UK sponsored research requirements: last date of access request plus 10 years

• iRODS spans storage technologies and enables federated access from other centres

User Profile: Wellcome Trust Sanger Institute

• Key genomics research centre

• 7 PB of storage managed by iRODS

Rich Metadata

 

attribute: library

attribute: total_reads

attribute: type

attribute: lane

attribute: is_paired_read

attribute: study_accession_number

attribute: library_id

attribute: sample_accession_number

attribute: sample_public_name

attribute: manual_qc

attribute: tag

attribute: sample_common_name

attribute: md5

 

Replication and Federation

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

User Profile: CyVerse

User Profile: Antelope RTS Sensor Integration

iRODS 4.2 and Beyond

In Parallel

• Reference implementations, plugins, clients

 

iRODS 4.2

• Pluggable Rule Engine: Automation in Python, any language; provability

• Cleaner install and configuration

• Continued advancement of enterprise message bus

 

iRODS 4.3

• Maintenance Release

 

iRODS 4.4

• Universal Metadata Access (GQV2)

• Next Generation API

• Pluggable Transport

 

iRODS 5

• Advanced Zone Management

• Plugin Registry