iRODS Overview

 

May, 2016

Agenda

• (Re-)Introduction

 

• What is iRODS?

 

• Getting Started

 

• How iRODS Is Used

 

• Future Directions

 

• Discussion

 

 

Introduction

Jason Coposky

Interim Executive Director

jasonc@renci.org

+1-919-445-9675

 

iRODS is free, open source software owned by a foundation called the iRODS Consortium.

  • Goal is to sustain iRODS as free open source software by:

    ▹ Building good software.  ▹ Growing the iRODS community.  ▹ Demonstrating value.

 

  • Funds a team of 10+ developers, application engineers, documentation, support staff

The iRODS Consortium and Sustainability

Contract Customers

and more ...

Meet the Team

What is iRODS?

          Policy-Based Data Management

The Integrated Rule-Oriented Data System:

• Open Source

• Manages data for indefinite lifespan

• Metadata and event-triggered rules

 

Designed to work with big, important, and/or complex data

 

Example applications:

   • Combining data from multiple sources

   • Annotation and search

   • Tiered storage

   • Preservation

   • Sharing in place

 

 

 

Data

Virtualization

Data

Discovery

Workflow

Automation

Secure

Collaboration

  Data Virtualization

 

 

 

  Data Discovery

 

 

 

  Workflow Automation

 

 

 

  Secure Collaboration

          The Four Pillars

← all your storage in a single namespace

← system and user-generated metadata

← event-driven and scheduled cron  policies

 federation

iRODS Clients

• Web-based and Standalone GUIs

  - iRODS Cloud Browser, MetaLnx, Kanki, Cyberduck

 

• Portals, External Systems

  - iPlant Discovery Environment, Islandora, Fedora Commons

 

• WebDAV for drag-and-drop access built in to the OS

• APIs: Python, REST, Qt, Java, C++

• Command Line Interface

Getting Started

User Group Meeting

June 7-9, 2016

Chapel Hill, NC

ugm2016.irods.org

Initial Trial

  • Google Group
  • Blog posts, social media
  • Cloud images
  • Documentation
  • Training workshops
  • iRODS Hub: The iRODS App Store

Proof of Concept to Pilot

  • Occasional 1-on-1 Support
  • iRODS Consortium Members
  • iRODS Partners
  • iRODS Consortium Service Contracts

Production

  • iRODS Consortium Membership

Building Community and Demonstrating Value

https://irods.org/documentation/

Getting Plugged In with iRODS...

How iRODS Is Used

User Profile: NASA Atmospheric Science Data Center

• 2 PB of archived satellite data

• Publicly available, subsetting on demand

• In-house ingest and archiving software: ANGe (Archive Next Generation)

Virtual Collections

 

ls –l

/CER_100100.2012053100
/CER_100100.2012053100.met
/CER_100100.2012053101
/CER_100100.2012053101.met
/CER_100100.2012053102
/CER_100100.2012053102.met

Visibility determined by "visibility attribute"

Logical collection of files spread across physical storage resources.

Single Interface to Multiple Clients

 

WebDAV, FUSE, Web UI, Cyberduck

REST, Python, R, Java C++

(And more!)

Federation

 

 

 

 

 

 

 

 

User Profile:

National Institute of Environmental Health Sciences

• Viral Vector Core creates designer viruses:

    request⟶transfection and amplification⟶sample delivery⟶reports

 

• Uses iRODS to combine, organize, and analyze sets of requests and instrument results

   • Produces packaged results in response to researcher requests

   • Quarterly cost reports for chargeback and trend analysis for quality control

User Profile: University College London

• Repository for research data that spans social science, physics, and genomics

• UK sponsored research requirements: last date of access request plus 10 years

• iRODS spans storage technologies and enables federated access from other centres

User Profile: Wellcome Trust Sanger Institute

• Key genomics research centre

• 7 PB of storage managed by iRODS

Rich Metadata

 

attribute: library

attribute: total_reads

attribute: type

attribute: lane

attribute: is_paired_read

attribute: study_accession_number

attribute: library_id

attribute: sample_accession_number

attribute: sample_public_name

attribute: manual_qc

attribute: tag

attribute: sample_common_name

attribute: md5

 

Replication and Federation

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

User Profile: CyVerse

User Profile: Antelope RTS Sensor Integration

Future Directions

iRODS 4.2 and Beyond

In Parallel

• Reference implementations, plugins, clients

 

iRODS 4.2

• Pluggable Rule Engine: Automation in Python, any language; provability

• Cleaner install and configuration

• Continued advancement of enterprise message bus

 

iRODS 4.3

• Maintenance Release

 

iRODS 4.4

• Universal Metadata Access (GQV2)

• Next Generation API

• Pluggable Transport

 

iRODS 5

• Advanced Zone Management

• Plugin Registry

 

Discussion

Thank You!

Jason Coposky

jasonc@renci.org

+1-919-445-9675

 

iRODS Overview

By jason coposky

iRODS Overview

Meet the iRODS team. Learn how we're talking about iRODS. Learn what we're doing. Learn where we're going.

  • 1,099
Loading comments...

More from jason coposky