South Big Data Hub

Data Sharing and Infrastructure Working Group

 

iRODS Demo and Discussion

September 30, 2016

Renaissance Computing Institute (RENCI)

Chapel Hill, NC

Terrell Russell, Ph.D.

@terrellrussell

Acting Chief Technologist, iRODS Consortium

Overview

iRODS is open source

data management software.

The iRODS Consortium and Sustainability

  • Goal is to sustain iRODS as free open source software by:

         ▹ Building good software.  ​▹ Growing the iRODS community.  ▹ Demonstrating value.

 

  • Funds a team of 10+ developers, application engineers, documentation, support staff

Contract Customers

and more...

The Four Pillars

Storage Virtualization

Data Discovery

Workflow Automation

Secure Collaboration

all your storage in a single namespace

← system and user-generated metadata

← event-driven and scheduled cron policies

← federation

Building Good Software

Modular, Maintainable Code

 

Modern

  • C++14
  • Clang
  • CMake

 

Plugin Interfaces

  • Microservices
  • Storage Resources
  • Authentication
  • Network
  • Database
  • API
  • Rule Engine
  • Transport (iRODS 4.3)

Agile Software Development

 

Testing

  • Continuous Integration
  • Topology
  • Federation

 

Retrospectives

  • Liked
  • Learned
  • Lacked
  • Longed For

Use Cases

The Wellcome Trust Sanger Institute

  • single-sign on, durability, HPC, metadata

 

National Institute of Environmental Health Sciences

  • regularized workflow, reports, billing

 

NASA Atmospheric Science Data Center

  • storage migration, publication, federation

 

University College London

  • tiering, retention

The Wellcome Trust Sanger Institute

Sanger - Federation

National Institute of Environmental Health Sciences

  • Creating designer viruses:
    • request > transfection and amplification > QC > sample delivery > reports
  • Uses iRODS to combine, organize, and analyze sets of requests and instrument results
  • Produces packaged results in response to researcher requests
  • Quarterly cost reports for chargeback and trend analysis for quality control

NASA Atmospheric Science Data Center

University College London

  • Repository for research data that spans social science to physics to genomics
  • UK sponsored research requirements: last date of access request plus 10 years
  • iRODS spans data storage technologies and enables federated access from other centers

Demo

Command Line Interface iCommands
Desktop Clients Kanki, Cyberduck
Web Clients Cloud Browser, MetaLnx
File System Integration Davrods, NFS-RODS, Fuse
Client Libraries C/C++, Java, Python, R

Thank you!

irods.org

 

 

Terrell Russell

tgr@renci.org

South BD Hub - Demo and Discussion

By iRODS Consortium

South BD Hub - Demo and Discussion

  • 267
Loading comments...

More from iRODS Consortium