EPA / RENCI
Demo and Discussion

Terrell Russell, Ph.D.
@terrellrussell
Senior Data Scientist, iRODS Consortium

May 2, 2016
Renaissance Computing Institute
UNC-Chapel Hill
Overview
iRODS is open source
data management software.
What is iRODS?


Storage Virtualization: iRODS is Middleware

User Application

"Logical" Layer






Storage Environment
"Physical" Layer
storagecluster.example.org:/managed
s3.amazonaws.com:/example/bitbucket
Storage Virtualization: iRODS Clients
• Web-based and Standalone GUIs
- iRODS Cloud Browser, MetaLnx, Kanki, Cyberduck
• Portals, External Systems
- iPlant Discovery Environment, Islandora, Fedora Commons
• WebDAV for drag-and-drop access built in to the OS
• APIs: Python, REST, Qt, Java, C++
• Command Line Interface


"Logical" Layer
"Physical" Layer
storagecluster.example.org:/managed/Vault/home/alice/training_jpgs/clown_fish.jpg
/tempZone/home/alice/training_jpgs/clown_fish.jpg

s3.amazonaws.com:/example/bitbucket/Vault/home/alice/training_jpgs/clown_fish.jpg
Storage Virtualization: Objects and Collections


Storage Virtualization: iRODS is Distributed

Storage Virtualization: Composable Resources

replResc
demoResc
s3Resc
archiveResc
cacheResc
Data Discovery

Attribute: filename | Value: seal.jpg
Attribute: animal | Value: seal
Attribute: photo_color | Value: gray and brown
Attribute: file_size | Value: 362833 | Units: bytes


acPostProcForPut {
if ($filePath like "*.jpg" || $filePath like "*.jpeg" || $filePath like "*.bmp" || $filePath like "*.tif" || $filePath like "*.tiff" || $filePath like "*.rif" || $filePath like "*.gif" || $filePath like "*.png" || $filePath like "*.svg" || $filePath like "*.xpm") {
msiget_image_meta($filePath, *meta);
msiString2KeyValPair(*meta, *meta_kvp);
msiAssociateKeyValuePairsToObj(*meta_kvp, $objPath, "-d");
} # if
} # acPostProcForPut
Policy Enforcement Points = Triggers
Microservices = Actions
Workflow Automation: Rules
↗
↘



Secure Collaboration: Federation
→
↗
↙
↖
↘




←


Some History




The Team


Static Analysis and Continuous Integration

https://jenkins.irods.org
Initial Trial
- Google Group
- Blog posts, social media
- Cloud images
- Documentation
- Training workshops
- iRODS Hub: The iRODS App Store
Proof of Concept to Pilot
- Occasional 1-on-1 Support
- iRODS Consortium Members
- iRODS Partners
- iRODS Consortium Service Contracts
Production
- iRODS Consortium Membership
Building Community and Demonstrating Value
https://irods.org/documentation/
Getting Plugged In with iRODS...

-
Goal is to sustain iRODS as free open source software by:
▹ Building good software. ▹ Growing the iRODS community. ▹ Demonstrating value.
-
Funds a team of 10+ developers, application engineers, documentation, support staff









The iRODS Consortium and Sustainability


Contract Customers
and more ...




Storage Virtualization
Data Discovery
Workflow Automation
Secure Collaboration
The Four Pillars
← all your storage in a single namespace
← system and user-generated metadata
← event-driven and scheduled cron → policies
← federation




Building Good Software: Plugins




• Microservices
• Storage Resources
• Authentication
• Network
• Database
• API
• Rule Engine (iRODS 4.2)
• Transport (iRODS 4.3)
Modern, modular, maintainable code
Thank You!
irods.org
Terrell Russell
tgr@renci.org
EPA / RENCI Demo
By iRODS Consortium
EPA / RENCI Demo
An Overview of iRODS and the Consortium.
- 1,976