EPA / RENCI
Demo and Discussion
Terrell Russell, Ph.D.
@terrellrussell
Senior Data Scientist, iRODS Consortium
May 2, 2016
Renaissance Computing Institute
UNC-Chapel Hill
Overview
iRODS is open source
data management software.
What is iRODS?
Storage Virtualization: iRODS is Middleware
User Application
"Logical" Layer
Storage Environment
"Physical" Layer
storagecluster.example.org:/managed
s3.amazonaws.com:/example/bitbucket
Storage Virtualization: iRODS Clients
• Web-based and Standalone GUIs
- iRODS Cloud Browser, MetaLnx, Kanki, Cyberduck
• Portals, External Systems
- iPlant Discovery Environment, Islandora, Fedora Commons
• WebDAV for drag-and-drop access built in to the OS
• APIs: Python, REST, Qt, Java, C++
• Command Line Interface
"Logical" Layer
"Physical" Layer
storagecluster.example.org:/managed/Vault/home/alice/training_jpgs/clown_fish.jpg
/tempZone/home/alice/training_jpgs/clown_fish.jpg
s3.amazonaws.com:/example/bitbucket/Vault/home/alice/training_jpgs/clown_fish.jpg
Storage Virtualization: Objects and Collections
Storage Virtualization: iRODS is Distributed
Storage Virtualization: Composable Resources
replResc
demoResc
s3Resc
archiveResc
cacheResc
Data Discovery
Attribute: filename | Value: seal.jpg
Attribute: animal | Value: seal
Attribute: photo_color | Value: gray and brown
Attribute: file_size | Value: 362833 | Units: bytes
acPostProcForPut {
if ($filePath like "*.jpg" || $filePath like "*.jpeg" || $filePath like "*.bmp" || $filePath like "*.tif" || $filePath like "*.tiff" || $filePath like "*.rif" || $filePath like "*.gif" || $filePath like "*.png" || $filePath like "*.svg" || $filePath like "*.xpm") {
msiget_image_meta($filePath, *meta);
msiString2KeyValPair(*meta, *meta_kvp);
msiAssociateKeyValuePairsToObj(*meta_kvp, $objPath, "-d");
} # if
} # acPostProcForPut
Policy Enforcement Points = Triggers
Microservices = Actions
Workflow Automation: Rules
↗
↘
Secure Collaboration: Federation
→
↗
↙
↖
↘
←
Some History
The Team
Static Analysis and Continuous Integration
https://jenkins.irods.org
Initial Trial
- Google Group
- Blog posts, social media
- Cloud images
- Documentation
- Training workshops
- iRODS Hub: The iRODS App Store
Proof of Concept to Pilot
- Occasional 1-on-1 Support
- iRODS Consortium Members
- iRODS Partners
- iRODS Consortium Service Contracts
Production
- iRODS Consortium Membership
Building Community and Demonstrating Value
https://irods.org/documentation/
Getting Plugged In with iRODS...
-
Goal is to sustain iRODS as free open source software by:
▹ Building good software. ▹ Growing the iRODS community. ▹ Demonstrating value.
-
Funds a team of 10+ developers, application engineers, documentation, support staff
The iRODS Consortium and Sustainability
Contract Customers
and more ...
Storage Virtualization
Data Discovery
Workflow Automation
Secure Collaboration
The Four Pillars
← all your storage in a single namespace
← system and user-generated metadata
← event-driven and scheduled cron → policies
← federation
Building Good Software: Plugins
• Microservices
• Storage Resources
• Authentication
• Network
• Database
• API
• Rule Engine (iRODS 4.2)
• Transport (iRODS 4.3)
Modern, modular, maintainable code
Thank You!
irods.org
Terrell Russell
tgr@renci.org
EPA / RENCI Demo
By iRODS Consortium
EPA / RENCI Demo
An Overview of iRODS and the Consortium.
- 1,837