April 1-3, 2019
72nd HPC User Forum
Santa Fe, New Mexico
Terrell Russell, Ph.D.
@terrellrussell
Chief Technologist, iRODS Consortium
Metadata and Archiving
at Scale
Metadata and Archiving
at Scale
Our Membership
Data Centric. Metadata Driven.
Provides insurance against your changing infrastructure:
Open Source Data Management
Data Management
"The development, execution, and supervision of plans, policies, programs, and practices that control, protect, deliver, and enhance the value of data and information assets."
Most organizations are still managing their assets with a collection of small scripts, tribal knowledge, vigilance, and hope.
Organizations, instead, need a future-proof solution to managing data and its surrounding infrastructure.
Why Data Management Matters
As data matures and reaches a broader community, data management policy must also evolve to meet these additional requirements.
The Data Lifecycle begins at Data Generation
When data management is involved from the point of data generation,
a system can address other hard problems:
A Small Matter of Policy
Two Simplified Assertions for Today:
Both can be handled abstractly through configuration and policy.
Automatic, policy-based solutions are resilient to future changes in technology.
iRODS Core Competencies
The underlying technology categorized into four areas
iRODS Policy Examples
iRODS Capabilities
Deployment Patterns
Data to Compute
Compute to Data
Filesystem Synchronization
The Data Management Model
Automated Ingest - Landing Zone
Automated Ingest - Filesystem Scanning
Storage Tiering
Data to Compute
Compute to Data
Take Aways
Ongoing and Upcoming Work
Resources
iRODS Open Source Code
iRODS Overview and Diagrams
https://irods.org/documentation
iRODS Software Documentation
iRODS Training Materials and Presentations
iRODS User Group Meeting
Questions?
Thank you.
iRODS Consortium
@irods
Terrell Russell, Ph.D.
@terrellrussell