Design Patterns in an Open Source Community

Terrell Russell, Ph.D.

@terrellrussell

Chief Technologist, iRODS Consortium

Design Patterns in an Open Source Community

October 24, 2017

All Things Open 2017

Raleigh, NC

Why Data Management Matters

As data matures and reaches a broader community, data management policy must also evolve to meet these additional requirements.

iRODS is

• Open Source
• Distributed
• Data Centric
• Metadata Driven

A flexible framework for the abstraction of infrastructure

iRODS as the Integration Layer

History

20+ year legacy

• 10 years of federal funding for grid storage research
• 10 years of federal funding for policy engine research
• iRODS Consortium founded in 2013

Our Membership

Community Driven

Input from the Open Source Community

• Support Requests
• Community Feedback
• Working Groups
• Use Cases
• Proofs of Concept

All with the Expectation of Public Discourse and Disclosure

Discovered a common enabling practice...

(aka metadata)

Annotation with meaning

Annotation is both descriptive and prescriptive.

It is useful

• for discovery of the past and the present
• to direct the future

Metadata Everywhere

With the appropriate abstractions, everything in a system can be described with metadata and therefore, all actions within a system can be driven by that metadata.

Metadata Driven Patterns:

• Good Metadata (Templates)
• Landing Zone / Ingest
• Replication
• Tiering
• Archiving
• Auditing
• Data to Compute
• Compute to Data

Metadata Templates

iRODS Capabilities

From Prototype to Production

Provenance and Reporting

Data to Compute Pattern

Compute to Data Pattern

An open community-driven process

• is hard
• is slow

But, it also

• sets clear expectations
• generates a shared language
• produces a strong culture
• produces a better 'product'
• is worth it

Discovering Design Patterns

Thank you!

Terrell Russell, Ph.D.

@terrellrussell

Chief Technologist, iRODS Consortium