Design Patterns in an Open Source Community

Terrell Russell, Ph.D.

@terrellrussell

Chief Technologist, iRODS Consortium

Design Patterns in an Open Source Community

October 24, 2017

All Things Open 2017

Raleigh, NC

Why Data Management Matters

As data matures and reaches a broader community, data management policy must also evolve to meet these additional requirements.

iRODS is

  • Open Source
  • Distributed
  • Data Centric
  • Metadata Driven

 

A flexible framework for the abstraction of infrastructure

iRODS as the Integration Layer

History

20+ year legacy

  • 10 years of federal funding for grid storage research
  • 10 years of federal funding for policy engine research
  • iRODS Consortium founded in 2013

Our Membership

Community Driven

Input from the Open Source Community

  • Support Requests
  • Community Feedback
  • Working Groups
  • Use Cases
  • Proofs of Concept

 

All with the Expectation of Public Discourse and Disclosure

Discovered a common enabling practice...

(aka metadata)

Annotation with meaning

Annotation is both descriptive and prescriptive.

 

It is useful

  • for discovery of the past and the present
  • to direct the future

 

Metadata Everywhere

With the appropriate abstractions, everything in a system can be described with metadata and therefore, all actions within a system can be driven by that metadata.

 

Metadata Driven Patterns:

  • Good Metadata (Templates)
  • Landing Zone / Ingest
  • Replication
  • Tiering
  • Archiving
  • Auditing
  • Data to Compute
  • Compute to Data

 

Metadata Templates

iRODS Capabilities

From Prototype to Production

Provenance and Reporting

Data to Compute Pattern

Compute to Data Pattern

An open community-driven process

  • is hard
  • is slow

 

But, it also

  • sets clear expectations
  • generates a shared language
  • produces a strong culture
  • produces a better 'product'
  • is worth it

Discovering Design Patterns

Thank you!

 

 

 

Terrell Russell, Ph.D.

@terrellrussell

Chief Technologist, iRODS Consortium

Design Patterns in an Open Source Community

By iRODS Consortium

Design Patterns in an Open Source Community

  • 1,834