The iRODS Roadmap

4.2 and Beyond

November 14, 2016
Supercomputing 2016

Salt Lake City, Utah

Jason Coposky

@jason_coposky

Executive Director, iRODS Consortium

iRODS 4.2 Progress

  • Lots of commits
    • 951 commits
    • 100 enhancements
    • 69 bugs

 

  • Lots of coordination
    • three major independent branches
    • merging is complete

iRODS 4.2 Progress

  • Lots of testing
    • 12 OS and DB combinations
    • upgrades
    • topology
    • federation
    • plugins

iRODS 4.2 Progress

Releasing Right Now!

iRODS 4.2 Major Features

  • First Class API Plugins
    • dynamic PEPs for every operation

 

  • Pluggable Rule Engine
    • full serialization into/out of rule languages

 

  • Messaging Framework
    • audit plugin, emits AMQP

iRODS 4.2 Architectural Changes

  • Clang and C++14 across all platforms
  • CMake
  • All control scripts in python
  • Packages separated, cleaner dependencies
  • Available via APT/YUM repositories

iRODS 4.2

Packages Available Now

 

          https://packages.irods.org

 

 

 

 

 

 

Questions?

The Future - iRODS 4.3

Nothing scary will happen

  • hardening of 4.2
  • testing
  • backlog of issues

Developed In Parallel as Plugins

  • Reference Implementations
    • audit plugin (done)
    • indexing
    • metadata storage
  • Multipart Data Objects
    • improve transport speeds (parallel and/or multisource)
    • cache-free object storage (S3, WOS, etc.)
    • reliable restarts
    • 8th plugin interface - pluggable transport
  • GenQuery v2 (GQv2)

Multipart Data Objects

Data Objects fractured into parts - still proper data objects

  • aggregated under a high level data objects within iRODS
  • small parts provide partial read/write capability
  • easily streamed and restarted given registration

The Big Picture of the Future

4.2 was about sending things out

  • Dynamic PEPs
  • Pluggable Rule Engine
  • Enterprise Messaging

4.3 (and friends) will be about bringing them back

  • Facilitated by GenQuery version 2      

GenQuery vs GenQuery version 2

iRODS 4.2 - The Scatter

iRODS 4.3 with GQv2 - Gather

Provide a unified query interface for all services leveraged by iRODS

Federation and Delegated Services

Access to virtualized metadata via Federation

 - exactly like virtualized storage, but with ontologies

Questions?

The iRODS Consortium - The Mission

To sustain the iRODS Ecosystem by:

  • Writing Good Software
  • Growing the Community
  • Showing Value

Staffing

Development Team:

        Terrell, Antoine, Zoey, Ben, Rick

User Interface / Application Engineer:

        Cesar, Justin, New Hire

Management / Support:

        Bonnie, Dawn, Stephanie, Jason

Currently 15 Members

Growing The Community

Last 12 months:

  • 5 new consortium members
  • 113 new members of irods-chat
  • 199 new Twitter followers
  • Various Proofs of Concept - Future Members
  • Expanding engagement at conferences and workshops

In The Next Year

Continue the mission:

  • Write good software
    • iRODS 4.2 and 4.3
  • Expand the community
    • Further PoCs
    • Conferences, Workshops
    • Marketing & Case Studies
  • Stakeholder Value
    • One-on-One technical support
    • Training
    • Proofs of Concept
    • Co-Marketing & Sales

Consortium Membership - Touch Benefits

Public Support of iRODS

  • Priority access to an audience of over 1,000 iRODS sites
  • Direct interaction with Consortium members in the IT, Bio/Pharma, Geosciences, and University Research industries

 

Co-Marketing/Co-Sales Opportunities

  • Joint press releases and product announcements
  • Joint conference participation (e.g., talks at SEG, SC, BioIT World)
  • Assistance on sales calls by web conference or in person
  • The Consortium makes introductions as a neutral third party
  • Discounts on iRODS event sponsorship

Consortium Membership - The User Group Meeting

  • Hosted in Chapel Hill, NC in 2016
  • One day of training, two days of case study/technology presentations
  • Growing annual attendance
  • Members receive one free registration
  • Sponsorship/presentation opportunities available

Consortium Membership - Technical Assistance

Direct access to senior data management researchers and highly skilled engineers

 

Joint Technology Development

  • Far-field technology planning
  • Implementation discussions for specific opportunities
  • Directed open- and closed-source feature development

 

Technical Assistance

  • Phone and email support                                                      
  • Training for internal staff and customers

 

Tier 3 Support Agreements

Consortium Membership - Committees

Technology Working Group

  • Monthly web conferences
  • Discusses iRODS technology development                                                        

 

Planning Committee

  • Monthly web conferences
  • Discusses iRODS Consortium policy, business practices
  • Votes on technology roadmap

 

Executive Board

  • Meets twice yearly
  • Votes on Consortium budget and structural changes
  • Determines the thematic priorities of the Consortium                                    

 

Other Committees formed ad hoc to address specific technology or governance issues

Consortium Membership - Tiers

SC16 - iRODS Roadmap and Beyond

By iRODS Consortium

SC16 - iRODS Roadmap and Beyond

  • 1,992