iRODS S3 API

Alan King

Senior Software Developer

iRODS Consortium

June 30 - July 02, 2026

iRODS User Group Meeting 2026

Barcelona, Spain

Overview

  • Motivation / Goals
  • Architecture / Implementation
  • History
  • Release updates
    • 0.4.0
    • 0.5.0
    • 0.6.0
  • Future Work

Motivation / Goals

  • Present iRODS as the S3 protocol
  • Multi-user / Multi-bucket
  • Load Balancer friendly
  • Maintainable
  • Performance similar to other S3 backends

Architecture / Implementation

  • Single binary
  • Single configuration file
  • Shipped as Docker image
  • Multi-user / Multi-bucket
  • Requires rodsadmin credentials

History

Aug 2021

S3 Working Group

Nov 2023

0.1.0

Mar 2024

0.2.0

Oct 2024

0.3.0

Aug 2025

0.4.0

Nov 2025

0.5.0

Jun 2026

0.6.0

Nov 2024

Last S3 API talk

Mar 2026

S3 Working Group ends

0.4.0 (August 2025)

  • Added iRODS 5 compatibility
  • Server configuration validated on startup
  • Added presigned URL support
    • Time-limited, anonymous data access
    • Enables interoperability between Dataverse and iRODS

0.5.0 (November 2025)

  • Added plugin architecture for user mapping and bucket mapping
    • Modeled after HTTP API 0.5.0 user mapping changes
    • Adjust mappings without restarting S3 API server
    • Mapping configurations separated from main config file
      • Currently supports local JSON file configs
      • Enables dynamic user / bucket mapping (in the future)

0.6.0 (June 2026)

  • Basic support for popular Windows-based S3 GUIs
    • Tested with S3 Browser and Cyberduck
    • Create, list, and delete empty prefixes
    • Implemented ListObjects(V1) and some stub endpoints
  • Automated build-and-test workflows across iRODS versions

Future Work

  • Dynamic user mapping
  • Dynamic bucket mapping
  • Additional endpoints:
    • Tagging
    • ACLs
    • Multipart management
  • Virtual Hosting-style addressing
  • Additional authentication schemes (irods, PAM)
  • Performance improvements

Thanks for listening

Made with Slides.com