iRODS S3 API:

User and Bucket Mapping, Presigned URLs, and Dataverse

Alan King

Senior Software Developer

iRODS Consortium

June 30 - July 02, 2026

iRODS User Group Meeting 2026

Barcelona, Spain

Overview

  • Motivation / Goals
  • Architecture / Implementation
  • History
  • Release updates
    • 0.4.0
    • 0.5.0
    • 0.6.0
  • Demo
  • Future Work

Motivation / Goals

  • Present iRODS as the S3 protocol
  • Multi-user / Multi-bucket
  • Load Balancer friendly
  • Maintainable
  • Performance similar to other S3 backends

Architecture / Implementation

  • Single binary
  • Single configuration file
  • Shipped as Docker image
  • Multi-user / Multi-bucket
  • Requires rodsadmin credentials

History

Aug 2021

S3 Working Group

Nov 2023

0.1.0

Mar 2024

0.2.0

Oct 2024

0.3.0

Aug 2025

0.4.0

Nov 2025

0.5.0

Jun 2026

0.6.0

Nov 2024

Last S3 API talk

Mar 2026

S3 Working Group ends

0.4.0 (August 2025)

  • Added iRODS 5 compatibility
  • Server configuration validated on startup
  • Added presigned URL download and upload support
    • Time-limited, anonymous data access
    • Enables large-file storage for Dataverse in iRODS

0.5.0 (November 2025)

Added plugin architecture for user and bucket mapping.

  • Modeled after HTTP API 0.5.0 user mapping changes
  • Enables dynamic user / bucket mapping
    • Adjust mappings without restarting S3 API server
  • Mapping configurations separated from main config file
    • Currently supports local JSON file configs

0.6.0 (June 2026)

  • Basic support for popular Windows-based S3 GUIs
    • Tested with S3 Browser and Cyberduck
    • Create, list, and delete empty prefixes
    • Implemented ListObjects(V1) and some stub endpoints
  • Automated build-and-test workflows across iRODS versions

Demo

  • Stand up S3 API 0.6.0 and iRODS 5.0.2 using irods_demo
  • Upload data to iRODS with Cyberduck via S3 API
  • Anonymous download with presigned URL from Cyberduck

Future Work

  • Additional S3 capabilities / endpoints:
    • Object and bucket tagging
    • ACLs
    • Multipart management
  • Virtual Hosting-style addressing
  • Additional authentication schemes (irods, PAM)
  • Performance improvements (multipart)

Thanks for listening

iRODS UGM 2026 - iRODS S3 API

By Alan King

iRODS UGM 2026 - iRODS S3 API

  • 52