June 8-11, 2021

UGM 2021

Virtual

Jason Coposky

@jason_coposky

Executive Director, iRODS Consortium

Towards a Cloud Native iRODS

Towards a Cloud Native iRODS

General Idea

• iRODS is increasingly deployed in a cloud context
• Automated deployment is currently complicated
• Currently not a great fit for containerized environments
• Client API implementations are entirely too complicated
• State remains the enemy

Step 1: Create a new HTTP Proxy role for the iRODS server

• Client RPC Invocation

  • connects to an iRODS Agent

  • agent re-packs C into xml protocol

  • possibly base64 encode

  • curl request to a configured http endpoint

• This will allow for on premises clients to work with cloud deployments such as

  • icommands

  • nfsrods

  • davrods

• The proxy agents will be responsible for managing persistent information such as L1 descriptors, etc. assuming http requests are atomic in nature

• We will need to re-package operations like READS and WRITES into new cloud friendly API invocations

Step 2: Statically Link Everything

• Build and statically link iRODS and all its dependencies for use by the new API

Markus!

Step 3: The fun part

• Using step 2 and the AWS Lambda Hello World Example:

  https://aws.amazon.com/blogs/compute/introducing-the-c-lambda-runtime/

• Fetch configuration from the configuration service

• Capture the proxy request

• Possibly base64 decode

• Send the xml requests through the existing iRODS machinery.

• The catalog will be hosted as a separate service, e.g. RDS
• The new executable assumes it is the Catalog Service Provider
• All resources assumed to operate in 'detached_mode': S3 and UFS

Assumptions

Step 4: Towards the glorious future

• The experimental iRODS API is all driven by JSON, which lends itself well to lambda-like deployment

• The iRODS CLI and newer clients can migrate gradually to the new API

• The addition of  read-some/write-some style API  interface without the need for persistence will allow new clients to work directly with an http style API

Questions?