June 8-11, 2021
UGM 2021
Virtual
Jason Coposky
@jason_coposky
Executive Director, iRODS Consortium
Towards a Cloud Native iRODS
Towards a Cloud Native iRODS
General Idea
- iRODS is increasingly deployed in a cloud context
- Automated deployment is currently complicated
- Currently not a great fit for containerized environments
- Client API implementations are entirely too complicated
- State remains the enemy
Step 1: Create a new HTTP Proxy role for the iRODS server
-
Client RPC Invocation
-
connects to an iRODS Agent
-
agent re-packs C into xml protocol
-
possibly base64 encode
-
curl request to a configured http endpoint
-
-
This will allow for on premises clients to work with cloud deployments such as
-
icommands
-
nfsrods
-
davrods
-
-
The proxy agents will be responsible for managing persistent information such as L1 descriptors, etc. assuming http requests are atomic in nature
-
We will need to re-package operations like READS and WRITES into new cloud friendly API invocations
Step 2: Statically Link Everything
-
Build and statically link iRODS and all its dependencies for use by the new API
Markus!
Step 3: The fun part
-
Using step 2 and the AWS Lambda Hello World Example:
https://aws.amazon.com/blogs/compute/introducing-the-c-lambda-runtime/
-
Fetch configuration from the configuration service
-
Capture the proxy request
-
Possibly base64 decode
-
Send the xml requests through the existing iRODS machinery.
- The catalog will be hosted as a separate service, e.g. RDS
- The new executable assumes it is the Catalog Service Provider
- All resources assumed to operate in 'detached_mode': S3 and UFS
Assumptions
Step 4: Towards the glorious future
-
The experimental iRODS API is all driven by JSON, which lends itself well to lambda-like deployment
-
The iRODS CLI and newer clients can migrate gradually to the new API
-
The addition of read-some/write-some style API interface without the need for persistence will allow new clients to work directly with an http style API
Questions?