Daniel Moore
Applications Engineer
iRODS Consortium
June 8-11, 2021
iRODS User Group Meeting 2021
Virtual Event
Python iRODS Client 1.0
Python iRODS Client 1.0
Python iRODS Client Library 1.0 - An 8-Year Journey
The original implementation was by Chris LaRose and Michael Gatto at CyVerse (then known as the iPlant Collaborative) in 2013.
The library was contributed in mid-2014 to the newly established iRODS Consortium and was taken over by Antoine de Torcy. Antoine brought the library up to PyPI standards with modern packaging and maintained it through v0.7.0.
Daniel Moore then took the reins in 2018 and has shepherded the code from v0.8.0 to today.
Thank you to the 41 contributors and many users over the years.
Python iRODS Client Library 1.0 - Major Updates
Atomic Metadata
- allows a batch of add/remove operations within one transaction
- increased efficiency over individual operations
- demonstrated up to 750x speedup for large batches
Parallel Transfer
- new client-controlled multi-connect with iRODS release 4.2.9+
- data object put() and get() spawn threads for transfer
- increased efficiency over single-threaded put/get operations
- demonstrated 3-4x speedup
Python iRODS Client Library 1.0 - Other Improvements
XML Protocol Hardening
-
BinBytesBuf for JSON API payloads in iRODS 4.2.9
- corrects cases like AVU searches containing '<', '>', '&'
- Remaining work includes handling control characters in object names
New Checksum Functionality
>>> from irods.test.helpers import make_session
>>> session = make_session()
>>> d = session.data_objects.create("/some/object")
>>> d.chksum()
'sha2:47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU='Python iRODS Client Library 1.0 - Other Improvements
We can now:
- list current ACLs, same as ils -A
- login via PAM/SSL using client environment configuration
- configure connection timeouts
- issue temporary passwords
- manage session cleanup
- query rules/tasks in the delay queue
- automate Python client testing in a Jenkins CI workflow
Python iRODS Client Library - Future Work
- fix known bugs
- prepare to drop support for Python2
- continue working towards full coverage of the iRODS API
Questions?
UGM 2021 - Python iRODS Client 1.0
By iRODS Consortium
