S3 Resource Plugin Updates

S3 vendors certified as compatible with iRODS:
 

  • Wasabi S3
  • Fujifilm
  • Oracle Cloud

 

Bug Fixes/Enhancements:

  • Streaming of a file from local cache
    • Reorganized code so that each thread can handle more than one part
      • Allows better handling of part timeouts
      • Reduces the part size so that the 2 minute timeout isn't triggered on large files.
      • Allowed streaming of files from cache that are larger than 50 GiB.
  • Fixed a bug where the catalog consumer thought it was the provider when detached mode was enabled.
  • Fixed 4 MiB to 32 MiB file replication failures due to unexpected number of transfer threads.

Globus Changes

Two issues were fixed when testing Globus.

 

45 - Update time not being updated due to the thread that closes the file is not writing any data.  Reorganized the code to make sure this thread takes part in the writing of data.

50 - We were getting conflicts in some environments due to duplicate base64_encode and base64_decode methods being defined.  These methods were moved into the Globus code with namespaces.  (The iRODS core code was also updated later to namespace these methods.)
 

S3 Globus Updates

By justinkylejames

S3 Globus Updates

S3 and Globus Updates

  • 275