June 8-11, 2021
iRODS User Group Meeting 2021
Virtual Event
Markus Kitsinger
github.com/SwooshyCueb
Software Engineer, iRODS Consortium
iCommands Userspace
Packaging
It is common for shared systems (such as HPC environments) to be pretty locked down. Being able to use the iCommands in these environments without jumping through a bunch of hoops would be incredibly convenient, and is something that the community has asked for.
iCommands Userspace Packaging - Overview
LD_LIBRARY_PATH
or other environment variables/opt/irods-externals
)Goals and Needs
Our Solution - Overview
Our Solution - Step-by-step - Phase 0: Initial Preparation
cmake_install.cmake
script for the iCommands build is invoked directly with CMake variables set to install the iCommands to a temporary prefix.Our Solution - Step-by-step - Phase 1: Gathering iRODS components
strip
tool, ldd
, and the LIEF Python module.
strip
tool is not found, this entire process is a no-op.strip
tool is found, but LIEF or ldd
is not found (or not usable), the second and third steps of this process do not take place.strip
tool.ldd
is used to determine which (if any) libraries that are imported by the binary are not used.Our Solution - Step-by-step - Strip and clean process
/opt/irods-externals
) and distro-provided.Our Solution - Step-by-step - Phase 2: Gathering external libraries
ldd
is used to get the locations of all dependency libraries of the gathered iRODS components.
readelf
tool (or LIEF, should one not be available).
Our Solution - Step-by-step - Phase 2: Gathering external libraries
If LIEF was not found or is not usable, an alternate process is used:
ldd
is used to get the locations of all dependency libraries of the gathered iRODS components.
Our Solution - Step-by-step - Phase 2: Gathering external libraries - Alternate
In order to eliminate the need to set the LD_LIBRARY_PATH
environment variable, we need to set the RUNPATH
or RPATH
in as many of the binaries as possible.
.
or empty, the packager attempts to set the binary's RUNPATH
/RPATH
to $ORIGIN
*.RUNPATH
/RPATH
to $ORIGIN/
* + the derived relative path
* $ORIGIN
is a substitution string in RUNPATHs
and RPATHs
for the directory in which the library or executable resides.
Our Solution - Step-by-step - Phase 3: Setting RUNPATH
/RPATH
RUNPATH
/RPATH
of a binary:
DF_ORIGIN
flag, which helps keep the runtime linker from misbehaving. It can also add a RUNPATH
to a binary that does not arleady have an RUNPATH
or RPATH
. However, it can also cause the entire packager to segfault, and has the potential to bungle up the resultant binaries.chrpath
can change the RPATH
dynamic entry to a RUNPATH
dynamic entry.chrpath
is prioritized over CMake. Whether LIEF is given top or bottom priority depends on a number of variables.Our Solution - Step-by-step - Phase 3: Setting RUNPATH
/RPATH
- The Tools
tar
tool that automatically compresses based on output filename.
Our Solution - Step-by-step - Phase 4: Creating Final Package
strip_util
and runpath_util
modules double as functional command line utilities that can be executed directly if needed.Our Solution - Other fun details
RUNPATH
/RPATH
shenanigans and the DF_ORIGIN
flagChallenges
objdump
as an alternative to readelf
.RUNPATH
/RPATH
with CMake on Centos.readelf
/objdump
to make stripping and RUNPATH
/RPATH
setting smarter.ldd
.TODO