Alan King
Senior Software Developer
iRODS Consortium
July 6, 2022
iRODS User Group Meeting 2022
KU Leuven, Belgium
iRODS Development and Testing Environment
iRODS Development and Testing Environment
Overview
Motivation
Tests should be meaningful and consistent.
Running tests should be reliable and accessible.
Passing tests build confidence in the changes developers make to software, asserting the correctness of the changes and of the entire system.
History of Build-and-Test Systems at iRODS
Limitations of the past
v8 - 2021/2022
iRODS Development Environment
https://github.com/irods/irods_development_environment
Set of Dockerized applications used to build, run, and debug packages for the iRODS server, iCommands, and plugins.
iRODS Testing Environment
https://github.com/irods/irods_testing_environment
Scripts built on python library which uses Docker Compose to stand up iRODS zones in various configurations and run tests.
Basic Build-and-Test Workflow
source code
↓
↓
packages
↓
↓
test results
irods_testing_environment
irods_development_environment
iRODS Development Environment
Features:
Advantages:
Basic Build-and-Test Workflow
source code
↓
↓
packages
↓
↓
test results
irods_testing_environment
irods_development_environment
iRODS Testing Environment
Features:
Advantages:
iRODS Testing Environment
$ python run_core_tests.py --project-directory projects/ubuntu-20.04/ubuntu-20.04-postgres-10.12 \
--irods-package-directory ~/hdd/builds/irods_packages/4-3-stable/ubuntu-20.04 \
--concurrent-test-executor-count 4 -v
<snip>
2022-06-23 15:38:30,381 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_2]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_irm', '--xml_output']]
2022-06-23 15:39:12,397 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_4]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_ireg', '--xml_output']]
2022-06-23 15:39:17,437 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_4]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_irmtrash', '--xml_output']]
2022-06-23 15:40:43,075 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_4]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_iscan', '--xml_output']]
2022-06-23 15:41:27,853 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_4]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_itouch', '--xml_output']]
2022-06-23 15:42:17,991 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_4]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_iuserinfo', '--xml_output']]
2022-06-23 15:42:21,349 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_4]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_negotiation', '--xml_output']]
2022-06-23 15:42:49,652 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_4]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_quotas', '--xml_output']]
2022-06-23 15:44:15,902 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_3]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_resource_types.Test_Resource_CompoundWithMockarchive', '--xml_output']]
2022-06-23 15:45:35,288 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_1]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_all_rules', '--xml_output']]
2022-06-23 15:49:06,316 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_2]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_irsync', '--xml_output']]
2022-06-23 15:49:19,821 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_2]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_isysmeta', '--xml_output']]
2022-06-23 15:49:39,938 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_2]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_itrim', '--xml_output']]
2022-06-23 15:49:46,930 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_1]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_control_plane', '--xml_output']]
2022-06-23 15:49:52,223 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_2]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_load_balanced_suite', '--xml_output']]
2022-06-23 15:49:55,635 ERROR - [ubuntu-2004-postgres-1012_irods-catalog-provider_2]: cmd succeeded [['python3', '/var/lib/irods/scripts/run_tests.py', '--run_specific_test', 'test_prep_genquery_iterator', '--xml_output']]
<snip>
2022-06-23 18:19:10,572 ERROR - ==== begin test run results ====
-----
results for [ubuntu-2004-postgres-1012_irods-catalog-provider_1]
passed tests:
[[1697.3373]s] [test_all_rules]
[[ 251.6414]s] [test_control_plane]
[[ 89.0497]s] [test_ibun]
[[3715.5025]s] [test_icommands_file_operations]
[[ 14.8783]s] [test_igroupadmin]
[[ 18.9958]s] [test_ilsresc]
[[ 26.3037]s] [test_imeta_interactive]
[[ 30.7978]s] [test_ipasswd]
[[ 5.0171]s] [test_ipwd]
[[1463.7713]s] [test_irepl]
[[ 9.3546]s] [test_irodsctl]
[[ 58.6852]s] [test_istream]
[[ 35.5464]s] [test_itree]
[[ 7.3381]s] [test_izonereport]
[[ 261.9476]s] [test_native_rule_engine_plugin]
[[ 14.3952]s] [test_resource_configuration]
[[ 837.4747]s] [test_resource_types.Test_Resource_Deferred]
[[ 960.3886]s] [test_resource_types.Test_Resource_Random]
[[1379.2288]s] [test_resource_types.Test_Resource_ReplicationToTwoCompound]
[[ 27.3775]s] [test_resource_types.Test_Resource_WeightedPassthru]
[[ 7.5812]s] [test_special_collections]
skipped tests:
failed tests:
return code:[0]
time elapsed: [1.091e+04]seconds ([ 3]hours [ 1.877]minutes)
-----
-----
results for [ubuntu-2004-postgres-1012_irods-catalog-provider_2]
passed tests:
[[ 3.2683]s] [test_auth]
[[ 385.5667]s] [test_delay_queue]
[[ 13.5900]s] [test_icd]
[[ 178.9789]s] [test_icp]
[[ 4.5107]s] [test_ihelp]
[[ 27.5451]s] [test_imeta_admin_mode]
[[ 273.2416]s] [test_imeta_set]
[[ 362.5961]s] [test_iphymv]
[[ 5.4457]s] [test_iqmod]
[[ 17.6846]s] [test_irm]
[[ 635.9353]s] [test_irsync]
[[ 13.5050]s] [test_isysmeta]
[[ 20.1163]s] [test_itrim]
[[ 12.2845]s] [test_load_balanced_suite]
[[ 3.4121]s] [test_prep_genquery_iterator]
[[1037.5428]s] [test_resource_types.Test_Resource_Compound]
[[ 899.7080]s] [test_resource_types.Test_Resource_MultiLayered]
[[ 965.3594]s] [test_resource_types.Test_Resource_RandomWithinRandom]
[[1485.8112]s] [test_resource_types.Test_Resource_ReplicationToTwoCompoundResourcesWithPreferArchive]
[[ 66.3305]s] [test_rule_engine_plugin_framework]
[[ 23.2960]s] [test_ssl]
skipped tests:
failed tests:
return code:[0]
time elapsed: [6.436e+03]seconds ([ 1]hours [ 47.26]minutes)
-----
-----
results for [ubuntu-2004-postgres-1012_irods-catalog-provider_3]
passed tests:
[[ 22.8854]s] [test_catalog]
[[ 18.0757]s] [test_dynamic_peps]
[[ 161.7720]s] [test_ichksum]
[[ 3.4590]s] [test_iexit]
[[ 13.3650]s] [test_ilocate]
[[ 18.2060]s] [test_imeta_error_handling]
[[ 18.8343]s] [test_imkdir]
[[ 46.6802]s] [test_iput]
[[ 121.4816]s] [test_iquest]
[[ 53.1015]s] [test_irmdir]
[[ 14.9655]s] [test_irule]
[[ 136.8253]s] [test_iticket]
[[ 30.0276]s] [test_iunreg]
[[ 77.1380]s] [test_misc]
[[ 3.0893]s] [test_python_rule_engine_plugin]
[[ 878.0375]s] [test_resource_types.Test_Resource_CompoundWithMockarchive]
[[ 723.4627]s] [test_resource_types.Test_Resource_NonBlocking]
[[1060.6564]s] [test_resource_types.Test_Resource_RandomWithinReplication]
[[1159.1791]s] [test_resource_types.Test_Resource_ReplicationWithinReplication]
[[ 4.4386]s] [test_rule_engine_plugin_passthrough]
[[ 14.9310]s] [test_stacktrace]
skipped tests:
failed tests:
return code:[0]
time elapsed: [4.581e+03]seconds ([ 1]hours [ 16.34]minutes)
-----
-----
results for [ubuntu-2004-postgres-1012_irods-catalog-provider_4]
passed tests:
[[ 30.1313]s] [test_collection_mtime]
[[ 860.5208]s] [test_iadmin]
[[ 59.1282]s] [test_ichmod]
[[ 13.1591]s] [test_ifsck]
[[ 58.4915]s] [test_ils]
[[ 9.2195]s] [test_imeta_help]
[[ 34.9259]s] [test_imv]
[[ 92.3246]s] [test_iput_options]
[[ 156.5395]s] [test_ireg]
[[ 5.0398]s] [test_irmtrash]
[[ 85.6373]s] [test_iscan]
[[ 44.7783]s] [test_itouch]
[[ 50.1372]s] [test_iuserinfo]
[[ 3.3577]s] [test_negotiation]
[[ 28.3033]s] [test_quotas]
[[1258.4760]s] [test_resource_types.Test_Resource_CompoundWithUnivmss]
[[ 777.6891]s] [test_resource_types.Test_Resource_Passthru]
[[2057.2540]s] [test_resource_types.Test_Resource_Replication]
[[ 874.8417]s] [test_resource_types.Test_Resource_Unixfilesystem]
[[ 924.4358]s] [test_rulebase]
[[ 77.2080]s] [test_symlink_operations]
skipped tests:
failed tests:
return code:[0]
time elapsed: [7.502e+03]seconds ([ 2]hours [ 5.027]minutes)
-----
All tests passed! :)
time elapsed: [10912.6214]seconds ([ 3]hours [ 1.8770]minutes)
==== end of test run results ====
Future Work and Dreams
The Build-and-Test Monitor
The Build-and-Test Monitor
The Build-and-Test Monitor
Thanks for listening