Getting Started
May 30, 2018
Great Plains Network 2018
Kansas City, MO
Jason Coposky
@jason_coposky
Executive Director, iRODS Consortium
Getting Started
Little Slips of Paper & Dependencies
curl -LO https://github.com/irods/irods_training/raw/ugm2017/training.pem
chmod 600 training.pem
ssh -i training.pem ubuntu@#.#.#.#
Logging in to your VM:
Install iRODS Build Requirements:
sudo apt-get update
sudo apt-get -y install git g++ make python-dev help2man unixodbc libfuse-dev libcurl4-gnutls-dev libbz2-dev zlib1g-dev libpam0g-dev libssl-dev libxml2-dev libkrb5-dev unixodbc-dev libjson-perl python-psutil python-jsonschema super python-exif odbc-postgresql
Little Slips of Paper & Dependencies
- Download Putty http://www.putty.org/
- Download Private Key https://github.com/irods/irods_training/raw/ugm2017/training.ppk
- Open Putty
- Enter Username
- Select Private Key
- Enter Target Host Name (or IP address)
- Save Session, Connect! (click Yes at the first-run Security Alert)
Windows (using Putty):
Little Slips of Paper & Dependencies
Open Putty
Little Slips of Paper & Dependencies
Enter Username
Little Slips of Paper & Dependencies
Select Private Key
C:\Users\Student\Desktop\training20
Little Slips of Paper & Dependencies
Enter Host Name (or IP address)
Name Session
Save Session
Open Connection
Little Slips of Paper & Dependencies
First-Run
Security Alert
Little Slips of Paper & Dependencies
Google Chrome can also connect via "Secure Shell" extension:
https://chrome.google.com/webstore/detail/secure-shell/pnhechapfaindjhompbnflcldabbghjo
Download public and private keys:
https://github.com/irods/irods_training/raw/ugm2017/irodstraining
https://github.com/irods/irods_training/raw/ugm2017/irodstraining.pub
Then:
- Install extension
- Open New Connection
- Enter "ubuntu@#.#.#.#"
- Import public/private keypair (select both files at the same time)
- Connect!
Acquire the Prerequisites
Clone the training repository:
git clone https://github.com/irods/irods_training
Stage the test data
cp ./irods_training/stickers.jpg /tmp/stickers.jpg
Install and configure Postgres
sudo apt-get -y install postgresql
sudo su - postgres
$ psql
CREATE DATABASE "ICAT";
CREATE USER irods WITH PASSWORD 'testpassword';
GRANT ALL PRIVILEGES ON DATABASE "ICAT" to irods;
\q
$ exit
As the ububtu user, prepare install the postgres server
As the postgres user, prepare the database
Configure the Repository, Install, Run setup
wget -qO - https://packages.irods.org/irods-signing-key.asc | sudo apt-key add - echo "deb [arch=amd64] https://packages.irods.org/apt/ $(lsb_release -sc) main" | \ sudo tee /etc/apt/sources.list.d/renci-irods.list sudo apt-get update
sudo apt-get -y install irods-server irods-database-plugin-postgres
Install public key and add repository:
Install from repository:
Read more: https://packages.irods.org
sudo python /var/lib/irods/scripts/setup_irods.py < /var/lib/irods/packaging/localhost_setup_postgres.input
Run setup with provided input file:
What to Consider in an iRODS Deployment
Things to consider
- Number of users and expected simultaneous connections
- Expected ingest rate
- Sizes of files
- many small
- Partial read / write vs whole file usage
- Replication for durability
- Replication for locality of reference
- Load balancing vs High Availability
iRODS will run on a laptop or a rack of servers
Upgrading Large Installations
Things to consider
- Database Snapshots
- Attempt a graceful grid-wide shutdown ahead of time
- Test Zones - do not upgrade blindly
- Conformance Tests - try your edge cases
- Federated Zones - how mixed is your deployment
Maintenance Window
- In the Lab:
- PostgreSQL 9
- 10M Data Objects
- VM with 10GB RAM, 4 VCPUs, Rotational Disk
- Upgrade from 4.1.8 to 4.2.0 took 13 minutes
- Estimate:
- 100M Data Objects to take ~2-3 hours
Anatomy of an iRODS installation
/etc/irods/core.* - iRODS Rule Language
/etc/irods/database_config.json - database configuration /etc/irods/host_access_control_config.json - hostname filtering
/etc/irods/hosts_config.json - local /etc/hosts style configuration
/etc/irods/server_config.json - primary server configuration
/etc/irods/service_account.config - service account information
/usr/bin/* - iCommands
/usr/sbin/irodsAgent
/usr/sbin/irodsPamAuthCheck
/usr/sbin/irodsReServer
/usr/sbin/irodsServer
/usr/sbin/irodsXmsgServer
/var/lib/irods - service account home directory
/usr/lib/irods/plugins - plugins location
Introduction to iCommands
iRODS command line equivalent to standard Unix operations
- ils
- icd
- ipwd
- iput
- iget
- irepl
use -h to get help with any particular iCommand
ihelp will show all available iCommands
Workshop Outline
Work from the left to the right side of the picture
Implement Ingest, Tiering and HPC integration
Questions?
Getting Started - GPN 2018
By jason coposky
Getting Started - GPN 2018
Hands on installation of iRODS and its dependencies
- 996