Getting Started

October 5, 2016

RENCI

Chapel Hill, NC

Terrell Russell, Ph.D.

@terrellrussell

Acting Chief Technologist, iRODS Consortium

Little Slips of Paper & Dependencies

wget ftp://ftp.renci.org/pub/irods/training/training2016.pem
chmod 600 training2016.pem
ssh -i training2016.pem ubuntu@#.#.#.#

Logging in to your VM:

Install iRODS Build Requirements:

sudo apt-get update

 

sudo apt-get -y install git g++ make python-dev help2man unixodbc libfuse-dev libcurl4-gnutls-dev libbz2-dev zlib1g-dev libpam0g-dev libssl-dev libxml2-dev libkrb5-dev unixodbc-dev libjson-perl python-psutil python-jsonschema super python-exif odbc-postgresql

Little Slips of Paper & Dependencies

  1. Download Putty http://www.putty.org/
  2. Download Private Key ftp://ftp.renci.org/pub/irods/training/training2016.ppk
  3. Open Putty
  4. Enter Username
  5. Select Private Key
  6. Enter Target Host Name (or IP address)
  7. Save Session, Connect! (click Yes at the first-run Security Alert) 

Windows (using Putty):

Little Slips of Paper & Dependencies

Open Putty

Little Slips of Paper & Dependencies

Enter Username

Little Slips of Paper & Dependencies

Select Private Key

Little Slips of Paper & Dependencies

Enter Host Name (or IP address)

 

Name Session

 

Save Session

 

Open Connection

Little Slips of Paper & Dependencies

First-Run

Security Alert

Acquire the Prerequisites

Clone the training repository:

wget ftp://ftp.renci.org/pub/irods/releases/4.1.8/ubuntu14/irods-icat-4.1.8-ubuntu14-x86_64.deb

 

wget ftp://ftp.renci.org/pub/irods/releases/4.1.8/ubuntu14/irods-database-plugin-postgres-1.8-ubuntu14-x86_64.deb

Download iRODS 4.1.8:

git clone https://github.com/irods/irods_training

git clone https://github.com/irods/contrib

Install and configure Postgres

ubuntu $ sudo apt-get -y install postgresql

ubuntu $ sudo su - postgres

Prepare database for iRODS use:

postgres $ psql

CREATE DATABASE "ICAT";

CREATE USER irods WITH PASSWORD 'testpassword';

GRANT ALL PRIVILEGES ON DATABASE "ICAT" to irods;

\q

postgres $ exit

Install iRODS 4.1.8

sudo dpkg -i ./irods-icat-4.1.8-ubuntu14-x86_64.deb

sudo dpkg -i ./irods-database-plugin-postgres-1.8-ubuntu14-x86_64.deb

sudo apt-get -f install

sudo /var/lib/irods/packaging/setup_irods.sh < /var/lib/irods/tests/localhost_setup_postgres.input

Install downloaded packages:

Run setup with provided input file:

Configure the Repository and Upgrade

Note: This is for Testing Purposes Only

wget -qO - https://unstable.irods.org/irods-unstable-signing-key.asc | sudo apt-key add -
echo "deb [arch=amd64] https://unstable.irods.org/apt/ $(lsb_release -sc) main" | \
  sudo tee /etc/apt/sources.list.d/renci-irods-unstable.list
sudo apt-get update

sudo apt-get -y install irods-server irods-database-plugin-postgres

Install public key and add repository:

Upgrade from repository:

sudo service irods start

Restart iRODS:

What to Consider in an iRODS Deployment

Things to consider

  • Number of users and expected simultaneous connections
  • Expected ingest rate
  • Sizes of files
    • many small
  • Partial read / write vs whole file usage
  • Replication for durability
  • Replication for locality of reference
  • Load balancing vs High Availability

iRODS will run on a laptop or a rack of servers

Upgrading Large Installations

Things to consider

  • Database Snapshots
  • Attempt a graceful grid-wide shutdown ahead of time
  • Test Zones - do not upgrade blindly
  • Conformance Tests - try your edge cases
  • Federated Zones - how mixed is your deployment

 

 

Maintenance Window

  • In the Lab:
    • 10M Data Objects
    • VM with 10GB RAM, 4 VCPUs, Rotational Disk
    • Upgrade from 4.1.8 to 4.2.0 took 13 minutes
  • Estimate:
    • 100M Data Objects to take ~2-3 hours

Questions?

Anatomy of an iRODS installation

/etc/irods/core.* - iRODS Rule Language

/etc/irods/database_config.json - database configuration /etc/irods/host_access_control_config.json - hostname filtering

/etc/irods/hosts_config.json - local /etc/hosts style configuration

/etc/irods/server_config.json - primary server configuration

/etc/irods/service_account.config - service account information

/usr/bin/* - iCommands

 

/usr/sbin/irodsAgent

/usr/sbin/irodsPamAuthCheck

/usr/sbin/irodsReServer

/usr/sbin/irodsServer

/usr/sbin/irodsXmsgServer

 

/var/lib/irods - service account home directory

Introduction to iCommands

iRODS command line equivalent to standard Unix operations

  • ils
  • icd
  • ipwd
  • iput
  • iget
  • ihelp
  • irepl

 

Some command line examples...

Questions?

RENCI - Getting Started

By iRODS Consortium

RENCI - Getting Started

  • 2,087