Getting Started

May 30, 2018

Great Plains Network 2018

Kansas City, MO

Jason Coposky

@jason_coposky

Executive Director, iRODS Consortium

Getting Started

Little Slips of Paper & Dependencies

curl -LO https://github.com/irods/irods_training/raw/ugm2017/training.pem
chmod 600 training.pem
ssh -i training.pem ubuntu@#.#.#.#

Logging in to your VM:

Install iRODS Build Requirements:

sudo apt-get update

 

sudo apt-get -y install git g++ make python-dev help2man unixodbc libfuse-dev libcurl4-gnutls-dev libbz2-dev zlib1g-dev libpam0g-dev libssl-dev libxml2-dev libkrb5-dev unixodbc-dev libjson-perl python-psutil python-jsonschema super python-exif odbc-postgresql

Little Slips of Paper & Dependencies

  1. Download Putty             http://www.putty.org/
  2. Download Private Key   https://github.com/irods/irods_training/raw/ugm2017/training.ppk
  3. Open Putty
  4. Enter Username
  5. Select Private Key
  6. Enter Target Host Name (or IP address)
  7. Save Session, Connect! (click Yes at the first-run Security Alert) 

Windows (using Putty):

Little Slips of Paper & Dependencies

Open Putty

Little Slips of Paper & Dependencies

Enter Username

Little Slips of Paper & Dependencies

Select Private Key

C:\Users\Student\Desktop\training20

Little Slips of Paper & Dependencies

Enter Host Name (or IP address)

 

Name Session

 

Save Session

 

Open Connection

Little Slips of Paper & Dependencies

First-Run

Security Alert

Little Slips of Paper & Dependencies

Google Chrome can also connect via "Secure Shell" extension:

 

https://chrome.google.com/webstore/detail/secure-shell/pnhechapfaindjhompbnflcldabbghjo

 

Download public and private keys:

 

 https://github.com/irods/irods_training/raw/ugm2017/irodstraining

 https://github.com/irods/irods_training/raw/ugm2017/irodstraining.pub

 

 

Then:

  1. Install extension
  2. Open New Connection
  3. Enter "ubuntu@#.#.#.#"
  4. Import public/private keypair (select both files at the same time)
  5. Connect!

Acquire the Prerequisites

Clone the training repository:

git clone https://github.com/irods/irods_training

Stage the test data

cp ./irods_training/stickers.jpg /tmp/stickers.jpg

Install and configure Postgres

sudo apt-get -y install postgresql

sudo su - postgres

$ psql

CREATE DATABASE "ICAT";

CREATE USER irods WITH PASSWORD 'testpassword';

GRANT ALL PRIVILEGES ON DATABASE "ICAT" to irods;

\q

$ exit

As the ububtu user, prepare install the  postgres server

As the postgres user, prepare the database

Configure the Repository, Install, Run setup

wget -qO - https://packages.irods.org/irods-signing-key.asc | sudo apt-key add -
echo "deb [arch=amd64] https://packages.irods.org/apt/ $(lsb_release -sc) main" | \
  sudo tee /etc/apt/sources.list.d/renci-irods.list
sudo apt-get update

sudo apt-get -y install irods-server irods-database-plugin-postgres

Install public key and add repository:

Install from repository:

sudo python /var/lib/irods/scripts/setup_irods.py < /var/lib/irods/packaging/localhost_setup_postgres.input

Run setup with provided input file:

What to Consider in an iRODS Deployment

Things to consider

  • Number of users and expected simultaneous connections
  • Expected ingest rate
  • Sizes of files
    • many small
  • Partial read / write vs whole file usage
  • Replication for durability
  • Replication for locality of reference
  • Load balancing vs High Availability

iRODS will run on a laptop or a rack of servers

Upgrading Large Installations

Things to consider

  • Database Snapshots
  • Attempt a graceful grid-wide shutdown ahead of time
  • Test Zones - do not upgrade blindly
  • Conformance Tests - try your edge cases
  • Federated Zones - how mixed is your deployment

 

 

Maintenance Window

  • In the Lab:
    • PostgreSQL 9
    • 10M Data Objects
    • VM with 10GB RAM, 4 VCPUs, Rotational Disk
    • Upgrade from 4.1.8 to 4.2.0 took 13 minutes
  • Estimate:
    • 100M Data Objects to take ~2-3 hours

Anatomy of an iRODS installation

/etc/irods/core.* - iRODS Rule Language

/etc/irods/database_config.json - database configuration /etc/irods/host_access_control_config.json - hostname filtering

/etc/irods/hosts_config.json - local /etc/hosts style configuration

/etc/irods/server_config.json - primary server configuration

/etc/irods/service_account.config - service account information

/usr/bin/* - iCommands

 

/usr/sbin/irodsAgent

/usr/sbin/irodsPamAuthCheck

/usr/sbin/irodsReServer

/usr/sbin/irodsServer

/usr/sbin/irodsXmsgServer

 

/var/lib/irods - service account home directory

 

/usr/lib/irods/plugins - plugins location

Introduction to iCommands

iRODS command line equivalent to standard Unix operations

  • ils
  • icd
  • ipwd
  • iput
  • iget
  • irepl

 

use -h to get help with any particular iCommand

 

ihelp will show all available iCommands

 

 

Workshop Outline

Work from the left to the right side of the picture

Implement Ingest, Tiering and HPC integration

Questions?

Made with Slides.com