Using Resource Hierarchies

Jason Coposky, Chief Technologist

Storage Virtualization

One of the "Pillars" of iRODS

 

An abstraction layer that allows for reach into various services with no change to the client

 

Functionality provided via plugin interfaces

  • Authentication
  • Database
  • Network
  • API
  • Microservice
  • Resource

Composable Resources

Uses a well known Tree Metaphor - Branches and Leaves

 

Two types of nodes:

  •     Coordinating Resource -
            pure decision making
  •     Storage Resource -
            instance managing the hardware

 

 

By convention Coordinating nodes do not have storage

(this is not enforced)

Coordinating Resources ( Branches )

Compound - provide POSIX interface to alternative storage

Load Balanced - use gathered load values to determine choices

Passthru - weight then delegate operations to  a child resource

Random - randomly choose a child for a write operation

Replication - ensure all data objects are consistent across children

Round Robin - delegate writes to each child in series

Storage Resources ( Leaves )

Non-Cached

    Unix File System - generic file system storage

    Ceph-RADOS - Ceph object storage

    HPSS - access to IBM High Performance Storage System

 

Cached (Archive)

    S3 - archive resource for Amazon S3

    WOS - DDN Web Object Scalar

    Universal MSS - script based access to generic archive storage

Building a Replication Hierarchy

Create two root-level unix file system storage resources:

 

iadmin mkresc unixA unixfilesystem `hostname`:/var/lib/irods/unixAVault

iadmin mkresc unixB unixfilesystem `hostname`:/var/lib/irods/unixBVault

 

Use 'ilsresc' to investigate the list of resources:

    demoResc

    unixA

    unixB

Building a Replication Hierarchy

Create root-level replication coordinating resource:

 

iadmin mkresc replResc replication

 

Use 'ilsresc' to investigate the list of resources:

    demoResc

    replResc:replication

    unixA

    unixB

Building a Replication Hierarchy

Create the parent-child relationships:

 

iadmin addchildtoresc replResc unixA

iadmin addchildtoresc replResc unixB

 

Use 'ilsresc' to investigate the list of resources:

    demoResc

    replResc:replication

    |__unixA

    |__unixB

Testing the Resource Hierarchy

iput -R replResc training_jpegs/beans.jpg

 

ils -L
/zone182/home/admin:
  admin             0 replResc;unixB      1128069 2015-09-20.15:37 & beans.jpg
        generic    /var/lib/irods/unixBVault/home/admin/beans.jpg
  admin             1 replResc;unixA      1128069 2015-09-20.15:37 & beans.jpg
        generic    /var/lib/irods/unixAVault/home/admin/beans.jpg

Voting within a Resource Hierarchy

Coordinating Resources delegate voting to their children

    for a given operation: create, write or open

 

Storage Resources vote given:

    hostname, operation, replica number and the list of
    data object replicas

 

Coordinating Resources then choose a child resource from the resulting list of votes from the children given its own heuristics

Voting for a Unix Filesystem Resource

For a Create operation:

  • if the resource is resident on server - vote 1.0
  • if the resource is on another server - vote 0.5

 

For an Open or Write operation:

  • if object is not resident on our resource - vote 0.0
  • if the object is resident and we are the server - vote 1.0
  • if the object is resident and we are not the server -
        vote 0.5
  • if the object is resident and the object is dirty -
        vote 0.25

Influencing the Vote - the weighted passthru

The passthru resource delegates all operations to a single child and honors two properties:

  • read weight
  • write weight

 

Weights are passed to the resource as a context string:

    "read=1.0;write=1.0"

 

Weights are multiplied by the child's vote then passed to the parent resource

Modifying the Replication Hiearchy

iadmin mkresc ptResc passthru

iadmin modresc ptResc context "read=1.0;write=0.5"

 

iadmin rmchildfromresc replResc unixB

iadmin addchildtoresc replResc ptResc

iadmin addchildtoresc ptResc unixB

 

ilsresc:

replResc:replication

|_ptResc:passthru

|   |_unixB

|_unixA

Testing the new Hierarchy

iput -R replResc training_jpgs/mouse.jpg

ils

ils -L mouse.jpg
  admin             0 replResc;unixA       392585 2015-09-20.15:45 & mouse.jpg
        generic    /var/lib/irods/unixAVault/home/admin/mouse.jpg
  admin             1 replResc;ptResc;unixB       392585 2015-09-20.15:45 & mouse.jpg
        generic    /var/lib/irods/unixBVault/home/admin/mouse.jpg

 

unixB consistently won the vote, and now consistently loses

 

Use 'iadmin modresc ptResc context "read=1.0;write=0.0" to prevent a replica from being created on unixB

Advanced Topic - rebalance

Each resource has its own concept of 'balanced' -

    a depth first recursive operation on the entire
    hierarchy

 

For the replication resource - every child has an up to date copy of every data object

 

Demonstrating a rebalance

itrim -N1 -n0 mouse.jpg

ils -L mouse.jpg
  admin             1 replResc;ptResc;unixB       392585 2015-09-20.15:45 & mouse.jpg
        generic    /var/lib/irods/unixBVault/home/admin/mouse.jpg
iadmin modresc replResc rebalance
ils -L mouse.jpg
  admin             1 replResc;ptResc;unixB       392585 2015-09-20.15:45 & mouse.jpg
        generic    /var/lib/irods/unixBVault/home/admin/mouse.jpg
  admin             2 replResc;unixA       392585 2015-09-20.16:43 & mouse.jpg
        generic    /var/lib/irods/unixAVault/home/admin/mouse.jpg

Questions?

Using Resource Hierarchies

By iRODS Consortium

Using Resource Hierarchies

An explanation of iRODS resource composition and some examples illustrating the resource hierarchy voting mechanism.

  • 1,653