Data Management for the Smart Farm
Jason Coposky
@jason_coposky
Executive Director, iRODS Consortium
Data Management for the Smart Farm
The iRODS Consortium
Our Membership
What is iRODS
Distributed - runs on a laptop, a cluster, on premises or geographically distributed
Open Source - BSD-3 Licensed, install it today and try before you buy
Metadata Driven & Data Centric - Insulate both your users and your data from your infrastructure
iRODS as the Integration Layer
iRODS Core Competencies
The underlying technology categorized into four areas
Data Virtualization
Combine various distributed storage technologies into a Unified Namespace
iRODS provides a logical view into the complex physical representation of your data, distributed geographically, and at scale.
Projection of the Physical into the Logical
Logical Path
Physical Path(s)
Data Discovery
Attach metadata to any first class entity within the iRODS Zone
iRODS provides automated and user-provided metadata which makes your data and infrastructure more discoverable, operational and valuable.
Metadata Everywhere
Workflow Automation
Integrated scripting language which is triggered by any operation within the framework
The iRODS rule engine provides the ability to capture real world policy as computer actionable rules which may allow, deny, or add context to operations within the system.
Dynamic Policy Enforcement
The iRODS rule may:
Secure Collaboration
iRODS allows for collaboration across administrative boundaries after deployment
iRODS provides the ability to federate namespaces across organizations without pre-coordinated funding or effort.
Federation - Shared Data and Services
Ingest to Institutional repository
As data matures and reaches a broader community, data management policy must also evolve to meet these additional requirements.
iRODS and the Smart Farm
Challenges with Sensor Networks
Challenges of the Smart Farm
The iRODS IoT Gateway - Data Collection
The iRODS IoT Gateway - Data Organization
The iRODS IoT Gateway - Data Harmonization
Prep data for analytics:
The iRODS IoT Gateway - Data Movement
Data movement can be initiated by policy or by the user
The iRODS IoT Gateway - Data Discovery
Metadata within the catalog may be attached to any entity within the system: data, collections, users, storage
The iRODS IoT Gateway - Architecture
Farm Zone
Agriculture Victoria Zone
Federation
Catalog
Catalog
The iRODS IoT Gateway - Architecture
Agriculture Victoria Zone
Catalog
Agriculture Victoria Zone
iRODS Service Integration
Once Data is at rest in the Agriculture Victoria Namespace
Catalog
Agriculture Victoria Zone
REST or Python
Interface
DPC API Gateway
Things to consider in an iRODS Deployment
iRODS will run on a RaspberryPi or a rack of servers
Questions?