Metagenomic analysis at the Edge with Jetson Xavier NX
Piotr Grzesik
dr hab. inż. Dariusz Mrozek, prof. PŚ
Silesian University of Technology
Agenda
- Metagenomic and Nanopore Sequencing
- MinION Nanopore
- Challenges of metagenomics at the edge
- Analysis workflow
- Testing environment
- Experiments
- Summary
- Next steps
Metagenomics analysis
Metagenomics - study of genetic material from environmental samples. It is used to sequence DNA (or RNA) based on the prepared sample, identify microorganisms, detect potential mutations or identify previously unknown species.
Nanopore sequencing
Nanopore sequencing - developed by Oxford Nanopore Technologies, it is a process of DNA sequencing that works by monitoring changes to an electrical current caused by DNA strand passing through a nanopore. The signal that is obtained as a result is decoded to specific DNA or RNA sequences. The process of such decoding is called basecalling.
Nanopore sequencing
MinION Nanopore
MinION Nanopore - portable sequencing device, released by Oxford Nanopore Technologies in 2014. It is the first device that enables portable sequencing at affordable price (1000$). It is powered via USB, weights under 100g, which makes it possible to use it as a field device.
MinION Nanopore
Metagenomics at the edge
Edge computing is a computing paradigm that brings the data processing and storage closer to a place where it is needed. It allows to reduce the volume of data that needs to be send over the Internet, allows to improve reaction time to the changing state of the system and improves resilience and allows for data loss prevention where Internet connection is not reliable or not available at all most of the time.
Challenges of metagenomics at the edge
- Computational complexity
- Volume of data that needs to be processed
- Limited network connectivity
- Limited access to power supplies
Analysis workflow
The considered analysis workflow consist of three separate steps - sequencing, basecalling and classification. During the first step, MinION device will sequence DNA, outputting electrical current measurements in form of FAST5 files. In the second step, basecaller will interpret electrical current and output genomic sequences. In the last step, genomic sequences will be classified and labeled with taxonomic labels.
Testing environment
Testing environment
-
Simulated MinION Nanopore that outputs reads in form of fast5 files
-
Edge device that runs basecalling and classification analysis with SSD Disk
-
Cloud-based service, long-term storage of sequenced data
-
Power supply with INA219 + Raspberry Pi 4 for current measurement
Edge device specification
- Single board computer - Jetson Xavier NX
-
GPU - NVIDIA Volta™ architecture
with 384 NVIDIA® CUDA® cores and 48 Tensor cores -
6-core NVIDIA Carmel ARM®v8.2 64-bit CPU
6 MB L2 + 4 MB L3 - 8 GB 128-bit LPDDR4x 51.2GB/s
- Storage - SDHC card (32 GB, class 10) + SSD NVMe
- OS - Ubuntu 18.04.5 LTS with kernel version 4.19.140-tegra
Edge device specification
Software used
- Guppy - It is state-of-the-art, closed-source, basecaller developed by Oxford Nanopore Technologies, creator of MinION Nanopore device. It offers support for both CPU and GPU acceleration on selected devices, works both on AArch64 and x86-64 architectures. It also supports two different modes - fast and hac, which stands for high accuracy.
- Bonito - In development, open-sourced, basecaller developed by Oxford Nanopore Technologies. Out of the box it only supports x86_64 architecture, but after several adjustments it was possible to run it on AArch64 as well. It uses Recurrent Neural Networks to perform basecalling and supports GPU acceleration.
Software used
- Deepnano-blitz - Open-source basecaller, developed by Vlado Boza et al., written in Rust and Python, based on RNNs and uses Beam Search as optimization step. It is heavily optimized for Intel processors that suport AVX2 instructions set. It was not used due to high dependency on specific instruction set not available on ARM.
- Causalcall - Open-source basecalled, developed by Jingwen Zeng et al., written in Python and takes advantage of Convolutional Temporal Network with CTC decoder. Unfortunatelly, it wasn't possible to run it on Jetson Xavier NX due to incompatibility with AArch64 architecture.
Software used
- Kraken 2 - taxonomic classification system using exact k-mer matches to achieve high accuracy and fast classification speeds. This classifier matches each k-mer within a query sequence to the lowest common ancestor (LCA) of all genomes containing the given k-mer. Due to significant RAM use of standard Kraken databases, a new, minimized database was prepared, dedicated for identification of bacterias and virals.
Experiments
-
Fast5 files with data from sequencing runs containing material of Escherichia coli and Klebsiella Pneumoniae
-
Measurement of samples processed per second by each basecaller in different power modes
-
Both Guppy and Bonito were tested with and without GPU acceleration enabled
-
During each run, current was measured with INA219 + Raspberry Pi 4 circuit
-
Classification with Kraken2 was also ran in different power modes
-
Available power modes in Jetson Xavier NX: 15W 6 CORE, 15W 4 CORE, 15W 2 CORE, 10W 2 CORE, 10W 4 CORE
Results
Samples per second processed by Guppy with Fast model
Results
Samples per second processed by Guppy with HAC model
Results
Average power for Guppy with Fast model
Results
Average power for Guppy with HAC model
Summary & next steps
-
Guppy with Fast model is able to support real-time basecalling for up to 3 MinION devices at the same time
-
Jetson Xavier NX is a sutiable device for running such experiments in places with limited network connectivity and limited power supplies
-
Using 10W power mode on Jetson Xavier NX is more efficient energy-wise than 15W power mode
-
CPU basecalling is not feasible on Jetson Xavier NX device (on similar ones as well)
-
As of right now, it seems like Guppy is the only basecaller suitable to use at the edge
-
Optimize alternative basecallers for edge use
-
Work on alternative approach where instead of basecalling + classification, raw signals are used to map against expected microorganisms