Processing marine DTM’s - like a boss!

Gareth Grewcock (Sam Franklin - stand in)

gareth.grewock@lr-senergy.com

https://slides.com/garethgrewcock/foss4guk-marine-dtm/

Purpose

We set out to take "standard workflows" for visualising marine Digital Terrain Model (DTMs) using proprietary software and replicate using FOSS

Overview

Background
XYZ challenges
Hillshading
End-to-end workflow
Results
Deployment/community

about

LR Survey & GeoEngineering

Offshore Project Management of "Geo" surveys
Geoscience/Geospatial/Engineering Consultancy
Operational Support

Topography (land)

Bathymetry - seabed

Bathymetry Acquisition

Multibeam Echosounder (MBES) systems
Acquired by survey contractors at significant expense
MH370 search = $130M

Receive latest bathymetry data from vessel daily

Onshore support for 24/7 vessel operations

Applications

Dredging Operations

Pipeline Burial

Receive bathymetry data from the vessel

Process for project status for data collection to date

“Industry standard” | Hydrographic survey format XYZ`

Ungridded or scatter data requires either extrapolation or interpolation to fill areas of no-data.

490313 5718729 -36.239
490348 5718738 -36.282
490359 5718720 -36.315
490361 5718705 -36.337
490314 5718651 -36.361
490308 5718728 -36.334
490353 5718661 -36.331
490358 5718700 -36.333
490308 5718721 -36.363
490312 5718741 -36.386
490313 5718702 -36.433

3-column ungridded XYZ file

“Industry standard” | Hydrographic survey format XYZ`

3-column gridded XYZ file types

XYZ without delete values

490300.00 5718650.00 -36.239
490301.00 5718650.00 -36.282
490302.00 5718650.00 -36.315
490303.00 5718650.00 -36.337
490304.00 5718650.00 -36.361
490305.00 5718650.00 -36.334
490306.00 5718650.00 -36.331
490307.00 5718650.00 -36.333
490308.00 5718650.00 -36.363
490309.00 5718650.00 -36.386
490310.00 5718650.00 -36.433

“Industry standard” | Hydrographic survey format XYZ`

XYZ file format is consistent across surveys however, the spatial extent and file size differ depending on survey contractor and job

For example,

25km pipeline route XYZ deliverable:

1 x XYZ file @ >5GB

25 x XYZ files @ 200MB

end-to-end workflow

the composite hillshade relief - WHY?

A false colour shaded image assists geoscientists with interpretation

current workflow

Good visual results from proprietary solutions ...

But ...

The need for an alternative?

Manual Workflow --> need to automate
2GB Output limit --> Need more flexible option
Network licence only --> need offline mode
Only 1 licence seat --> locking issues / service restarts

Generally, was an opportunity learn more about FOSS workflows and then share that knowledge with the marine GIS sector

There are numerous workflows which work with 'XYZ' data however, there are few which handle the gridded, "multi-block" hydrographic XYZ file format

reading the XYZ = stuck!

The power of the community!

Chance meeting @

finding the right tool...

we can build on top of GMT using core functionality from other FOSS tools and bring together into one end-to-end workflow

What we need:

A command line tool with the ability to handle gridded hydrographic XYZ file format

the bench mark

How do we achieve the same output?

uncharted waters

Combining the hillshade with colour relief

Comparable output?
Extensible?
Deployment?

r.shade - Drapes a color raster over an shaded relief or aspect map.

r.shaded.relief - Creates shaded relief map from an elevation map (DEM).

altitude= Altitude of the sun

azimuth=Azimuth of the sun

zmult=Exaggerate Z value

PyGRASS

the ignition

workflow fed by input CSV config file

Input XYZ file directory
XYZ Cell Size
EPSG Dataset Reference
Invert Z Value | YES/NO Flag
Seabed Gradient Threshold
Contour | YES/NO Flag
Min/Max Contour Values
Contour Resample Value
Output Image Format | RGB or Color Paletted

input CSV config file pathway is added to python script

All input XYZ files must be the same cell size and EPSG ref

contour smooth

resample the raster to achieve smooth contours

seabed gradient analysis

gdal_calc

gdal_polygonize

back. of. the. net!

slope enhanced composite relief

hillshade composite relief

composite colour-relief hillshade

existing

xyz processor

Existing

"xyz processor"

Existing

"xyz processor"

Food for thought

Slope+Colour-relief only? Or Slope+Hillshade+colour-relief
BUT Matched output = achievement unlocked!
False-colour = subjective
Each XYZ is different and requires different hill-shading parameters to draw out features from the seabed
It is up to the geophysicists really !