Processing marine DTM’s - like a boss!

Gareth Grewcock (Sam Franklin - stand in)

gareth.grewock@lr-senergy.com

@gordy99

https://slides.com/garethgrewcock/foss4guk-marine-dtm/

Purpose

We set out to take "standard workflows" for visualising marine Digital Terrain Model (DTMs) using proprietary software and replicate using FOSS

Overview

• Background
• XYZ challenges
• Hillshading
• End-to-end workflow
• Results
• Deployment/community

about

LR Survey & GeoEngineering

• Offshore Project Management of "Geo" surveys
• Geoscience/Geospatial/Engineering Consultancy
• Operational Support

Topography (land)

Bathymetry - seabed

Bathymetry Acquisition

• Multibeam Echosounder (MBES) systems
• Acquired by survey contractors at significant expense
• MH370 search = $130M

Receive latest bathymetry data from vessel daily

Onshore support for 24/7 vessel operations

Applications

Dredging Operations

Pipeline Burial

Receive bathymetry data from the vessel

Process for project status for data collection to date

“Industry standard” | Hydrographic survey format XYZ`

Ungridded or scatter data requires either extrapolation or interpolation to fill areas of no-data.

490313    5718729    -36.239
490348    5718738    -36.282
490359    5718720    -36.315
490361    5718705    -36.337
490314    5718651    -36.361
490308    5718728    -36.334
490353    5718661    -36.331
490358    5718700    -36.333
490308    5718721    -36.363
490312    5718741    -36.386
490313    5718702    -36.433

3-column ungridded XYZ file

“Industry standard” | Hydrographic survey format XYZ`

3-column gridded XYZ file types

 XYZ without delete values

490300.00    5718650.00    -36.239
490301.00    5718650.00    -36.282
490302.00    5718650.00    -36.315
490303.00    5718650.00    -36.337
490304.00    5718650.00    -36.361
490305.00    5718650.00    -36.334
490306.00    5718650.00    -36.331
490307.00    5718650.00    -36.333
490308.00    5718650.00    -36.363
490309.00    5718650.00    -36.386
490310.00    5718650.00    -36.433

“Industry standard” | Hydrographic survey format XYZ`

XYZ file format is consistent across surveys however, the spatial extent and file size differ depending on survey contractor and job

For example,

25km pipeline route XYZ deliverable:

1 x XYZ file @ >5GB

OR

25 x XYZ files @ 200MB

end-to-end workflow

the composite hillshade relief - WHY?

A false colour shaded image assists geoscientists with interpretation

current workflow

Good visual results from proprietary solutions ...

But ...

The need for an alternative?

• Manual Workflow -->  need to automate
• 2GB Output limit --> Need more flexible option
• Network licence only --> need offline mode
• Only 1 licence seat --> locking issues / service restarts

• Generally, was an opportunity learn more about FOSS workflows and then share that knowledge with the marine GIS sector

There are numerous workflows which work with 'XYZ' data however, there are few which handle the gridded, "multi-block" hydrographic XYZ file format

reading the XYZ = stuck!

The power of the community!

Chance meeting @

finding the right tool...

 we can build on top of GMT using core functionality from other FOSS tools and bring together into one end-to-end workflow

What we need:

A command line tool with the ability to handle gridded  hydrographic XYZ file format

the bench mark

How do we achieve the same output?

uncharted waters

Combining the hillshade with colour relief

• Comparable output?
• Extensible?
• Deployment?

r.shade - Drapes a color raster over an shaded relief or aspect map.

r.shaded.relief - Creates shaded relief map from an elevation map (DEM).

altitude= Altitude of the sun

azimuth=Azimuth of the sun

zmult=Exaggerate Z value

=

PyGRASS

the ignition

workflow fed by input CSV config file

• Input XYZ file directory
• XYZ Cell Size
• EPSG Dataset Reference
• Invert Z Value | YES/NO Flag
• Seabed Gradient Threshold
• Contour | YES/NO Flag
• Min/Max Contour Values
• Contour Resample Value
• Output Image Format | RGB or Color Paletted

input CSV config file pathway is added to python script

All input XYZ files must be the same cell size and EPSG ref

contour smooth

resample the raster to achieve smooth contours

seabed gradient analysis

gdal_calc

gdal_polygonize

back. of. the. net!

slope enhanced composite relief

hillshade composite relief

composite colour-relief  hillshade

existing

xyz processor

Existing

"xyz processor"

Existing

"xyz processor"

Food for thought

• Slope+Colour-relief only? Or Slope+Hillshade+colour-relief

• BUT Matched output = achievement unlocked!

• False-colour = subjective

• Each XYZ is different and requires different hill-shading parameters to draw out features from the seabed

• It is up to the geophysicists really !

• Mailing list
• Forums/StackExchange
• Google+
• Twitter
• Here Today!!

the FOSS community giveth ...

'You will find that help will always

be given at Hogwarts to those who ask for it.'

 Professor Albus Dumbledore

the FOSS community takeith away (JOKE!)

• Add slope-enhanced shading option to this new QGIS feature?
• No GMT, so won't deal with hydro-XYZ either.
• Will vow to work with GDAL devs on this

deployment &

release

Phase 1     Rollout internally, OSGeo4W + GMT

In progress/Testing

Phase 2      DTM Cheat Sheet

https://github.com/senergyworld/dtm-cheat-sheet

Phase 3      XYZ Processor Python (MIT licence)

Coming Soon (Gareth!)

Thanks!

Gareth Grewcock (Sam Franklin - stand in)

gareth.grewock@lr-senergy.com

@gordy99

https://slides.com/garethgrewcock/foss4guk-marine-dtm/