Bringing complex features

from extendable

GML application schemas

into a compact

database representation

Felix Kunde

Attributes

+

Exchanging Simple Features

GIS

CSV

GeoDB

WebGIS

Web

 

Shapefiles

Keyhole Markup Language

Geography Markup Language

GeoPackage

GeoJSON

TopoJSON

+

Why do we want that?!

Cleaner modelling

Data consistency

Less redundancy

Exchanging Complex Features

???

Talking standard?!

ETL

OGC

Users of OGC-Compliant Software

Application Schema

But why?!

  • Too complex. I want JSON.
  • There's metadata for this.
  • Who is using it anyway?
  • MVP, hello?! Let's sprint!
  • Customer doesn't care...

Darum

  • Exit data integration hell
  • Watch out data producers!
  • Yes, there are glitches ... but tests are getting better and easier.
  • Protection shield against start ups
  • Standardized labeling to use with Machine Learning (just think about it...)

@potree

Requirememts

vs. Standard

What if the standard doesn't go far enough?

Option 1 - DIY

https://xkcd.com/927/

Option 2 - Make it fit somehow

  • Use generic classes in case the standard provides them

  • Pro: Existing tools can be used

  • Con: Cannot validate the semantic content

Option 3 - Join SWG

http://dilbert.com/strip/2011-08-02

Option 4 - Get hooked

FeatureType

properties

AbstractFeature

New Type

properties

properties

<CityGML>

<CityGML>

properties

FeatureType

<ADE>

<ADEElement>

New Type

properties

<Feature>

New propertoies

New feature classes

New sub classes

CityGML

Namespace

ADE

Namespace

Hook

_GenericApplicationPropertyOf<Featuretypename>

Validation

So what about the database?

Objekt <-> Relational

  • Oooold topic (impedence mismatch) ...

  • One table for each class and complex type etc.?

  • Mapping of hierachies and inheritance?

  • De-/Normalization (Avoiding JOINs)?

  • Extendable XML schema: Map once vs. map often

3D City Database

  • UML classes edited manually

  • Database schema edited manually

  • ~ 60 table for CityGML v2.0

  • Hard-coded Im/Exporter in Java

  • WFS-to-SQL using an XML config file

  • New: Extendable by abritrary CityGML ADEs

  • Repo: github.com/3dcitydb

  • hub.docker.com/r/tumgis/3dcitydb-postgis

http://www.dgpf.de/src/tagung/jt2017/proceedings/proceedings/papers/30_DGPF2017_Yao_Kolbe.pdf

Atributed Graph Grammar

Homepage: http://www.user.tu-berlin.de/o.runge/agg/index.html

Repo(?): https://github.com/de-tu-berlin-tfs/Henshin-Editor/tree/master/de.tub.tfs.agg

ADE Management

ade

3DCityDB CORE Schema

Noise ADE

Energy ADE

Dynamizer ADE

  • Automatic mapping into tables and generation of stored procedures*
  • Core schema isn't changed
  • Class extensions are stored in new tables
  • Some implementation still required for enabling imports and exports

deegree Feature Store

Docs: http://download.deegree.org/documentation/3.3.0/html/featurestores.html#application-schemas

  • One-time mapping via CLI (v3.4+)
  • CityGML = 422 tables (maybe more)
  • BLOB-Mode for faster exports
  • Configurable mapping (XML)
  • Support for more GMLAS (XPlan Toolbox, AIXM)
  • PostGIS, Oracle, MS-SQL and (soon) GeoPackage

deegree Feature Store

CLI interface: https://github.com/lat-lon/deegree-cli-utility

Stenger 2017, FOSSGIS Passau

Geoserver app-schema

Docs: http://docs.geoserver.org/latest/en/user/data/app-schema/tutorial.html

  • Separate extension to Geoserver
  • Database schema must already exist
  • 1:n relationships: Feature Chaining and Joining
  • n:m relationships: Denormalized Views
  • Use Hale to create the mapping and export as GeoServer app-schema
  • PostGIS, Oracle, MongoDB, SolR

Geoserver app-schema

Tutorial: https://geoserver.geo-solutions.it/edu/en/complex_features/index.html

BRGM QGIS GMLAS Toolbox

GDAL: http://www.gdal.org/drv_gmlas_mapping_examples.html

BRGM: https://github.com/BRGM/gml_application_schema_toolbox

  • Based on GDAL GMLAS driver
  • Very generic, not much to configure
  • CityGML = 1412 tables
  • Modes: Create, Update, Append, Overwrite
  • PostGIS, SQLite

More OS solutions

UML to DB schema:

ShapeChange: https://github.com/ShapeChange

xmi2db: https://github.com/pkorduan/xmi2db

ili2db: https://github.com/claeis/ili2db

 

ETL:

stetl: https://github.com/geopython/stetl

 

You know more? Please, tell me!

NoSQL

  • Schemaless = support for extendable GMLAS out-of-the-box
  • Fast I/O, ideal as application state, e.g. GeoRocket (MongoDB)
  • Not relational vs. NoSQL, relational AND NoSQL
  • Existing research: MongoDB, BaseX, Neo4j, ArangoDB

Conclusion

  • Customizable mapping with graphs

  • Still a very complex topic

  • But entrance barrier is getting lower

  • ADEs: More independence, more reliable

  • What is the right tool?

    • CityGML? > 3DCityDB

    • GML to DB? > GDAL

    • GML to DB + WFS > deegree

    • DB to GML > HALE + Geoserver

    • Just storage > NoSQL

That's it. Questions?

Felix Kunde

Beuth Hochschule

@FlxKu

 

Slides: https://slides.com/fxku/gmlas_db_foss4g

Made with Slides.com