Bringing complex features
from extendable
GML application schemas
into a compact
database representation
Felix Kunde
Attributes
+
Exchanging Simple Features
GIS
CSV
GeoDB
WebGIS
Web
Shapefiles
Keyhole Markup Language
Geography Markup Language
GeoPackage
GeoJSON
TopoJSON
+
Why do we want that?!
Cleaner modelling
Data consistency
Less redundancy
Exchanging Complex Features
???
Talking standard?!
ETL
OGC
Users of OGC-Compliant Software
Application Schema
But why?!
- Too complex. I want JSON.
- There's metadata for this.
- Who is using it anyway?
- MVP, hello?! Let's sprint!
- Customer doesn't care...
Darum
- Exit data integration hell
- Watch out data producers!
- Yes, there are glitches ... but tests are getting better and easier.
- Protection shield against start ups
- Standardized labeling to use with Machine Learning (just think about it...)
@potree
Requirememts
vs. Standard
What if the standard doesn't go far enough?
Option 1 - DIY
https://xkcd.com/927/
Option 2 - Make it fit somehow
-
Use generic classes in case the standard provides them
-
Pro: Existing tools can be used
-
Con: Cannot validate the semantic content
Option 3 - Join SWG
http://dilbert.com/strip/2011-08-02
Option 4 - Get hooked
FeatureType
properties
AbstractFeature
New Type
properties
properties
<CityGML>
<CityGML>
properties
FeatureType
<ADE>
<ADEElement>
New Type
properties
<Feature>
New propertoies
New feature classes
New sub classes
CityGML
Namespace
ADE
Namespace
Hook
_GenericApplicationPropertyOf<Featuretypename>
Validation
So what about the database?
Objekt <-> Relational
-
Oooold topic (impedence mismatch) ...
-
One table for each class and complex type etc.?
-
Mapping of hierachies and inheritance?
-
De-/Normalization (Avoiding JOINs)?
-
Extendable XML schema: Map once vs. map often
3D City Database
-
UML classes edited manually
-
Database schema edited manually
-
~ 60 table for CityGML v2.0
-
Hard-coded Im/Exporter in Java
-
WFS-to-SQL using an XML config file
-
New: Extendable by abritrary CityGML ADEs
-
Repo: github.com/3dcitydb
-
hub.docker.com/r/tumgis/3dcitydb-postgis
http://www.dgpf.de/src/tagung/jt2017/proceedings/proceedings/papers/30_DGPF2017_Yao_Kolbe.pdf
Atributed Graph Grammar
Homepage: http://www.user.tu-berlin.de/o.runge/agg/index.html
Repo(?): https://github.com/de-tu-berlin-tfs/Henshin-Editor/tree/master/de.tub.tfs.agg
ADE Management
ade
3DCityDB CORE Schema
Noise ADE
Energy ADE
Dynamizer ADE
- Automatic mapping into tables and generation of stored procedures*
- Core schema isn't changed
- Class extensions are stored in new tables
- Some implementation still required for enabling imports and exports
deegree Feature Store
Docs: http://download.deegree.org/documentation/3.3.0/html/featurestores.html#application-schemas
- One-time mapping via CLI (v3.4+)
- CityGML = 422 tables (maybe more)
- BLOB-Mode for faster exports
- Configurable mapping (XML)
- Support for more GMLAS (XPlan Toolbox, AIXM)
- PostGIS, Oracle, MS-SQL and (soon) GeoPackage
deegree Feature Store
CLI interface: https://github.com/lat-lon/deegree-cli-utility
Stenger 2017, FOSSGIS Passau
Geoserver app-schema
Docs: http://docs.geoserver.org/latest/en/user/data/app-schema/tutorial.html
- Separate extension to Geoserver
- Database schema must already exist
- 1:n relationships: Feature Chaining and Joining
- n:m relationships: Denormalized Views
- Use Hale to create the mapping and export as GeoServer app-schema
- PostGIS, Oracle, MongoDB, SolR
Geoserver app-schema
Tutorial: https://geoserver.geo-solutions.it/edu/en/complex_features/index.html
BRGM QGIS GMLAS Toolbox
GDAL: http://www.gdal.org/drv_gmlas_mapping_examples.html
BRGM: https://github.com/BRGM/gml_application_schema_toolbox
- Based on GDAL GMLAS driver
- Very generic, not much to configure
- CityGML = 1412 tables
- Modes: Create, Update, Append, Overwrite
- PostGIS, SQLite
More OS solutions
UML to DB schema:
ShapeChange: https://github.com/ShapeChange
xmi2db: https://github.com/pkorduan/xmi2db
ili2db: https://github.com/claeis/ili2db
ETL:
stetl: https://github.com/geopython/stetl
You know more? Please, tell me!
NoSQL
- Schemaless = support for extendable GMLAS out-of-the-box
- Fast I/O, ideal as application state, e.g. GeoRocket (MongoDB)
- Not relational vs. NoSQL, relational AND NoSQL
- Existing research: MongoDB, BaseX, Neo4j, ArangoDB
Conclusion
-
Customizable mapping with graphs
-
Still a very complex topic
-
But entrance barrier is getting lower
-
ADEs: More independence, more reliable
-
What is the right tool?
-
CityGML? > 3DCityDB
-
GML to DB? > GDAL
-
GML to DB + WFS > deegree
-
DB to GML > HALE + Geoserver
-
Just storage > NoSQL
-
That's it. Questions?
Felix Kunde
Beuth Hochschule
@FlxKu
Slides: https://slides.com/fxku/gmlas_db_foss4g
Bringing complex features from extendable GML application schemas into a compact database representation Felix Kunde
Bringing complex features from extendable GML application schemas into a compact database representation
By fxku
Bringing complex features from extendable GML application schemas into a compact database representation
- 3,215
