Road Mapping in Lidar Images Using a Joint-Task

Dense Dilated Convolutions Merging Network

Qinghui Liu, Michael Kampffmeyer,

Robert Jenssen, and Arnt-Børre Salberg

July. 29, 2019

Introduction

Motivation

Forest roads are used for timber transport by trucks with long trailers.

Accurate, detailed and complete mapping of forestroads is important.

Maps of small roads like forest roads that are difficut to observe in optical remote sensing images.

The national coverage of airborne laser scanning (ALS) data provides large opportunities for new mapping products.

Airborne laser scanning (ALS) Data

3D point cloud ALS data

Digital elevation model (DEM)

DEM created from 3D point cloud ALS data

DEM resolution: 0.5m x 0.5m

Gradient image and gound truth

Road-3

Road-1

Road-2

Gradient (slope) image of the DEM

Ground truth (binary and 3-classes)

Methods

Example of the DDCM architecture

composed of N {1, 2, 3, ..., N} DC blocks with various dilation rates.

Dense Dilated Convolutions Merging (DDCM) module

1-D example of DDCM [1, 2, 4]

DDCM-R50 model with pretrained ResNet50

Single Task DDCM-Net for Land Cover Mapping.

Joint-Task DDCM-Net for Road Mapping

JT-DDCM-R50 model with 2 more joint learning tasks.

Iterative-Random Weighting Losses

L_{total}^i = L_{mce, w_c}^i + w_{r1, i} \ast L_{bce}^i + w_{r2, i} \ast L_{lovasz}^i

i \in \{1, 2, 3, ...\}

w_{r1, i}

w_{r2, i}

We sample the weights

and

uniformly between

[ 0, 1 ]

denotes the current iteration of training,

L_{mce, w_c}^i

L_{bce}^i

L_{lovasz}^i

the MFB weighted cross entropy loss for the main task

the binary cross entropy loass for the classification task

lovasz-softmax loss for the binary segmentation task

Where,

w_c =\frac{\text{median} (\{f_c | c\in \mathcal{C} \})}{f_c}

: the median frequency balancing (MFB) weights.

[13] M. B. A. R. T. Matthew and B. Blaschko, “The Lovasz-softmax loss: ´A tractable surrogate for the optimization of the intersection-over-union measure in neural networks,” 2018.

Experiments and Results

ISPRS Vaihingen: http://www2.isprs.org/commissions/comm3/wg4/2d-sem-label-vaihingen.html

ISPRS Potsdam: http://www2.isprs.org/commissions/comm3/wg4/2d-sem-label-potsdam.html

ISPRS Vaihingen

ALS (Gradient + Hillshade)

Datasets

Comparison: Parameters Size, GFLOPs and mIoU

(-1.9%)

(+9x)

(+4x)

(GFLOPs measured on input size 3 x 256 x256),

Computational Cost

Vaihingen hold-out (17 unseen images) test results.

Performance on ISPRS dataset

(+4.6%)

(+5.6%)

Test Results on Our Lidar dataset

Test image

Full-class GT

DDCM-R50

JT-DDCM-R50

Analysis of Results

JT-DDCM-R50 mode obtained less fragmented mapping than the original DDCM-R50 model without joint-task learning.

Both models achieved overall higher (+2.1%) accuracy than our previous work, which used FCN and SegNet models.

However, both models still tend to easily mis-classify Road2 as Road1, and also there are many missed mapping for the road that is difficult to see in the gradient image.

Conclusions and Future Work

We presented the DDCM-Net that can effectively recognize multi-scale and complex shaped objects with similar texture and colors, and also is shown to have superior performance over existing methods.

We proposed the use of a joint-task learning strategy and an iterative-random-weighting method, which further pushes the network towards learning more robust representations.

Including other data sources like optical data or by using other DEM-derived products.
Adding more training data or improving the labeling quailitiy.
Exploit some post-processing methods to reduce fragmenting isssues.

Conlusions

Future work

Thanks for your attention!

Q & A

Brian.Liu@nr.no