Road Mapping in Lidar Images Using a Joint-Task
Dense Dilated Convolutions Merging Network
Qinghui Liu, Michael Kampffmeyer,
Robert Jenssen, and Arnt-Børre Salberg
July. 29, 2019
Introduction
Motivation
- Forest roads are used for timber transport by trucks with long trailers.
- Accurate, detailed and complete mapping of forestroads is important.
- Maps of small roads like forest roads that are difficut to observe in optical remote sensing images.
- The national coverage of airborne laser scanning (ALS) data provides large opportunities for new mapping products.
Airborne laser scanning (ALS) Data
3D point cloud ALS data
Digital elevation model (DEM)
DEM created from 3D point cloud ALS data
DEM resolution: 0.5m x 0.5m
Gradient image and gound truth
Road-3
Road-1
Road-2
Gradient (slope) image of the DEM
Ground truth (binary and 3-classes)
Methods
Example of the DDCM architecture
composed of N {1, 2, 3, ..., N} DC blocks with various dilation rates.
Dense Dilated Convolutions Merging (DDCM) module
Dense Dilated Convolutions Merging (DDCM) module
1-D example of DDCM [1, 2, 4]
DDCM-R50 model with pretrained ResNet50
Single Task DDCM-Net for Land Cover Mapping.
Joint-Task DDCM-Net for Road Mapping
JT-DDCM-R50 model with 2 more joint learning tasks.
Iterative-Random Weighting Losses
L_{total}^i = L_{mce, w_c}^i + w_{r1, i} \ast L_{bce}^i + w_{r2, i} \ast L_{lovasz}^i
i \in \{1, 2, 3, ...\}
w_{r1, i}
w_{r2, i}
We sample the weights
and
uniformly between
[ 0, 1 ]
denotes the current iteration of training,
L_{mce, w_c}^i
L_{bce}^i
L_{lovasz}^i
the MFB weighted cross entropy loss for the main task
the binary cross entropy loass for the classification task
lovasz-softmax loss for the binary segmentation task
Where,
w_c =\frac{\text{median} (\{f_c | c\in \mathcal{C} \})}{f_c}
: the median frequency balancing (MFB) weights.
[13] M. B. A. R. T. Matthew and B. Blaschko, “The Lovasz-softmax loss: ´A tractable surrogate for the optimization of the intersection-over-union measure in neural networks,” 2018.
Experiments and Results
ISPRS Vaihingen: http://www2.isprs.org/commissions/comm3/wg4/2d-sem-label-vaihingen.html
ISPRS Potsdam: http://www2.isprs.org/commissions/comm3/wg4/2d-sem-label-potsdam.html
ISPRS Vaihingen
ALS (Gradient + Hillshade)
Datasets
Comparison: Parameters Size, GFLOPs and mIoU
(-1.9%)
(+9x)
(+4x)
(GFLOPs measured on input size 3 x 256 x256),
Computational Cost
Vaihingen hold-out (17 unseen images) test results.
Performance on ISPRS dataset
(+4.6%)
(+5.6%)
Test Results on Our Lidar dataset
Test image
Full-class GT
DDCM-R50
JT-DDCM-R50
Analysis of Results
- JT-DDCM-R50 mode obtained less fragmented mapping than the original DDCM-R50 model without joint-task learning.
- Both models achieved overall higher (+2.1%) accuracy than our previous work, which used FCN and SegNet models.
- However, both models still tend to easily mis-classify Road2 as Road1, and also there are many missed mapping for the road that is difficult to see in the gradient image.
Conclusions and Future Work
- We presented the DDCM-Net that can effectively recognize multi-scale and complex shaped objects with similar texture and colors, and also is shown to have superior performance over existing methods.
- We proposed the use of a joint-task learning strategy and an iterative-random-weighting method, which further pushes the network towards learning more robust representations.
- Including other data sources like optical data or by using other DEM-derived products.
- Adding more training data or improving the labeling quailitiy.
- Exploit some post-processing methods to reduce fragmenting isssues.
Conlusions
Future work
Thanks for your attention!
Q & A
Brian.Liu@nr.no
roadmapping
By Brian Liu
