It Takes a Village to Find a Village:

Supporting Health Research with Geospatial


Stace Maples

Stanford Geospatial Center

https://link.springer.com/article/10.1186/s41018-018-0030-y

Hannah:

"I can see the settlements in Google Earth, but I know that those images are old and those villages are long gone, but is there a way to get satellite imagery that good, and do a complete settlement survey within a few weeks of the image being taken?"

Me:

"You have come to the right place!"

Defining the Area of Interest with OpenStreetMap and the HOT Task Manager

https://tasks.hotosm.org/

#2883 - Lower Omo Valley, Ethiopia Settlement Survey

http://tasks.hotosm.org/project/2883

Obtaining and Prepping Imagery

http://foundation.digitalglobe.com/

~5000 sqkm
200gb+

(before pansharpening)

ArcMap crashed after 9 hrs processing

the Geospatial Data Abstraction Library

GDAL

http://www.gdal.org/

http://blog.cleverelephant.ca/2015/02/geotiff-compression-for-dummies.html

So I wrote a python script...

  • GeoTIFF
  • with JPEG compression
  • internally tiled
  • in the YCBCR color space
  • with internal overviews
  • Mosaicked using .vrt (Virtual Raster Mosaic)
#Import modules
import glob, os, gdal, fnmatch

#Set base directory to location of script
base_directory = os.path.abspath('')

#Make a list of Multispectral directories with 'MUL' in the path
multi_directory_list = glob.glob(os.path.join(base_directory, '*MUL'))

#Make Panchromatic directory [pan_directory] path by replacing 'MUL' with 'PAN'
for multi_directory in multi_directory_list:
    pan_directory = multi_directory.replace('MUL','PAN')
    print(multi_directory)
    print(pan_directory)
    #Make a list of TIF files in the multi_directory
    multi_tiffs_list = fnmatch.filter((os.listdir(multi_directory)), '*.TIF')
    for multi_tiff_name in multi_tiffs_list:
        print(multi_tiff_name)
#Get the matching panchromatic image for the current multispectral by searching for the RowCol
        multi_rowcol_wildcard = '*'+multi_tiff_name[19:24]+'*'
        #print(multi_rowcol)
        pan_tiffs_list = fnmatch.filter((os.listdir(pan_directory)), '*.TIF')
        pan_tiff_name = fnmatch.filter(pan_tiffs_list, multi_rowcol_wildcard)
        #print(pan_tiff_name[0])
#Pan-sharpen the multispectral using it's corresponding panchromatic image using JPEG compression and internal tiles
        print('pansharpen')
        os.system('python gdal_pansharpen.py -co NBITS=12 -co COMPRESS=JPEG -co JPEG_QUALITY=50 -co PHOTOMETRIC=YCBCR -co TILED=YES ' + os.path.join(pan_directory,pan_tiff_name[0]) + ' ' + os.path.join(multi_directory,multi_tiff_name) + ',band=7 ' + os.path.join(multi_directory,multi_tiff_name) + ',band=5 ' + os.path.join(multi_directory,multi_tiff_name) + ',band=3 ' + base_directory + '/sharp_' + multi_tiff_name)
#Add overviews for resulting tiffs
        os.system('gdaladdo --config COMPRESS_OVERVIEW JPEG --config PHOTOMETRIC_OVERVIEW YCBCR --config INTERLEAVE_OVERVIEW PIXEL -r average ' + base_directory + '/sharp_' + multi_tiff_name + ' 2 4 8 16')
#Create a Virtual Raster Mosaic of VRT type from all resulting sharpened TIFF images
print('create VRT mosaic from SHARP images')
os.system('gdalbuildvrt sharpmosaic.vrt *.TIF')
#Haven't finished figuring out how to get a decently stretched image out of this part. More to do!
print('MBtiles out')
print('gdal_translate sharpmosaic.vrt sharpmosaic.mbtiles -of MBTILES')
print('gdaladdo -r average sharpmosaic.mbtiles 2 4 8 16')

https://github.com/mapninja/DGF_parser

Creating an Editing App

Can't use
OpenStreetMap

(ODbL!, conflict, etc...)

ArcGIS Editing Application

https://stanford.maps.arcgis.com/apps/webappviewer/index.html?id=92d2e5729be04c3f9ebc5ab204fca023

Reconnaissance

https://icme.stanford.edu/resources/hive

HIVE

13440 x 5400 x .53m pixels
=
20 sq km

 

Typical Monitor

1440 x 900 x .53m pixels
=
.7 sq km

 

Or,...
250 shift+arrow pans vs. 7000+

 

~ 6 hrs

Outcomes

  • 5,000+ sq km
  • 225 settlements
  • only 2 abandoned by arrival

...aligning with the national priorities expressed in the 2016 Ethiopia DHS Key Indicator Report, our survey content focused on
Maternal Health Characteristics.

 

  • We surveyed 347 women of reproductive age

  • recorded anthropometric measurements and illness histories for 824 of their children 15 years of age and younger

  • 572 of whom were under five.

Machine Learning?

  • Daily 3m Planet Imagery

  • Excellent OSM-Based Training Data

  • Cheap GPUs and ML Cloud Infrastructure...
     

To support the use of geospatial technologies in research and teaching at Stanford

(whatever that means at any given time)

Help researchers solve ANY problem that involves 'where'

Software

ArcGIS Desktop
ArcGIS Pro
ArcGIS Online
CARTO.com
QGIS
GDAL
R

and more...

gis.stanford.edu

How to:

Download Software

Find Online Resources

Sign up for workshops

Find GIS Assistance

github.com/StanfordGeospatialCenter/

Tutorials, R Code, Python Code, etc...

locator.stanford.edu

Use this address in ArcGIS Desktop to geocode an unlimited number of North American Street Addresses. 

earthworks.stanford.edu

Sign up for the

stanfordgis Listserv!

https://mailman.stanford.edu/mailman/listinfo/stanfordgis

Thanks!

Questions?

 

@mapninja

stacemaples@stanford.edu

gis.stanford.edu
github.com/stanfordgeospatialcenter

 

@mapninja

stacemaples@stanford.edu

GISDay2018

By Stace Maples

GISDay2018

  • 536

More from Stace Maples