Spatial Data 101
Points, Lines, Polygons & Pixels
where is your data? gis.stanford.edu
Open a LIVE version of these slides at:
slides.com/staceymaples/spatial101/live
about me
gis.stanford.edu
All things are related. Near things are more related than distant things.
-Waldo Tobler's 1st Law of Geography
Everything is somewhere and that somewhere matters.
-Stace's 1st Law of Geography
Spatial Data is Everywhere
Modeling the World
Data Models
Everything is somewhere, but how do we say where?
Coordinate Systems
Geographic Coordinates
Projections
Think of them as translations from 3d to 2d
Credit: @chris_whong
Demo / Hands-on (if you like)
Vector Data
Vector Formats
Shapefiles*
*(a shapefile isn't really a file)
- *.shp
- *.shx
- *.dbf
- *.prj
GeoJSON
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {
"Label": "\"Null Island\""
},
"geometry": {
"type": "Point",
"coordinates": [
0,
0
]
}
},Relational Databases
SELECT * FROM clowns
WHERE clown_name LIKE '%BUB%'Tabular
| OBJECTID | Num_Cases | Address | xcoord | ycoord |
|---|---|---|---|---|
| 1 | 1 | 29 Kingly Street | -0.140033817 | 51.51326827 |
| 2 | 3 | 27 Heddon Street | -0.139874596 | 51.51122647 |
| 3 | 1 | 31 Heddon Street | -0.13991682 | 51.51127459 |
| 4 | 2 | 18 Fouberts Place | -0.139587437 | 51.51353757 |
| 5 | 1 | 18 Kingly Street | -0.139681937 | 51.51299401 |
| 6 | 2 | 18 Fouberts Place | -0.139587437 | 51.51353757 |
| 7 | 1 | 1 Ramillies Place | -0.139478997 | 51.51500072 |
| Extension | File Type |
|---|---|
| Geography Markup Language (GML) | .GML |
| Google Keyhole Markup Language (KML/KMZ) | .KML .KMZ |
| GPS eXchange Format (GPX) | .GPX |
| IDRISI Vector | .VCT .VDC |
| MapInfo TAB | .TAB .DAT .ID .MAP .IND |
| OpenStreetMap OSM XML | .OSM |
| Digital Line Graph (DLG) | .DLG |
| Geographic Base File-Dual Independent Mask Encoding (GBF-DIME) | |
| ArcInfo Coverage |
Other Formats...
Raster Data
Raster Data is Pixels
Pixels are numeric values
Hands-on with QGIS
But imagery is special, so let's talk a little more about THAT...
Spatial Imagery
a tiny introduction to
remote sensing
Digital images are made of pixels, and pixels are representations of numeric values
Electromagnetic energy is everywhere and everything reflects or absorbs it, differently
(and measurably).
Finally, cadence...
How often does a satellite revisit locations?
- Landsat - 16 days
- Sentinel 2 - 10 days at the equator, 2-3 at mid-latitudes
- Planet.com - Daily
- Digital Globe - varies, tasked
“Often it turns out to be more efficient to move the questions than to move the data.
- Jim Gray”
Raster Formats
GeoTIFF
(a TIFF with Geo)
The GeoTIFF file format was developed as a distribution format for satellite or aerial imagery as well as other data such as Digital Elevation Model (DEM) data, and Digital Ortho Quadrangle data.
COG
(Cloud-optimized GeoTIFF)
A Cloud Optimized GeoTIFF (COG) is a regular GeoTIFF file, hosted on a HTTP file server, with an internal organization that enables clients issuing HTTP GET range requests to ask for just the parts of a file they need.
Esri GRID
(proprietary format)
A proprietary binary format, developed and supported by Esri, QGIS and GDAL
Now go to:
Login and wait for the workshop repo to load. You should then see this item in your GEE Scripts Panel
Web Maps
The "special" case of Web Mercator
Credit: @chris_whong
Vector Tiles are a little different
Thinking "Spatially"
Spatial Relationships
- Adjacency
- Containment
- Intersection
- Proximity
Cartographic Modeling
- Local
- Neighborhood
- Zonal
- Global
Simulation
- Hillshade
- Viewshed
- Hydrology
- Movement/Least-cost paths
- Agent-based models
APIs / SDKs / Libraries
GDAL/OGR
Geospatial Data Abstraction Library
(it basically runs everything)
Basemap & Vector Tile Services
- Mapbox Studio - style OSM data and add your own, great free tier, integrates with many APIs, SDKs, etc... - https://www.mapbox.com/mapbox-studio/
- Tegola - lightweight vector tile minting and server to roll your own- https://tegola.io/
- Quick Map Services Registry - basemap tile services, for QGIS Plugin, but can be used directly - https://qms.nextgis.com/
Geocoding
Turning text-based locations (addresses, placenames, localities, ...) into latitude & longitude coordinates.
(reverse geocoding searching nearby places or features with a lat/long)
- Stanford's https://locator.stanford.edu service provides North & South American and European geocoding services, within the Stanford IP ranges
- Stanford's super-new, not well benchmarked Global Geocoding Services are still being built, but available, here: https://sul-geocoding-web.stanford.edu/arcgis/rest/services/geocode
- US Census Address Geocoding API (open, no key, slow!) - https://geocoding.geo.census.gov/geocoder/Geocoding_Services_API.pdf
- Google Maps Geocoding API (fast, restrictive, not totally free, keyed)- https://developers.google.com/maps/documentation/geocoding/start
- Geonames.org (fast, permissive, free, public domain data, keyed) - https://www.geonames.org/export/ws-overview.html
- Geolocate (fast-ish, permissive, free, good with uncertainty and esoteric localities) - https://www.geo-locate.org/developers/default.html
Routing
- Directions
- Isochrones
- Distance Matrices
- Route Optimization
Routing Services
- Google Directions (free quota, restrictive terms) - https://developers.google.com/maps/documentation/directions/start
- Mapbox (Valhalla/OSM-based, no official free tier, but we have a relationship with and can get access for projects) - https://docs.mapbox.com/api/navigation/
- OpenRouteService (generous free tier, REST API, Jupyter Notebooks) - https://openrouteservice.org/
Roll your own Routing Services
Machine Learning
- Mapbox Robosat - https://github.com/mapbox/robosat
- Development Seed ML Project (Skynet) https://developmentseed.org/projects/?topic=machine%20learning
- Tensorflow & Google Earth Engine - https://developers.google.com/earth-engine/tf_examples
Data Sources
- COVID-19 Resources (curated by TAMU) - https://docs.google.com/spreadsheets/d/1GmV0FyH-hdTpvVgqo6OW87NbTTk-C7xr46qPv14_UiA/edit#gid=1541355750
- https://earthworks.stanford.edu
- Humanitarian Data Exchange (HDX) - https://data.humdata.org/
- Planet.com Imagery - http://planet.com
https://developers.planet.com/planetschool/ - Overpass Turbo - GUI for Overpass, the OSM Data API - https://overpass-turbo.eu/
Data Collection/Creation
- https://tasks.hotosm.org/project/2883
- Forms
- GPS
3D / LIDAR / Elevation
3D / Lidar
Digitization
Spatial Data Carpentry
- Classification & Segmentation
- Placement (geocoding)
- Conversion (Point>Line>Polygon>Pixels)
- Aggregation
- Transfer (spatial join)
Desktop Software
- ArcGIS Desktop & ArcGIS Pro (W)
- QGIS Desktop (Mac, W, Linux, Android)
- GDAL (Mac, W, Linux)
Cloud GIS
- ArcGIS.com
- Carto.com
- Mapbox.com
- Google Earth Engine
- CloudGIS
Getting Help
- http://gis.stanford.edu
- https://guides.library.stanford.edu/remotesgc/main
- stanfordgis@lists.stanford.edu
- https://stanfordcop.slack.com/ #stanford-geospatial-center
Points, Lines, Polygons & Pixels: Spatial Data 101
By Stace Maples
Points, Lines, Polygons & Pixels: Spatial Data 101
Slide deck for Basic Introduction to Spatial Data Models, Formats and Carpentry.
