Cooking with Open Source tools
KC CSO Real-time Viewer
June 4, 2014
Peter Keum
peter.keum@kingcounty.gov /mapitpk.com
King County Wastewater Treatment Division
King County WTD GIS Specialist
what is open source software?
"Software distributed with three basic conditions"
"Free to Share"
"Free to Modify"
"Free to Share Modifications"
cooking with open source tools
- Background Information
- Data Viewer (beta)
- Work Flow
-
Description of Tools
BackGround Information
King County Wastewater Treatment Division
Mission:
- Protect Public Health
-
Enhance Environment by Treating and Reclaiming water, Recycling Solids and Generating Energy
WTD Facts by numbers
- 1.5 million people
- 420 square miles
- 391 miles sewer pipeline
- 47 Pump Stations
- 38 CSO Outfalls
- 25 Regulator Stations
- 3 Regional Treatment Plants
- 2 Local Treatment Plants
Combined sewer overflow (CSO)
Stormwater + Sewer = CSO
KC CSO Outfalls
Federally (EPA) mandated to show the CSO events and locations.
King County: 38 CSO outfalls
City of Seattle: ~ 90 CSO outfalls
15% of Service area is combined sewer
1950 : 20 Billion Gallons
1980: Average 2.3 Billion Gallons
2005: Reduced to 1 Billion Gallons
Current Real-time CSO viewer
- Locations
- Current Status 10min interval
- Static basemap (no slippy map)
(Beta) Real-time CSO viewer
- Slippy map - zoom/pan/identify features
- Detail information for user
Workflow
E T L - Extract, Transform, Load
E T L
Extract Transform Load
1. Extract - python to take csv status file from FTP site.
2. Tranform - python to generate geoJSON file.
3. Load - "push" newly created geoJSON file into GitHub repository
tools
- Python
- GitHub
- geoJSON
python
Open source scripting language that is extremely flexible and powerful.
Natural language that is human-readable expression like Ruby, Visual Basic and Java
Guido van Rossum
Developed in 1989
PYthon
-
Web-based code sharing and publishing service using Git
- Launched on 2008
-
13 million repositories, largest code host
-
Free accounts for open source projects
-
Paid plan for private repositories
git
Code version control system that manages and stores code projects.
Open source project initally started 2005 by Linus for Linux
GIthub
Three important concepts:
"forking"
"pull request"
"merge"
github
"Forking"
Copying a repository from one user's account to another.
You make copy and work as your own.
github
"pull request"
Sending a notification to the original owner that you've made some changes that owner might be interested in incorporating
"merge"
Process of incorporating modification into one's own project
geojson
- Data format, format finalized in June 2008
- JSON (JavaScript Object Notation) + Geospatial data
- More compact and easily loaded in JavaScript than XML
- "Fat Free" XML
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {},
"geometry": {
"type": "Point",
"coordinates": [
-122.33103632926941,
47.59887827375554
]
}
}
]
}
GeoJSON
GeoJSON tools
Geojson.io : Web-Editor for GeoJSON
GeoJSONLint: GeoJSON validator
OpenStreetmap
Github basemap
OSM: OpenStreetMap
- Wikipedia of world map
- Freely editable dataset of the globe
- Easy to edit/modify data onto OSM
- Emphasis on local knowledge
- It is NOT
- Rendered map
- Proprietary
- Static
- Controlled by authority
project status
- Live Prototype
- Initial Testing is promising
- Migrate python script to enterprise server
-
Internal Staff Usage Test in near future
Summary
- Background Information - WTD
- CSO outfalls
- Beta Real-time CSO status viewer
- Extract Transform Load
- Various Tools
- Python - scripting language
- GitHub - web service based on Git
- geoJSON - data format type
CUGOS
Cascadia Users of Geospatial Open Source
Formed in 2007
Meet 3rd Wednesday at ClipCard
Downtown Seattle
International FOSS4G 2014
Portland, OR
CSO Real-Time Status Viewer
THank you!
questions / comments?
KCGIS User Group Meeting
By Peter Keum
KCGIS User Group Meeting
Presentation for KCGIS User group meeting June 4, 2014
- 3,971