Cooking with Open Source tools

KC CSO Real-time Viewer

June 4, 2014

Peter Keum /
King County Wastewater Treatment Division

King County WTD GIS Specialist

what is open source software?

"Software distributed with three basic conditions"

"Free to Share"

"Free to Modify"

"Free to Share Modifications"

cooking with open source tools

  • Background Information
  • Data Viewer (beta)
  • Work Flow
  • Description of Tools

BackGround Information

King County Wastewater Treatment Division


  • Protect Public Health 
  • Enhance Environment by Treating and Reclaiming water, Recycling Solids and Generating Energy

WTD Facts by numbers

  • 1.5 million people
  • 420 square miles
  • 391 miles sewer pipeline 
  • 47 Pump Stations
  • 38 CSO Outfalls
  • 25 Regulator Stations
  • 3 Regional Treatment Plants
  • 2 Local Treatment Plants

Combined sewer overflow (CSO)

Stormwater + Sewer = CSO

KC CSO Outfalls

Federally (EPA) mandated to show the CSO events and locations.

King County:  38 CSO outfalls 
City of Seattle:  ~ 90 CSO outfalls 
15% of Service area is combined sewer
1950 :    20 Billion Gallons 
1980:    Average 2.3 Billion Gallons
2005:   Reduced to 1 Billion Gallons

Current Real-time CSO viewer

  • Locations
  • Current Status 10min interval
  • Static basemap (no slippy map)

(Beta) Real-time CSO viewer

  • Slippy map - zoom/pan/identify features 
  • Detail information for user


E T L - Extract, Transform, Load

E  T  L

Extract  Transform  Load

1. Extract - python to take csv status file from FTP site.

2. Tranform - python to generate geoJSON file.

3. Load - "push" newly created geoJSON file into GitHub repository


  • Python
  • GitHub
  • geoJSON


Open source scripting  language that is extremely flexible and powerful. 

Natural language that is human-readable expression like  Ruby, Visual Basic and Java

Guido van Rossum 

Developed in 1989



  • Web-based code sharing and publishing service using Git
  • Launched on 2008 
  • 13 million repositories, largest code host

  • Free accounts for open source projects
  • Paid plan for private repositories


Code version control system that manages and stores code projects. 

Open source project initally started 2005 by Linus for Linux


Three important concepts:
"pull request"



Copying a repository from one user's account to another.

You make copy and work as your own. 


"pull request"

Sending a notification to the original owner that you've made some changes that owner might be interested in incorporating 

Process of incorporating modification into one's own project 


  • Data format, format finalized in June 2008
  • JSON (JavaScript Object Notation) + Geospatial data
  • More compact and easily loaded in JavaScript than XML  
  • "Fat Free" XML

  "type": "FeatureCollection",
  "features": [
      "type": "Feature",
      "properties": {},
      "geometry": {
        "type": "Point",
        "coordinates": [


GeoJSON tools : Web-Editor for GeoJSON

GeoJSONLint: GeoJSON validator


Github basemap

  • Wikipedia of world map
  • Freely editable dataset of the globe
  • Easy to edit/modify data onto OSM
  • Emphasis on local knowledge
  • It is NOT 
    • Rendered map
    • Proprietary
    • Static
    • Controlled by authority

project status

  • Live Prototype
  • Initial Testing is promising
  • Migrate python script to enterprise server
  • Internal Staff Usage Test in near future


  • Background Information - WTD
  • CSO outfalls
  • Beta Real-time CSO status viewer
  • Extract Transform Load
  • Various Tools 
    • Python - scripting language
    • GitHub - web service based on Git
    • geoJSON - data format type


Cascadia Users of Geospatial Open Source

Formed in 2007

Meet 3rd Wednesday at ClipCard
Downtown Seattle

International FOSS4G 2014 
Portland, OR

CSO Real-Time Status Viewer

THank you!

questions / comments?

KCGIS User Group Meeting

By Peter Keum

KCGIS User Group Meeting

Presentation for KCGIS User group meeting June 4, 2014

  • 2,423
Loading comments...

More from Peter Keum