Cooking with Open Source tools

KC CSO Real-time Viewer

June 4, 2014

Peter Keum

peter.keum@kingcounty.gov /mapitpk.com

https://github.com/keum

King County Wastewater Treatment Division

King County WTD GIS Specialist

what is open source software?

"Software distributed with three basic conditions"

"Free to Share"

"Free to Modify"

"Free to Share Modifications"

cooking with open source tools

Background Information

Data Viewer (beta)

Work Flow

Description of Tools

BackGround Information

King County Wastewater Treatment Division

Mission:

Protect Public Health
Enhance Environment by Treating and Reclaiming water, Recycling Solids and Generating Energy

WTD Facts by numbers

1.5 million people

420 square miles

391 miles sewer pipeline

47 Pump Stations

38 CSO Outfalls

25 Regulator Stations

3 Regional Treatment Plants

2 Local Treatment Plants

Combined sewer overflow (CSO)

Stormwater + Sewer = CSO

KC CSO Outfalls

Federally (EPA) mandated to show the CSO events and locations.

King County: 38 CSO outfalls

City of Seattle: ~ 90 CSO outfalls

15% of Service area is combined sewer

1950 : 20 Billion Gallons

1980: Average 2.3 Billion Gallons

2005: Reduced to 1 Billion Gallons

Current Real-time CSO viewer

Locations
Current Status 10min interval
Static basemap (no slippy map)

(Beta) Real-time CSO viewer

Slippy map - zoom/pan/identify features
Detail information for user

Workflow

E T L - Extract, Transform, Load

E T L

Extract Transform Load

1. Extract - python to take csv status file from FTP site.

2. Tranform - python to generate geoJSON file.

3. Load - "push" newly created geoJSON file into GitHub repository

tools

Python

GitHub

geoJSON

python

Open source scripting language that is extremely flexible and powerful.

Natural language that is human-readable expression like Ruby, Visual Basic and Java

Guido van Rossum

Developed in 1989

PYthon

Web-based code sharing and publishing service using Git
Launched on 2008
13 million repositories, largest code host

Free accounts for open source projects
Paid plan for private repositories

git

Code version control system that manages and stores code projects.

Open source project initally started 2005 by Linus for Linux

GIthub

Three important concepts:

"forking"

"pull request"

"merge"

Learning GitHub

github

"Forking"

Copying a repository from one user's account to another.

You make copy and work as your own.

github

"pull request"

Sending a notification to the original owner that you've made some changes that owner might be interested in incorporating

"merge"

Process of incorporating modification into one's own project

geojson

Data format, format finalized in June 2008
JSON (JavaScript Object Notation) + Geospatial data
More compact and easily loaded in JavaScript than XML
"Fat Free" XML

 {
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "properties": {},
      "geometry": {
        "type": "Point",
        "coordinates": [
          -122.33103632926941,
          47.59887827375554
        ]
      }
    }
  ]
}