Open Data Overview

Maksim Pecherskiy

Performance & Analytics Department

Who Is This Guy?

• Chief Data Officer, City of San Diego
• Performance & Analytics Department
• Lots of years doing software engineering
• Worked in Puerto Rico through Code for America
• Saw how powerful data can be when used inside government

Why?

• Provide High Quality Public Service
• Work in partnership with all our communities to achieve safe and livable neighborhoods.
• Create and sustain a resilient, economically prosperous city.

Opening data ties in directly with each of our Strategic Goals and allows us to monitor how we're doing at meeting them

San Diego

Open Data Policy

• Passed December 2014.
• Mayor strong support.
• Council strong support.

• Draws on other existing policies.

• Defines terms, making sure data meets "open criteria"

• Assigns responsibilities to Chief Data Officer and to City Departments

• Sets timeline 

• Includes reporting requirements to Mayor and Council

Let's Talk About Data

State of Data, SD, 2015

State of Data, SD, 2015

• Closed
• Misunderstood
• Hard to find
• Unknown
• Scattered
• Uncentralized

State of Data, SD, 2015

• PRA Nightmares
• Misinterpretation
• Slowdowns
• Re-Work
• Lack of Innovation
• Frustration

And How It Feels to Work With It

• Download PDF
• Download Tabula
• Download Java
• Extract Page #5
• Run Tabula
• Select and Extract Table Data
• Import into Excel
• Fix headers
• Paste into viz tool
• Visualize

Total Time: 1 Hour

• Accessible
• Described
• Reusable
• Timely
• Complete
• Treated like asset
• Centralized
• Machine Readable

Where we can be

• Efficient PRA
• No Re-Work
• Innovation
• Empowerment
• Efficiency
• Engaged Citizens
• Minimized Misinterpretation
• Reliability
• Data Usage for decisions
• Cool apps!

We Can...

• Empower people to build applications.
• Empower consumers of those applications.
• Allow city employees to be more efficient and innovative.
• Allow taxpayers to benefit from a more efficient, nimble government
• Build our own dashboards
• Communicate with our citizens
• Spend less time on PRAs
• Do our own analysis and make data driven decisions
• Give data back to their owner - the taxpayer!

"Open data is not just about putting spreadsheets on the internet.

It means being deliberate and thoughtful about what is released, how it's released, and how it's described. 

Treating data like an asset and releasing it properly goes directly hand-in-hand with making sure that it's secure and mitigating opportunities for misuse"

We Can go From This

To This

San Diego

Can Be A Leader

In This Space.

Quick Vocab Lesson

• Machine Readable, Open Format
  • Reasonably structured to allow automated processing
  • Non proprietary, publicly available, with no licensing restrictions (XLS vs CSV)
• Datasource

  • Technology or system that stores data, including databases, certain spreadsheets, information systems, business/online applications, etc.

• Dataset
  • Contents of a single database table, worksheet or defined view;
  • Anything that can be used to build a table / chart / map

What is [Open] Data?

What is Data?

A value or set of values representing a specific concept or concepts.

Data become “information” when analyzed and possibly combined with other data in order to extract meaning and to provide context.

The meaning of data can vary depending on its context.

What is Data?

"Data is something you can take, and do something else with."

(Besides print it or send it in an e-mail).

What is Data?

"Data is something you can take, and do something else with."

(Besides print it or send it in an e-mail).

NOT Data

Data

Open Data

Open Data is

Data in an Open Machine Readable Format

• CSV (not XLS)
• ShapeFiles
• GeoJSON
• iCal
• JSON
• XML
• API

Data

Not Beautiful

[Open] Data is Not

A Website

A Dashboard or Chart

A Map

A PDF / Mobile App or E-Mail

But It Can Enable Them

Data

OR

OR

We MUST allow the capability to separate data from how it's shown or used in order for it to have value beyond what the visualization / tool intended

The user may not share your opinion of how to use the data

Open Data can be separated from how it's and used.

Quick Vocab Lesson

• ETL
  • Extract, Transform, Load
  • Pull data from a database, change it around based on specification for release, upload it to a portal.
• Metadata
  • Data that describes data
• Data Portal
  • A place on the internet for our data to live

Machine Readable, Open Formats

Great! How Can I Help?

• Inventory
• Begin thinking of our data as a valuable asset.
• Open-Data Friendly Technology Decisions.
  • Can data be easily extracted out of this tool?
  • Does this tool expose its data in a re-usable and open format?
  • Who owns the data in this tool? Us or the vendor?
  • Call Maksim and Let's Work Together!

Upcoming Key Dates

• March 31, 2015 - Guidelines for Data Inventory
• June 1, 2015 - Inventory to be completed by departments
• July 1, 2015 - Technical Guidelines (To address release protocol and PII)
• July 1, 2015 - Initial Written Status Report

The Inventory

• Due June 1, 2015
• Continous, Recurring Process
• Need to get a bird's eye view of information we have
• Will help people across departments find data faster

Not All Inventoried Datasets will be Released

The Inventory - 3 Main Steps

• Identify Data Sources - April
• Identify Data Sets - May 1
• Complete Catalog - June 1

After the Inventory?

• Technical Guidelines
  • PII
  • Publishing Plan
  • Data Review 
• Prioritization Plan
• Review Inventory, prioritize

See this

presentation Online!

Presentation: bit.ly/odoverview

Yes, There's a PDF version:  bit.ly/odoverviewpdf

Let's Do

Something

AWESOME

Together!

maksimp@sandiego.gov