Open Data in the City of San Diego

Performance & Analytics Department

City of San Diego

Open Data Policy

• Passed December 2014

• Strong support from Mayor and Council

• Draws on other existing policies

• Defines terms, making sure data meets "open criteria"

• Assigns responsibilities to Chief Data Officer and to City Departments

• Sets timeline 

• Includes reporting requirements to Mayor and Council

Past | Present | Future

Why Open Data matters

• Provide high quality public service
• Work in partnership with all our communities to achieve safe and livable neighborhoods
• Create and sustain a resilient, economically prosperous city

Opening data ties in directly with each of our Strategic Goals and allows us to monitor progress.

How Open Data helps

As a Resident

• Look at the city budget 
  

• Pull in a calendar of events into my phone

• Avoid construction in my commute

• See when my street or sidewalk will be fixed

As a City Employee

• Access data from other departments
• Be transparent
• Reduce time responding to PRA

Benefits of Open Data

• Improve service delivery without increasing resources
• Facilitate intra-departmental data sharing

• Build and integrate city data into applications
• Provide most up-to-date and accurate data to consumers

• Provide city data to power new businesses and startups

Efficiency

Empowerment

Economic Development

Where We Are Today

• Designated information coordinators in City departments

• Identified people working with specific sets of data

• Completed preliminary inventory

• Started the effort to verify and prioritize sets of data

• Investigated options for data portal

How We Got Here

• Databases
• Department spreadsheets
• Shared drives
• Online apps

1. Identify data sources

How We Got Here

What are all the single datasets
you can pull from the data sources?

2. Identify all datasets

How We Got Here

The individual in charge of the datasets – the Data Steward – answers questions
about the data and completes a catalog.

3. Complete dataset catalog

The Backend

Internal dashboards and metrics tracked
each department's progress.

What We Found

• Hundreds of self-reported datasets and data sources
• Lots of low hanging fruit
• We need a way to automate the data inventory process in the future

Where We're NOT Going

Where We're Going

• Timely
• Well-Described
• Reliable 
• Complete
• Used

Where We're Going

Find

Publish

Prioritize

Describe

Clean/Transform

Evaluate

Update

• Value
• Security
• Quality 
• Readiness

Where We're Going

Prioritize

Components

• Base metadata in inventory
• Metadata Schema
• Each Dataset
• Conform to Federal Open Standards

Where We're Going

Describe

• How is the data collected?
• Are there more reliable sources?
• Can we merge the sources together?
• Is the data of high quality?
• Are there gaps in the data that prevent analysis?
• Is there Personally Identifiable Information (PII) in text fields?

Where We're Going

Evaluate

• Systematically Remove PII
• Mold to standard or tidy data
• Combine multiple sources
• Make data useful

Where We're Going

Clean / Transform

Where We're Going

Publish and Update

Today

Manual

Find

Prioritize

Describe

Clean / Transform

Evaluate

Publish

Update

Automatic

Tomorrow

Manual

Find

Prioritize

Describe

Clean / Transform

Evaluate

Publish

Update

Automatic

The Vision

Get the proper stakeholders with the right skills,

involved in a timely manner,

equipped with the appropriate technology and accurate data

to facilitate good decisions

and innovative solutions for our residents.

See this

presentation Online!

Presentation: 

http://sdgo.io/od-council-15