prove your stories

WITH DATA


https://slid.es/aplayford/2014-li-press-data/

ADAM PLAYFORD

NEWSDAY

@adamplayford




Hello.




my name is Adam.




i work for Newsday



(that's RIGHT HERE)


I WRITE:


STORIES

COMPUTER PROGRAMS

DATABASE QUERIES




WHO ARE

YOU?






why we're here



he said/she said

sucks

(sorry)


life goal:

suck

less

(A LITTLE  every day)





proving things

helps me suck less




data

helps me prove things


you only need...


  1. A theory

  2. A way to test it

  3. A computer



ok SURE, MR. DATA GUY

but how?





A THEORY

(OR HYPOTHESIS TESTING)

(WOW, THAT'S A LONG WORD.)




A THREAT FROM THE UNION




ELECTION DAY




THE HUNCH




The Post matched employee databases from the school district with county voter records. To ensure accuracy, reporters then checked hundreds of records by hand.



"ELECTION DAY CAME

AND THE VAST MAJORITY OF TEACHERS

DIDN'T SHOW UP."





Teacher turnout was 24 percent, the Post found. Although significantly higher than the turnout of other county voters, it wasn't enough to change the outcome of a single district race in August.






In the county, the belief that most teachers don't vote has been talked about -- quietly -- for years.

the rankings

42%: school librarians

33%: History Teachers

32%: Principals/asst. principals

29%: Music teachers

25%: Math teachers

24%: English teachers

22%: Physical ed teachers

20%: Elementary teachers

data source #1


bread and butter STUFF


  • Convictions
  • Public employee names & salaries
  • Land sales
  • Corporate records
  • Voter registration
  • Vendor lists
  • Check registers
For nearly five years, the Broward County School District has been paying a former head custodian more than $100,000 a year to teach school janitors the finer points of cleaning.
Reynolds Hedland III, 52, has no college degree or state teaching certificate, yet earns more than 99 percent of Broward County's teachers. His lessons include how to mop and scrub bathrooms, strip and wax floors, and "maintain the cleanliness, orderliness, appearance and safe condition of schools."
He's scheduled to teach only 58 days this year ...
http://bit.ly/1gVdEuV



case study


CONFESSIONS

IN SUFFOLK


YOU NOTICE...


IN SUFFOLK,

THE COPS always get their man...


...to confess




HUH!


WHERE FROM HERE?



A year-long Newsday study has found that 94 percent of the murder defendants in Suffolk between 1975 and 1985 allegedly made incriminating statements. None of the other six suburban counties studied by Newsday has a rate close to Suffolk's. Their rates range from 54 percent in Montgomery County, Maryland, to 74 percent in DuPage County, Ill.

NEWSDAY, 1986
By THOMAS MAIER and REX SMITH 



DATA SOURCE #2


MAKE IT


Police misconduct


Nassau's meeting lasted only seven minutes, which is not unusual. Since 2007, 80 percent of the committee's meetings lasted less than half an hour. Half ended in less than 10 minutes.
Newsday / For Their Eyes Only

stand your ground



In the most comprehensive effort of its kind, the Tampa Bay Times has identified nearly 200 "stand your ground'' cases and their outcomes. The Times identified cases through media reports, court records and dozens of interviews with prosecutors and defense attorneys across the state.

Stand your ground


• People often go free under "stand your ground" in cases that seem to make a mockery of what lawmakers intended. One man killed two unarmed people and walked out of jail. Another shot a man as he lay on the ground. Others went free after shooting their victims in the back. In nearly a third of the cases the Times analyzed, defendants initiated the fight, shot an unarmed person or pursued their victim — and still went free.
Tampa Bay Times / Stand Your Ground project

DATA SOURCE #4



FOIL




45 seconds on

FOILing

for data




if they have data electronically,

you can get it
that way


new york state law:

When an agency has the ability to retrieve or extract a record or data maintained in a computer storage system with reasonable effort, it shall be required to do so. When doing so requires less employee time than engaging in manual retrieval or redactions from non-electronic records, the agency shall be required to retrieve or extract such record or data electronically.
Committee on Open Gov't site


in other words...

always ask for everything

(which is great advice!)


you can use language like...


As this information is stored in an electronic database, I request it be provided electronically, in a spreadsheet or other delimited text-file.

I LIKE:

I request these records be provided in an electronic format that can be imported into standard database software. Examples of such formats include an Excel .xls or .xlsx file, an Access .mdb or .accdb file, a text-based delimited file such as .csv or tab-delimited .txt, a .dbf file or an SQL dump readable by standard open-source database software. (A PDF file would not comply with this request because PDF files are not readable by database software.) If this information is stored in a relational database, I request it be provided in its original relational format, not "flattened" or de-normalized. 


Translation:

1) GIVE ME YOUR NERDS

2) I can take anything



... but no pdfs



no pdfs


case study



"We don't have it

that way"

[RE: Day care inspections]


Q: how do you collect it?


A: inspectors write reports on computer tablets


Q: ... uhh ...


(we got the data)

in other words...


the same things that make

a good reporter

make you good at

negotiating for data



understand the process







HOW DO THEY KEEP IT?


ADAM'S FAVORITE:



MULTIPLE

DATABASES

(WORKING THE INTERSECTIONS...)


"JOINING':

HARD, AWESOME



tools


spreadsheets


  • Sorting
  • Charting
  • Counting/summing/grouping
    (pivot tables)
  • Math!


EXCEL, GOOGLE SPREADSHEETS


databases

  • Joining
    (combining multiple
    data sources together)
  • Analyzing more complicated
    data (nerds say: "relational")


    MICROSOFT ACCESS

    SQL [SQLITE, MYSQL, POSTGRESQL]


    today we'll be playing

    with data from...



    national center

    for education statistics

    (http://nces.ed.gov)



    all done!


    Questions?


    https://slid.es/aplayford/2014-li-press-data/

    @adamplayford
    adam.playford@newsday.com