prove your stories
WITH DATA
ADAM PLAYFORD
NEWSDAY
@adamplayford
Hello.
my name is Adam.
i work for Newsday
(that's RIGHT HERE)
I WRITE:
STORIES
COMPUTER PROGRAMS
DATABASE QUERIES
WHO ARE
YOU?
why we're here
he said/she said
sucks
(sorry)
life goal:
suck
less
(A LITTLE every day)
proving things
helps me suck less
data
helps me prove things
you only need...
-
A theory
- A way to test it
- A computer
ok SURE, MR. DATA GUY
but how?
A THEORY
(OR HYPOTHESIS TESTING)
(WOW, THAT'S A LONG WORD.)
A THREAT FROM THE UNION
ELECTION DAY
THE HUNCH
The Post matched employee databases from the school district with county voter records. To ensure accuracy, reporters then checked hundreds of records by hand.
"ELECTION DAY CAME
AND THE VAST MAJORITY OF TEACHERS
DIDN'T SHOW UP."
Teacher turnout was 24 percent, the Post found. Although significantly higher than the turnout of other county voters, it wasn't enough to change the outcome of a single district race in August.
In the county, the belief that most teachers don't vote has been talked about -- quietly -- for years.
the rankings
42%: school librarians
33%: History Teachers
32%: Principals/asst. principals
29%: Music teachers
25%: Math teachers
24%: English teachers
22%: Physical ed teachers
20%: Elementary teachers
data source #1
bread and butter STUFF
- Convictions
- Public employee names & salaries
- Land sales
- Corporate records
- Voter registration
- Vendor lists
- Check registers
For nearly five years, the Broward County School District has been paying a former head custodian more than $100,000 a year to teach school janitors the finer points of cleaning.
Reynolds Hedland III, 52, has no college degree or state teaching certificate, yet earns more than 99 percent of Broward County's teachers. His lessons include how to mop and scrub bathrooms, strip and wax floors, and "maintain the cleanliness, orderliness, appearance and safe condition of schools."
He's scheduled to teach only 58 days this year ...http://bit.ly/1gVdEuV
case study
CONFESSIONS
IN SUFFOLK
YOU NOTICE...
IN SUFFOLK,
THE COPS always get their man...
...to confess
HUH!
WHERE FROM HERE?
A year-long Newsday study has found that 94 percent of the murder defendants in Suffolk between 1975 and 1985 allegedly made incriminating statements. None of the other six suburban counties studied by Newsday has a rate close to Suffolk's. Their rates range from 54 percent in Montgomery County, Maryland, to 74 percent in DuPage County, Ill.
NEWSDAY, 1986
By THOMAS MAIER and REX SMITH
DATA SOURCE #2
MAKE IT
Police misconduct
Nassau's meeting lasted only seven minutes, which is not unusual. Since 2007, 80 percent of the committee's meetings lasted less than half an hour. Half ended in less than 10 minutes.Newsday / For Their Eyes Only
stand your ground
In the most comprehensive effort of its kind, the Tampa Bay Times has identified nearly 200 "stand your ground'' cases and their outcomes. The Times identified cases through media reports, court records and dozens of interviews with prosecutors and defense attorneys across the state.
Stand your ground
• People often go free under "stand your ground" in cases that seem to make a mockery of what lawmakers intended. One man killed two unarmed people and walked out of jail. Another shot a man as he lay on the ground. Others went free after shooting their victims in the back. In nearly a third of the cases the Times analyzed, defendants initiated the fight, shot an unarmed person or pursued their victim — and still went free.Tampa Bay Times / Stand Your Ground project
DATA SOURCE #4
FOIL
45 seconds on
FOILing
for data
if they have data electronically,
you can get it
that way
new york state law:
When an agency has the ability to retrieve or extract a record or data maintained in a computer storage system with reasonable effort, it shall be required to do so. When doing so requires less employee time than engaging in manual retrieval or redactions from non-electronic records, the agency shall be required to retrieve or extract such record or data electronically.
in other words...
always ask for everything
(which is great advice!)
you can use language like...
As this information is stored in an electronic database, I request it be provided electronically, in a spreadsheet or other delimited text-file.
I LIKE:
I request these records be provided in an electronic format that can be imported into standard database software. Examples of such formats include an Excel .xls or .xlsx file, an Access .mdb or .accdb file, a text-based delimited file such as .csv or tab-delimited .txt, a .dbf file or an SQL dump readable by standard open-source database software. (A PDF file would not comply with this request because PDF files are not readable by database software.) If this information is stored in a relational database, I request it be provided in its original relational format, not "flattened" or de-normalized.
Translation:
1) GIVE ME YOUR NERDS
2) I can take anything
... but no pdfs
no pdfs
case study
"We don't have it
that way"
[RE: Day care inspections]
Q: how do you collect it?
A: inspectors write reports on computer tablets
Q: ... uhh ...
(we got the data)
in other words...
the same things that make
a good reporter
make you good at
negotiating for data
understand the process
HOW DO THEY KEEP IT?
ADAM'S FAVORITE:
MULTIPLE
DATABASES
(WORKING THE INTERSECTIONS...)
"JOINING':
HARD, AWESOME
tools
spreadsheets
- Sorting
- Charting
- Counting/summing/grouping
(pivot tables) - Math!
EXCEL, GOOGLE SPREADSHEETS
databases
- Joining
(combining multiple
data sources together) - Analyzing more complicated
data (nerds say: "relational")
MICROSOFT ACCESS
SQL [SQLITE, MYSQL, POSTGRESQL]
today we'll be playing
with data from...
national center
for education statistics
(http://nces.ed.gov)
all done!
Questions?
@adamplayford
adam.playford@newsday.com
Prove your story with data
By Adam Playford
Prove your story with data
LI Press Club 3/6/2014
- 1,980