Social and Political Data Science: Introduction

Methods of Data Collection and Production

Karl Ho

School of Economic, Political and Policy Sciences

University of Texas at Dallas

Event Data

What is Event Data?

Event data is any data measuring about an event. Event data can come from many sources. 

- IBM Predictive Maintenance and Quality

For example, an event is recording an inspection result, one might want to record: who was the inspector, when did the event take place, which product lot was it based on, and what was the result of the inspection

What is Political Event Data?

Political event data are records of interactions (event) among political actors (source, target).  

This kind of event data allows for aggregate analysis of political behaviors (e.g. country as a unit of analysis).   

Individual events can also be used as unit of analysis (e.g. terrorist attack, negotiation, etc.)

Conflict and Mediation Event Observation  Ontology (CAMEO)

  • who-did-what-to-whom

  • Categories: 1-20

    • Sub-categories: 00-99

  • e.g. The PRC on Tuesday demanded that the US cancel plans to sell air-to-surface anti-tank weapons to Taiwan to avoid “new damage” to US-PRC relations.

Conflict and Mediation Event Observation  Ontology (CAMEO)

"_id" "code" "src_actor" "month" 
"tgt_agent" "country_code" "year" "date8_val" 
"id" "source" "date8" "src_agent" 
"tgt_actor" "latitude" "src_other_agent" "quad_class"
"source_text" "root_code" "tgt_other_agent" "day" 
"target" "goldstein" "geoname" "longitude" 
"url" 

Workshop: UTDEventData_Demo.R

#######################
#Description: This is a script to pull event data using the UTDEventData R package.
#Publication: Kim, HyoungAh, Vito D’Orazio, Patrick T. Brandt, Jared Looper, Sayeed Salam, Latifur Khan, and Michael Shoemate. 2019. “UTDEventData: An R package to access political event data.” Journal of Open Source Software. 4(36), 1322, doi:doi:10.21105/joss.01322
#Author of this script version: Marcus Sianan
#Main project website: https://eventdata.utdallas.edu
#UTDEventData R package GitHub: https://github.com/KateHyoung/UTDEventData
#UTDEventData R package vignette: https://github.com/KateHyoung/UTDEventData/raw/UTDEventData/UTDEventData.pdf
#API key signup: https://eventdata.utdallas.edu/signup (do not have students sign up all at once). Check your spam/junk folders if it does not appear in your inbox!
#List of sources from which we scrape data: https://github.com/openeventdata/scraper/blob/master/whitelist_urls.csv
#Dictionaries (e.g., to help with deciphering acronyms): https://github.com/openeventdata/petrarch2/tree/master/petrarch2/data/dictionaries
#GitHub repo with instructions on one way to visualize event data: https://github.com/eventdata/igraph
#######################

#Install the packages and load the libraries.
install.packages("devtools")
library(remotes)
install_github("KateHyoung/UTDEventData")
library(devtools)
library(UTDEventData)

#Insert your API key between the below quotes.
k <- ""

table <- "phoenix_rt" #Phoenix_RT is the name of the data table that stores the event data (other data tables can be found at https://eventdata.utdallas.edu).
#At the moment, Phoenix_RT data is only available from 20171031 to 20200825. We are currently testing new data to eventually expand the size of Phoenix_RT.

countries <- c("CHN","USA") #It accepts full names and ISO codes (e.g., "Germany" or "DEU" will work for Germany). If one country is desired, use list("CAN") or list("JPN"). 

start_date <- "20171031" #Make sure there is a "0" in front of the single-digit months and days.

end_date <- "20171107" #Make sure there is a "0" in front of the single-digit months and days.

EventData <- pullData(k, table, countries, start_date, end_date, citation = FALSE) #This forms the URL query to get Phoenix_RT data from the server. You can copy/paste the URL output into your browser to see the data.
#If you get the below warning message, that is okay. It deals with open API calls and indicates that repeated attempts to the server were made.
#Warning message:
#In .Internal(gc(verbose, reset, full)) :
#closing unused connection...

View(EventData)