Database Journalist, Politics - FiveThirtyEight
Democracy and Technology Fellow - AshCenter
dhrumil.mehta@fivethirtyeight.com
@datadhrumil
@dmil
# Install Homebrew Package Manager
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
# Install Caskroom (to learn more...https://caskroom.github.io/)
brew install caskroom/cask/brew-cask
brew tap caskroom/versions
# Install Some Coding Tools
brew cask install iterm2
brew cask install sublime-text3
# Install Node and Underscore CLI
brew install node
npm install -g underscore-cli
# Default package manager is Apt-Get
# Install Sublime Text
sudo add-apt-repository ppa:webupd8team/sublime-text-3
sudo apt-get update
sudo apt-get install sublime-text-installer
# Install Node and Underscore CLI
sudo apt-get install nodejs
sudo apt-get install npm
npm install -g underscore-cli
MAC
Ubuntu
Agenda
whoami # your username
hostname # my computer's network name
pwd # print working directory
cd # change directory
ls # list directory
mkdir # make directory
rmdir # remove directory
cp # copy a file or directory
mv # move a file or directory
cat # print a whole file
head # print first few lines of a file
tail # print last few lines of a file
less # page through a file
find # find files
grep # find things inside files
man # read a manual page
env # look at your environment
echo # print some arguments
export # export/set a new environment variable
exit # exit the shell
sudo # DANGER! become super user root DANGER!
chmod # change permission modifiers
chown # change ownership
echo "Hello, World." > hello.txt
# ">" Grabs the output from the command on the
# left and feeds it to the file on the right
echo "Goodbye, World." >> hello.txt
# ">>" is similar to ">" except if the file already
# exits it will append to the end of it rather than
# overwriting the file
grep World hello.txt
# grep is a command that searches a file for a
# regular expression, we can try it on hello.txt
ls -a1 | grep .txt
# "|" takes the output of the command to the left and
# uses it as the input for the command to the right
# Initialize the local directory as a Git repository.
git init
# Add the files to your new repository's staging area.
git add <filename>
# View which files are in the staging area
git status
# Commit the files that you've staged in your local repository
# with a descriptive commit message.
git commit -m "<commit message>"
# Sets the new remote
git remote add origin remote repository URL
# Verifies the new remote URL
git remote -v
# Push the changes in your local repository to GitHub.
git push origin master
git add <filename>
git commit -m "<commit message>"
git push
(Open Source Collaboration Model)
http://stackoverflow.com/questions/3611256/forking-vs-branching-in-github
(Open Source Collaboration Model)
# 1) Fork this repo in github
# https://github.com/dmil/tools-of-the-trade
# 2) Make a local clone of your fork
git clone <url_of_your_fork>
# 3) Add yourself to the list of people who know how to fork
echo "Dhrumil Mehta" >> forkers.md
# 4) Commit your changes
git add forkers.md
git commit -m "Add dhrumil to list of people who fork"
# 5) Push the file to your own github
git push origin master
# 6) Issue a pull request to me on github
# 7) Wait for me to merge the pull requests
# 8) the changes from upstream
git remote add upstream https://github.com/dmil/tools-of-the-trade.git
git pull upstream
(and sometimes Pull Requests)
# Head over to GitHub and create a new repository
# named username.github.io, where username is your username
# Clone your new repository
git clone <your new repo>
# Create an index.html file
cd <your_username>.github.io
~$echo "Hello World" > index.html
# Push to github
git commit -m "Create personal github page"
git add --all
git push -u origin master
# Voila!
open http://<your_username>.github.io
What is going on when a page loads?
GET | POST |
---|---|
Requests data from a specific resource. | Submits data to be processed by a specific resource. |
Data is submitted as part of the URL | Data is submitted in the request body |
Less secure but faster | More secure but slower |
Can be cached by browser | Not Cached by Browser |
Length Limited by URL size | MaxLength determined by server |
Lets try it!
(When people want you to have data)
FEC API - https://api.open.fec.gov/developers/
HuffPost Pollster API - https://api.open.fec.gov/developers/
Some other good ones:
(and validation)
curl -L https://github.com/dmil/tools-of-the-trade/blob/master/roster.xlsx?raw=true
in2csv roster.xlsx > roster.csv
csvcut roster.csv -c 8 | tail -n +2 | awk '{gsub("\"", "")}1' | cut -d'|' -f 1- --output-delimiter=$'\n' | sed 's/^[ \t]*//;s/[ \t]*$//' | sort | uniq -cd
# Or if I wanted to be even more obnoxious
in2csv roster.xlsx | csvcut -c 8 | tail -n +2 | awk '{gsub("\"", "")}1' | cut -d'|' -f 1- --output-delimiter=$'\n' | sed 's/^[ \t]*//;s/[ \t]*$//' | sort | uniq -cd
dhrumil.mehta@fivethirtyeight.com
@datadhrumil
@dmil