UNIX SHELL PROGRAMMING
OR
HOW CAN I DO CRAZY $H17 ON THE COMMAND LINE?
Zoltan Hawryluk, zoltan.dulac@gmail.com
Code Examples available at:
https://github.com/zoltan-dulac/unix-examples-presentation
WHY SHOULD I CARE ABOUT THE COMMAND LINE?
- The command line is the Swiss Army Knife for any developer
- It is cross OS compatible.
- You can intimidate designers into thinking you are manipulating the matrix
- You don't have to spend money on apps to do complex things
- It's like the LEGO of programming tools.
obligitory history
Ken Thompson and Dennis Richie, circa 1970
HISTORY OF UNIX
- Invented circa 1970 by Ken Thompson and Dennis Richie
- Rewritten in a then new programming language called C in 1972
- Two main forks (System V and BSD).
- System V is the basis for Linux
- BSD is the basis for OSX.
- Designed with a simple philosophy
UNIX PHILOSOPHY
- Write programs that do one thing and do it well.
- Write programs to work together.
- Write programs to handle text streams, because that is a universal interface.
A SIMPLE TEXT FILE
mcdonalds.csv
find all lines with "cookies" using Grep
grep -i "cookie" mcdonalds.csv
LIST ALL THE ITEMS IN MENU WITHOUT PRICES
awk -F"$" '{print $1}' mcdonalds.csv
PUT THE DATA IN TABLE MARKUP
awk -F"$" '{print "<tr><td>"$1"</td><td>$2</td></tr>"}' mcdonalds.csv
PUT THE SORTED DATA IN TABLE MARKUP
sort mcdonalds.csv | awk -F"$" '{print "<tr><td>"$1"</td><td>$2</td></tr>"}'
PUT THE SORTED DATA IN TABLE
sort mcdonalds.csv |
awk -F"$" '
BEGIN {print "<table>"}
{print "<tr><td>"$1"</td><td>$2</td></tr>"}
END {print "</table>"}
'
PUT THE SORTED table IN A VALID FORMATTED HTML FILE
sort mcdonalds.csv |
awk -F"$" '
BEGIN {print "<table>"}
{print "<tr><td>"$1"</td><td>"$2"</td></tr>"}
END {print "</table>"}
' |
tidy -indent
SAVE THE SORTED table INTO AN HTML FILE
sort mcdonalds.csv |
awk -F"$" '
BEGIN {print "<table border=\"1\">"}
{print "<tr><td>"$1"</td><td>"$2"</td></tr>"}
END {print "</table>"}
' |
tidy -indent > sorted_price.html
WAIT - WHY ARE ERRORS ON THE SCREEN?
- UNIX has a concept of stdout and stderr
- Output goes to stdout
- Error messages go to stderr
- You can redirect stdout with > and stderr with 2>
REDIRECTING STDERR
sort mcdonalds.csv |
awk -F"$" '
BEGIN {print "<table border=\"1\">"}
{print "<tr><td>"$1"</td><td>"$2"</td></tr>"}
END {print "</table>"}
' |
tidy -indent > sorted_price.html 2> errors.txt
IGNORING STDERR
sort mcdonalds.csv |
awk -F"$" '
BEGIN {print "<table border=\"1\">"}
{print "<tr><td>"$1"</td><td>"$2"</td></tr>"}
END {print "</table>"}
' |
tidy -indent > sorted_price.html 2> /dev/null
doing mindless stuff on a whole bunch of files
LIST ALL HTML FILES
find . -name "*.html"
PUT THIS LIST IN A VARIABLE
X=`find . -name "*.html"`
sort THe list using Sort
sort mcdonalds_prices.csv
sort by second column numerically
sort -t$ -n -k2 mcdonalds_prices.csv
save the output to a file using redirection
sort -t$ -n -k2 mcdonalds_prices.csv > sorted_prices.csv
COMBINE FUNCTIONALITY USING A PIPE
grep -i burger mcdonalds_prices.csv | awk -F"$" '{print $1}'
COMBINE FUNCTIONALITY USING A PIPE
grep -i burger mcdonalds_prices.csv | awk -F"$" '{print $1}' | sort
UNIX SHELL PROGRAMMING
By Zoltan Hawryluk
