UNIX SHELL PROGRAMMING

OR

 

HOW CAN I DO CRAZY $H17 ON THE COMMAND LINE?

Zoltan Hawryluk, zoltan.dulac@gmail.com


Code Examples available at:
https://github.com/zoltan-dulac/unix-examples-presentation

WHY SHOULD I CARE ABOUT THE COMMAND LINE?

  • The command line is the Swiss Army Knife for any developer
  • It is cross OS compatible.
  • You can intimidate designers into thinking you are manipulating the matrix
  • You don't have to spend money on apps to do complex things
  • It's like the LEGO of programming tools.

obligitory history

Ken Thompson and Dennis Richie, circa 1970

HISTORY OF UNIX

  1. Invented circa 1970 by Ken Thompson and Dennis Richie
  2. Rewritten in a then new programming language called C in 1972
  3. Two main forks (System V and BSD).
  4. System V is the basis for Linux
  5. BSD is the basis for OSX.
  6. Designed with a simple philosophy

UNIX PHILOSOPHY

  • Write programs that do one thing and do it well.
  • Write programs to work together.
  • Write programs to handle text streams, because that is a universal interface.

A SIMPLE TEXT FILE


mcdonalds.csv

find all lines with "cookies" using Grep


grep -i "cookie" mcdonalds.csv

LIST ALL THE ITEMS IN MENU WITHOUT PRICES


awk -F"$" '{print $1}' mcdonalds.csv

PUT THE DATA IN TABLE MARKUP


awk -F"$" '{print "<tr><td>"$1"</td><td>$2</td></tr>"}' mcdonalds.csv

PUT THE SORTED DATA IN TABLE MARKUP


sort mcdonalds.csv | awk -F"$" '{print "<tr><td>"$1"</td><td>$2</td></tr>"}'

PUT THE SORTED  DATA IN TABLE

sort mcdonalds.csv |

    awk -F"$" '

        BEGIN {print "<table>"}

        {print "<tr><td>"$1"</td><td>$2</td></tr>"}

        END {print "</table>"}

    '

PUT THE SORTED  table IN A VALID FORMATTED HTML FILE

sort mcdonalds.csv |

    awk -F"$" '

        BEGIN {print "<table>"}

        {print "<tr><td>"$1"</td><td>"$2"</td></tr>"}

        END {print "</table>"}

    ' |

    tidy -indent

SAVE THE SORTED  table INTO AN HTML FILE

sort mcdonalds.csv |

    awk -F"$" '

        BEGIN {print "<table border=\"1\">"}

        {print "<tr><td>"$1"</td><td>"$2"</td></tr>"}

        END {print "</table>"}

    ' |

    tidy -indent > sorted_price.html

WAIT - WHY ARE ERRORS ON THE SCREEN?

  • UNIX has a concept of stdout and stderr
  • Output goes to stdout
  • Error messages go to stderr
  • You can redirect stdout with > and stderr with 2>

REDIRECTING STDERR

sort mcdonalds.csv |

    awk -F"$" '

        BEGIN {print "<table border=\"1\">"}

        {print "<tr><td>"$1"</td><td>"$2"</td></tr>"}

        END {print "</table>"}

    ' |

    tidy -indent > sorted_price.html 2> errors.txt

IGNORING STDERR

sort mcdonalds.csv |

    awk -F"$" '

        BEGIN {print "<table border=\"1\">"}

        {print "<tr><td>"$1"</td><td>"$2"</td></tr>"}

        END {print "</table>"}

    ' |

    tidy -indent > sorted_price.html 2> /dev/null

doing mindless stuff on a whole bunch of files

LIST ALL HTML FILES

find . -name "*.html"

PUT THIS LIST IN A VARIABLE

X=`find . -name "*.html"`

sort THe list using Sort


sort mcdonalds_prices.csv

sort by second column numerically


sort -t$ -n -k2 mcdonalds_prices.csv

save the output to a file using redirection


sort -t$ -n -k2 mcdonalds_prices.csv > sorted_prices.csv

COMBINE FUNCTIONALITY USING A PIPE


grep -i burger mcdonalds_prices.csv | awk -F"$" '{print $1}'

COMBINE FUNCTIONALITY USING A PIPE


grep -i burger mcdonalds_prices.csv | awk -F"$" '{print $1}' | sort

Made with Slides.com