Introduction to GNU Text Manipulation Tools

Consoles/Terminals/Shells

Consoles are system level objects that represent a port for interaction

Consoles/Terminals/Shells

Shells are programs that process commands and return output

Consoles/Terminals/Shells

Terminals are wrapper programs that run a shell

tty0

tty1

tty2

tty3

Display Server

(X11,Quartz)

plain terminal

Window

Manager

Terminal Window

Consoles

Command Line Basics

# create an empty file
touch newfile.txt

# Create a directory
mkdir ~/somedir

# change directory
cd ~/somedir

# even better - maintains a directory stack
pushd ~/somedir
mkdir ./files
pushd ./files
popd #back to ~/somedir

# view directory contents
ls ~/somedir

# use git
git add *.js
git status
git commit -m "This commit does blank"

# view the raw html of a website
curl ui.bitcake.us

#edit a file
vi somefile.txt
nano somefile.txt

Files & Directories

# create an empty file
touch newfile.txt

# write the word Hello to a file
printf "Hello" > newfile.txt

# display the contents of the file
# note: dog is better than cat
cat newfile.txt 

# append the word World to the file
printf "World" >> newfile.txt

#rename the file
mv newfile.txt hello.txt

# Create a directory
mkdir ~/somedir

# Create a hierachy of folders using brace expansion
mkdir -p ~/project/src/{assets,templates,styles}

# change directory
cd ~/project/src

# even better - maintains a directory stack
pushd ~/project
pushd ~/src/assets
printf "My Project" >> readme.md
popd #back to ~/projects

Scripting and Functions

#!/bin/bash

# multiple commands can be chained together on single line
ls /dev; lsblk; lvs;


# the shell can be "scripted"
# essentially you can write programs made up of shell commands

create-project() {
    [[ $# -gt 2 ]] || return 0
    
    mkdir ${2}

}

Grep

finds text matching a pattern

Regular Expressions

Using Grep

# Display all the links in a webpage
curl www.google.com 2>&1 | grep -o -E 'href="([^"#]+)"'

# Results
href="/search?"
href="/images/branding/product/ico/googleg_lodp.ico"
href="http://www.google.com/imghp?hl=en&tab=wi"
href="http://maps.google.com/maps?hl=en&tab=wl"
href="https://play.google.com/?hl=en&tab=w8"
href="http://www.youtube.com/?tab=w1"
href="http://news.google.com/nwshp?hl=en&tab=wn"
href="https://mail.google.com/mail/?tab=wm"
href="https://drive.google.com/?tab=wo"
href="https://www.google.com/intl/en/options/"
href="http://www.google.com/history/optout?hl=en"
href="/preferences?hl=en"
href="https://accounts.google.com/ServiceLogin?hl=en&passive=true&continue=http://www.google.com/"
href="/advanced_search?hl=en&authuser=0"
href="/language_tools?hl=en&authuser=0"
href="/intl/en/ads/"
href="/services/"
href="/intl/en/about.html"
href="/intl/en/policies/privacy/"
href="/intl/en/policies/terms/"

Grep Variants

egrep, zgrep

sed

stream editor

sed

examples

# replace 'Nick' with 'John'in report.txt
sed 's/Nick/John/g' report.txt > report_new.txt

# number non-empty lines
sed '/./=' mystuff.txt | sed 'N; s/\n/ /'

# print lines matching a pattern
sed -n '/erors/p' example.txt

# print lines 12 through 18 of a file
sed -n 12,18p file.txt

# delete all lines matching a pattern
sed '/erors/d' example

awk

an editor and reporting tool

Named after it's creators A. Aho, B. W. Kernighan and P. Weinberger

Futher Reading

http://superuser.com/questions/144666/what-is-the-difference-between-shell-console-and-terminal

me - lodolabs[at]gmail[dot]com

http://wiki.bash-hackers.org/doku.php

http://www.ibm.com/developerworks/library/l-sed1/

https://www.ibm.com/developerworks/library/l-awk1/