Introduction to GNU Text Manipulation Tools
Consoles/Terminals/Shells
Consoles are system level objects that represent a port for interaction
Consoles/Terminals/Shells
Shells are programs that process commands and return output
Consoles/Terminals/Shells
Terminals are wrapper programs that run a shell
tty0
tty1
tty2
tty3
Display Server
(X11,Quartz)
plain terminal
Window
Manager
Terminal Window
Consoles
Command Line Basics
# create an empty file
touch newfile.txt
# Create a directory
mkdir ~/somedir
# change directory
cd ~/somedir
# even better - maintains a directory stack
pushd ~/somedir
mkdir ./files
pushd ./files
popd #back to ~/somedir
# view directory contents
ls ~/somedir
# use git
git add *.js
git status
git commit -m "This commit does blank"
# view the raw html of a website
curl ui.bitcake.us
#edit a file
vi somefile.txt
nano somefile.txt
Files & Directories
# create an empty file
touch newfile.txt
# write the word Hello to a file
printf "Hello" > newfile.txt
# display the contents of the file
# note: dog is better than cat
cat newfile.txt
# append the word World to the file
printf "World" >> newfile.txt
#rename the file
mv newfile.txt hello.txt
# Create a directory
mkdir ~/somedir
# Create a hierachy of folders using brace expansion
mkdir -p ~/project/src/{assets,templates,styles}
# change directory
cd ~/project/src
# even better - maintains a directory stack
pushd ~/project
pushd ~/src/assets
printf "My Project" >> readme.md
popd #back to ~/projects
Scripting and Functions
#!/bin/bash
# multiple commands can be chained together on single line
ls /dev; lsblk; lvs;
# the shell can be "scripted"
# essentially you can write programs made up of shell commands
create-project() {
[[ $# -gt 2 ]] || return 0
mkdir ${2}
}
Grep
finds text matching a pattern
Regular Expressions
Using Grep
# Display all the links in a webpage
curl www.google.com 2>&1 | grep -o -E 'href="([^"#]+)"'
# Results
href="/search?"
href="/images/branding/product/ico/googleg_lodp.ico"
href="http://www.google.com/imghp?hl=en&tab=wi"
href="http://maps.google.com/maps?hl=en&tab=wl"
href="https://play.google.com/?hl=en&tab=w8"
href="http://www.youtube.com/?tab=w1"
href="http://news.google.com/nwshp?hl=en&tab=wn"
href="https://mail.google.com/mail/?tab=wm"
href="https://drive.google.com/?tab=wo"
href="https://www.google.com/intl/en/options/"
href="http://www.google.com/history/optout?hl=en"
href="/preferences?hl=en"
href="https://accounts.google.com/ServiceLogin?hl=en&passive=true&continue=http://www.google.com/"
href="/advanced_search?hl=en&authuser=0"
href="/language_tools?hl=en&authuser=0"
href="/intl/en/ads/"
href="/services/"
href="/intl/en/about.html"
href="/intl/en/policies/privacy/"
href="/intl/en/policies/terms/"
Grep Variants
egrep, zgrep
sed
stream editor
sed
examples
# replace 'Nick' with 'John'in report.txt
sed 's/Nick/John/g' report.txt > report_new.txt
# number non-empty lines
sed '/./=' mystuff.txt | sed 'N; s/\n/ /'
# print lines matching a pattern
sed -n '/erors/p' example.txt
# print lines 12 through 18 of a file
sed -n 12,18p file.txt
# delete all lines matching a pattern
sed '/erors/d' example
awk
an editor and reporting tool
Named after it's creators A. Aho, B. W. Kernighan and P. Weinberger
Futher Reading
http://superuser.com/questions/144666/what-is-the-difference-between-shell-console-and-terminal
me - lodolabs[at]gmail[dot]com
http://wiki.bash-hackers.org/doku.php
http://www.ibm.com/developerworks/library/l-sed1/
https://www.ibm.com/developerworks/library/l-awk1/
Introduction to GNU Text Manipulation Tools
By Matt Sprague
Introduction to GNU Text Manipulation Tools
- 1,356