Module 5-1
Technical Environment & Scripting

ATTENTION PLEASE
When you see slides with a white background:
- listen to your instructor
- takes note if you want
- ask questions ;)
🚀 HANDS-ON
When you see slides with a colored background:
- do the exercices
- talk to your instructor
- ask questions ;)
AGENDA
FEW DAYS WITH
- Command line
- Shell expansion
- Pipes and commands
- vi
- Scripting
- Local user management
- File security
Back to basics
Computer ?

What's inside ?

Processor ?


Computer ?
- Personnal Computer
- Workstation, Laptop
- Mac
- ...
- Terminal
- Service
Server ?
Client-Server ?

Operating System
-
Human / Machine interface:
- Textual
- Graphical
-
Management role:
- Resources (files, memory, devices)
- Users data & applications
- Conflicts btw users (multi-user architecture)
- Ex: Windows, Linux, UNIX, MacOS, AS400
- Commande language: shell/DOS, powershell ...
Back to basics #2
Computer Networks
Networking
Computer Network
A collection of computing devices that are connected in various ways in order to communicate and share resources.
Usually, the connection between computer in a network are made using physical wires or cables.
However, some connections are wireless, using radio waves or infrared signals.
Networking
- The generic term node or host refers to any device on a network.
- Data transfer rate - The speed with which data is moved from one place on a network to another.
- Data transfer rate is a key issue in computer networks.
Networking
- Computer networks have opened up an entire frontier in the world of computing called the client/server model

Networking
- File server - A computer that stores and manages files for multiple users on a network.
- Web server - A computer dedicated to responding to requests (from the browser client) for web pages.
Types of Networks
- Local-area network (LAN) - A network that connects a relatively small number of machines in a relatively close geographical area.
Types of Networks
- Various configurations, called topologies, have been used to administer LANs
- Ring topology - A configuration that connects all nodes in a closed loop on which messages travel in one direction.
- Star topology - A configuration that centers around one node to which all others are connected and through which all messages are sent
- Bus topology - All node are connected to a single communication line that carries messages in both directions
Types of Networks

Types of Networks
- Wide-area network (WAN) - A network that connects two or more local-area networks over a potentially large geographic distance.
Often one particular node on a LAN is set up to server as a gateway to handle all communication going between that LAN and other networks.
Communication between networks is called internetworking. The Internet, as we know it today, is essentially the ultimate wide-are network, spanning the entire globe.
Types of Networks
- Metropolitan-area network (MAN) - The communication infrastructures that have been developed in and around large cities.
So, who owns the Internet?
Well, nobody does. No single person or company owns the Internet or even controls it entirely. As a wide-are network, it is made up of many smaller networks. These smaller networks are often owned and managed by a person or organization.
The Internet, then, is really defined by how connections can be made between these networks.
Internet Connections
- Internet backbone - A set of high-speed networks that carry Internet traffic. These network are provided by companies such as AT&T, GTE, IBM, ...
- Internet service provider (ISP) - A company that provides other companies or individuals with access to the Internet.
Internet Connections
-
There are various technologies available that you can use to connect a home computer to the Internet :
- A phone modem - converts computer data into an analog audio signal for transfer over a telephone line, and then a model at the destination converts it back again into data.
- A digital subscriber line (DSL) uses regular copper phone lines to transfer digital data to and from the phone company's central office
- A cable model uses the same line that you cable TV signals come in on to transfer the data back and forth.
Internet Connections
-
Broadband - A connection in which transfer speeds are faster than 128 bits per second
- DSL connections and cable modems are broadband connections
- The speed for downloads (getting data from the Internet to your home computer) may not be the same as uploads (sending data from your home computer to the Internet).
Packet Switching
- To improve the efficiency of transferring information over a shared communication line, messages are divided into fixed-sized, numbered packets.
- Network devices called routers are used to direct packets between networks.

Open Systems
- Proprietary system - A system that uses technologies kept private by a particular commercial vendor.
- Interoperability - The ability of software and hardware on multiple machines and from multiple commercial vendors to communicate.
- Open systems - Systems based on a common model of network architecture and a suite of protocols used in its implementation.
Network Protocols
- Network protocols are layered such that each one relies on the protocols that underlie it
- Sometimes referred to as a protocol stack

TCP/IP
- TCP stands for Transmission Control Protocol
- TCP software breaks messages into packets, hand them off to the IP software for delivery, and the orders and reassembles the packets at their destination.
- IP stands for Internet Protocol
- IP software deals with the routing of packets through the maze of interconnected networks to their final destination.
TCP/IP (cont.)
- UDP stands for User Datagram Protocol
- It is an alternative to TCP
- The main difference is that TCP is highly reliable, at the cost of decreased performance, while UDP is less reliable, but generally faster.
High-Level Protocols
- Other protocols build on the foundation established by the TCP/IP protocol suite
- Simple Mail Transfer Protocol (SMTP)
- File Transfer Protocol (FTP)
- Telnet
- Hyper Text Transfer Protocol (HTTP)
MIME Types

Firewalls
- Firewall - A machine and its software that serve as a special gateway to a network, protecting it from inappropriate access
- Filters the network traffic that comes in, checking the validity of the messages as much as possible and perhaps denying some messages altogether
- Enforces an organization's access control policy
Network Addresses
-
Hostname - A unique identification that specifies a particular computer on the Internet
- For example :
- ipme07.northeurope.cloudapp.azure.com
- For example :
Network Addresses
- Network software translates a hostname into its corresponding IP address
- For example
-
ipme07.northeurope.cloudapp.azure.com
==
40.112.67.50
-
- For example
Network Addresses
- An IP address can be split into :
- Network address, which specifies a specific network
- Host number, which specifies a particular machine in that network

Domain Name System
- A hostname consists of the computer name followed by the domain name
- A domain name is separated into two or more sections that specify the organization, and possibly a subset of an organization, of which the computer is a part
- Two organizations can have a computer named the same thing because the domain name makes it clear which one is being referred to.
Domain Name System
- The very last section of the domain is called its top-level domain (TLD) name

Domain Name System
- Organizations based in countries other than the United States use a top-level domain that corresponds to their two-letter country codes

Domain Name System
- The domain name system (DNS) is chiefly used to translate hostnames into numeric IP addresses
- DNS is an example of distributed database
- If that server can resolve the hostname, it does so
- If not, that server asks another domain name server
History
Birth of free software
1983 - Richard Stallman, GNU project and free software concept
==> Beginning of the development off gcc, gdb, glibc and important other tools
1991 - Linux Torvalds, Linux kernel project, an Unix-like operating system kernel
==> Together with GNU software and many other open-source components: a completely free operating system : GNU/Linux
1995 - Linux on server systems
2000 - Linux on embedded systems
2008 - Linux on mobile devices
2010 - Linux on phones

Free software ?
A program is considered free when its license offers to all its users the following 4 freedoms :
- Freedom to run the software for any purpose
- Freedom to study the software and to change it
- Freedom to redistribute copies
- Freedom to distribute copies of modifies versions
These freedoms are granted for both commercial and non-commercial use.
They imply the availability of source code, software can be modified and distributed to customers.
Linux philosophy
There are nine major tenets to the Linux philosophy:
- Small is beautiful
- Each program does one thing well
- Build a prototype as soon as possible
- Choose portability over efficiency
- Store data in flat text files
- Use software leverage to your advantage
- Use shell scripts to increase leverage and portability
- Avoid captive user interface
- Make every program a filter
Linux distributions
Distribution = Kernel + Application + Branding

Linux
- Powerful
- Runs on multiple hardware platform
- Users like its speed and stability
- No requirement for the latest hardware
- It's "free"
- Licensed under GPL
- Vendors are distributors who packages Linux
Linux
Multi-user
A multi-user OS allows for multiple users to use the same computer at the same time and/or different time
Multi-processing
An operating system capable of supporting and utilizing more than one computer processor
Multi-tasking
An operating system capable of allowing multiple software processes to run at the same time
Multi-threading
OS that allows different parts of a software program to run concurrently
Linux

COMMAND LINE
COMMAND LINE
MAN PAGES
man
# man $command
$ man whois
$ man man
# man $configfile
$ man syslog.conf
# man deamon
$ man syslogd
Press q to quit
whatis
To see just the description of a manual page, use
whatis followed by a string
$ whatis route
route (8) - affiche / manipule la table de routage IP
whereis
whereis locates source/binary and manual sections for specified files.
$ whereis httpd
httpd: /usr/sbin/httpd /usr/lib64/httpd /etc/httpd /usr/share/httpd
COMMAND LINE
WORKING WITH DIRECTORIES
pwd
The tool displays your current directory.
$ pwd
/root
cd
Change your current directory.
$ pwd
/root
$ cd ..
$ pwd
/
$ cd /var/log/
$ pwd
/var/log/
$ cd /etc
$ pwd
/etc
$ cd -
$ pwd
/var/log
$ cd ~
$ pwd
/root
ls
Display directory content.
$ ls
hello.txt
$ ls -l
total 0
-rw-r--r-- 1 root root 0 7 janv. 21:41 hello.txt
$ ls -al
total 8
drwxr-xr-x 2 root root 4096 7 janv. 21:42 .
dr-xr-x---. 8 root root 4096 7 janv. 21:42 ..
-rw-r--r-- 1 root root 0 7 janv. 21:42 hello.txt
-rw-r--r-- 1 root root 0 7 janv. 21:42 .hidded_file
mkdir
Create your own directories with
mkdir.
$ mkdir a
$ cd a
$ ls -al
total 8
drwxr-xr-x 2 root root 4096 7 janv. 21:44 .
drwxr-xr-x 4 root root 4096 7 janv. 21:44 ..
$ mkdir b
$ mkdir c
$ ls -l
total 16
drwxr-xr-x 4 root root 4096 7 janv. 21:44 .
drwxr-xr-x 4 root root 4096 7 janv. 21:44 ..
drwxr-xr-x 2 root root 4096 7 janv. 21:44 b
drwxr-xr-x 2 root root 4096 7 janv. 21:44 c
$ mkdir -p d/e/f
$ cd d
$ ls -l
total 4
drwxr-xr-x 3 root root 4096 7 janv. 21:45 e
rmdir
Use
rmdir to remove a directory.
$ ls -l
total 12
drwxr-xr-x 2 root root 4096 7 janv. 21:44 b
drwxr-xr-x 2 root root 4096 7 janv. 21:44 c
drwxr-xr-x 3 root root 4096 7 janv. 21:45 d
$ rmdir c/
$ rmdir -p d/e/f
🚀 HANDS-ON
- Display your current directory
- Change to /tmp
- List the content of /tmp directory
- Create the useless folder useless/
- Remove the directory useless/
COMMAND LINE
WORKING WITH FILES
FACTS
- All files are case sensitive
- Everything is a file !
$ ls
winter.txt Winter.txt
file
file utility determines the file type.
$ file winter.txt
winter.txt: ASCII text
$ file winter.tar.gz
winter.tar.gz: gzip compressed data, from Unix, last modified: Sun Jan 7 21:53:10 2018
touch
One easy way to create an empty file is with
touch.
$ ls -l
total 0
$ touch a.txt
$ touch b.txt
$ ls -l
total 0
-rw-r--r-- 1 root root 0 7 janv. 21:57 a.txt
-rw-r--r-- 1 root root 0 7 janv. 21:57 b.txt
rm
Use
rm to remove a file.
$ ls
a.txt b.txt c.txt
$ rm a.txt
$ ls
b.txt c.txt
$ rm -i b.txt
rm : supprimer fichier vide « b.txt » ? y
$ ls
c.txt
$ rm -i c.txt
rm : supprimer fichier vide « b.txt » ? n
$ ls
c.txt
$ mkdir folder1/ folder2/
$ rm -r folder1/
rm : supprimer répertoire « folder1 » ? y
$ rm -rf folder2/
cp
Use
cp to copy a file/directory.
$ ls
a.txt b.txt c.txt
$ cp a.txt a_old.txt
$ ls
a_old.txt a.txt b.txt c.txt
$ mkdir folder1/
$ cp -r folder1/ folder1_backup/
mv
Use
mv to rename a file/directory.
$ ls
a.txt b.txt c.txt
$ mv a.txt d.txt
$ ls
b.txt c.txt d.txt
$ mkdir folder1
$ mv folder1/ folder2/
$ ls
b.txt c.txt d.txt folder2
🚀 HANDS-ON
- List files of
/bindirectory - Display the file type of
/bin/cat,/etc/passwd,/etc/shadow - Create a directory
~/petsand enter it. - Create files
wolf.txtandcat.txt - Rename
cat.txttounicorn.txt - Copy
unicorn.txtindragon.txt - Remove all
*.txtfiles in~/petsfolder
COMMAND LINE
WORKING WITH FILES CONTENT
head
Use
head to display the first ten lines of a file.
$ head /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
$ head -2 /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
tail
Use tail to display the last ten lines of a file.
$ tail /etc/services
3gpp-cbsp 48049/tcp # 3GPP Cell Broadcast Service Protocol
isnetserv 48128/tcp # Image Systems Network Services
isnetserv 48128/udp # Image Systems Network Services
blp5 48129/tcp # Bloomberg locator
blp5 48129/udp # Bloomberg locator
com-bardac-dw 48556/tcp # com-bardac-dw
com-bardac-dw 48556/udp # com-bardac-dw
iqobject 48619/tcp # iqobject
iqobject 48619/udp # iqobject
matahari 49000/tcp # Matahari Broker
cat
Use cat to display a file on the screen.
$ cat /etc/resolv.conf
# Generated by NetworkManager
search openstacklocal ovh.net
nameserver 213.186.33.99
# Use cat to concatenate
$ echo one > part1
$ echo two > part2
$ echo three > part3
$ cat part1
one
$ cat part2
two
$ cat part1 part2 part3 > all
$ cat all
one
two
three
tac
Use tac to display a file (backwards) on the screen.
$ cat count
one
two
three
four
$ tac count
four
three
two
one
🚀 HANDS-ON
- Display the first 12 lines of
/etc/services. - Display the last line of
/etc/passwd. - Use more to display
/etc/services. - Use ls to find the biggest file in
/etc. - Display
/var/log/files based on last modified time (in reverse order)
COMMAND LINE
THE LINUX FILE TREE
THE ROOT DIRECTORY /
All Linux systems have a directory structure that starts at the root directory. The root directory is represented by a forward slash, like this: /.
$ cd /
$ ls
bin boot dev etc home lib lib64 lost+found media mnt opt
proc root run sbin srv sys tmp usr var
BINARY DIRECTORIES
Binaries are file that contains compiled code (or machine code). Binaries can be executed on the computer. Sometimes binaries are called executables.
# The /bin directory contains binaries for use by all users
$ ls /bin
[ gpg-error nl-addr-list setpriv
a2p gpgparsemail nl-class-add setsid
ab gpgsplit nl-class-delete setterm
...
# /sbin contains binaries to configure the operating system
accessdb e2image intel-microcode2ucode nl-qdisc-list
addgnupghome e2label ip nologin
...
# Binaries found in /bin and /sbin ofter use shared libraries located in /lib
audit libcrack.so.2 libnss_files.so.2
binfmt.d libcrack.so.2.9.0 libnss_hesiod-2.17.so
debug libcrypt-2.17.so libnss_hesiod.so.2
/opt
The purpose of /opt is to store optional software. In many cases this is software from outside the distribution repository. You may find an empty /opt directory on many systems.
CONFIGURATION DIRECTORIES
The /boot directory contains all files needed to boot the computer. These files don't change very often.
/etc: contains (almost) all of the machine-specific configuration files.
$ ls /etc/*.conf
/etc/asound.conf /etc/GeoIP.conf /etc/libaudit.conf /etc/mke2fs.conf
/etc/chrony.conf /etc/host.conf /etc/libuser.conf /etc/nsswitch.conf
/etc/dnsmasq.conf /etc/kdump.conf /etc/locale.conf /etc/resolv.conf
/etc/dracut.conf /etc/krb5.conf /etc/logrotate.conf /etc/rsyncd.conf
/etc/e2fsck.conf /etc/ld.so.conf /etc/man_db.conf /etc/rsyslog.conf
DATA DIRECTORIES
# /home : store personal or project data
$ ls /home
pierre paul jacques david alexis
# /root : default location for personal data and profile of the root user
# /srv : contains site-specific data which is served by this system.
# /media : directory serves as mount point for removable media devices
$ ls /media
cdrom cdrom0 usbdisk
# /mnt : used for temporary mount points
# /tmp : Applications and users should use /tmp to store temporary data when needed.
IN-MEMORY DIRECTORIES
# /dev : populated with files as the kernel recognize hardware.
Not actually located on hard disk.
# /proc : special directory, not taking up disk space. View of
the kernel & what kernel manages.
# /sys : contains kernel information about hardware.
/usr UNIX SYSTEM RESOURCES
The /usr hierarchy should contain shareable, read only data.
# /usr/bin: contains a lot of commands
$ ls /usr/bin | wc -l
1395
# /usr/include: contains general use include files for C.
aalib.h expat_config.h features.h
...
# /usr/lib: contains libraries that are not executed directly by users of scripts.
# /usr/local: can be used by administrator to install software locally
# /usr/src: directory recommended location for kernel source file
/var VARIABLE DATA
Files that are unpredictable in size, such as log, cache shoud be located in /var.
# /var/log: directory serves as a central point to contain all log files
$ ls /var/log
anaconda btmp dmesg maillog-20171224
audit btmp-20180101 dmesg.old maillog-20171231
boot.log chrony firewalld messages
...
# /var/log/messages: contains information on what just happened to the system
# /var/cache: contains cache data for several application
# /var/spool: contains spool directories for mail, cron ...
# /var/lib: contains application states
# /var: also contains Process ID files in /var/run and temporary files that survive
# after a reboot
🚀 HANDS-ON
- Does the file
/bin/catexist ? What about/bin/ddand/bin/echo? What is the type of these files ? - Use cat to display
/etc/hostsand/etc/resolv.conf. What is your idea about the purpose of these files ? - Are there any files in
/etc/skel? Check also hidden files - Display
/proc/cpuinfo. On what architecture is your Linux running ? - Is
/var/loga file or a directory ? What about/var/spool?
SHELL EXPANSION
SHELL EXPANSION
COMMANDS AND ARGUMENTS
Commands and arguments
The command line interface or shell used on most Linux systems is called bash, which stands for Bourne again shell.
The bash shell incorporates features from sh (the original Bourne shell), csh (the C shell) and ksh (the Korn shell).
echo
Display each argument it receives from the shell
$ echo Hello World
Hello World
$ echo Hello World
Hello World
$ echo Hello World
Hello World
$ echo Hello World
Hello World
$ echo 'A line with single quotes'
A line with single quotes
$ echo "A line with double quotes"
A line with double quotes
$ echo -e "A line with \na newline"
A line with
a newline
$ echo -e 'A line with \ta tab'
A line with a tab
type
To find out whether a command given to the shell will be executed as an external command or as a builtin command.
$ type cd
cd is a shell builtin
$ type cat
cat is /bin/cat
$ type ls
ls is aliased to `ls --color=auto`
$ type echo
echo is a shell builtin
echo is /bin/echo
$ /bin/echo Running the external echo command...
Running the external echo command...
which
To search for binaries in the $PATH environment variable.
$ which cp ls mkdir pwd
/bin/cp
/bin/ls
/bin/mkdir
/bin/pwd
alias
Alias are often used to create an easier to remember name for an existing command.
$ cat count.txt
one
two
three
$ alias dog=tac
$ dog count.txt
three
two
one
$ alias ll='ls -lh --color=auto'
$ alias c='clear'
$ alias c ll
alias c='clear'
alias ll='ls -lh --color=auto'
$ unalias c
🚀 HANDS-ON
- Is
taca shell builtin command ? - Is there an existing alias for
rm? - Execute:
alias rm='rm -i'. Test you alias with a test file. Does this work as expected ? - Read the man page of
rm, make sure you understand the-ioption ofrm. Create and remove a file to test the-ioption. - List all current aliases.
SHELL EXPANSION
CONTROL OPERATORS
; semicolon
Put two or more commands on the same line separated by ;
$ echo Hello
Hello
$ echo World
World
$ echo Hello ; echo World
Hello
World
& ampersand
When a line ends with an ampersand &, the shell will not wait for the command to finish.
$ sleep 20 &
[1] 4512
$
...wait 20 seconds
$
[1]+ Done sleep 20
&& double ampersand
Shell interprets && as a logical AND
$ echo first && echo second
first
second
$ zecho first && echo second
-bash: zecho: command not found
$ cd /tmp && ls
file1 file3 file5
file2 file4 file6
|| double vertical bar
|| represents a logical OR.
$ echo first || echo second ; echo third
first
third
$ zecho first || echo second ; echo third
second
third
combining && and ||
$ rm file1 && echo It worked! || echo It failed!
It worked!
$ rm file1 && echo It worked! || echo It failed!
rm: cannot remove `file1`: No such file or directory
It failed!
$
MISCELLANEOUS
# Pound sign (#) is ignored by the shell
$ mkdir test # we create a directory
# \ escaping special characters
$ echo hello \; world
hello ; world
$ echo hello \ \ \ world
hello world
$ echo escaping \\\ \#\ \&\ \"\ \'
escaping \ # & " '
$ echo escaping \\\?\*\"\'
escaping \?*"'
# End of line backslash
$ echo This command line \
> is split in three \
> parts
This command line is split in three parts
$
🚀 HANDS-ON
- When you type
passwd, which file is executed ? What kind of file is that ? - Execute
lsaftercd /etc, but only ifcd /etcdid not error. - Execute
cd /etcaftercd etc, but only ifcd etcfails. - Echo it worked when
touch test42works, and echo it failed when thetouchfailed. All on one command line. Test it in your home directory and in/bin/. - Execute
sleep 6, what is this command doing ?
SHELL EXPANSION
SHELL VARIABLES
$ dollar sign
The shell will look for an environment variable named like the string following the dollar sign and replace it with the value.
$ echo This is the $SHELL shell
This is the /bin/bash shell
$ echo This is $SHELL on computer $HOSTNAME
This is /bin/bash on computer RHELv4u3.localdomain
$ echo The userid of $USER is $UID
The userid of paul is 500
$ echo My homedir is $HOME
My homedir is /home/paul
creating variables
$ MyVar=555
$ echo $MyVAR
555
# Case sensitive !
$ echo Hello $USER
paul
$ echo Hello $user
Hello
# quotes
$ echo "$MyVar"
555
$ echo '$MyVar'
$MyVar
set / unset
$ MyVar=8472
$ echo $MyVar
8472
$ unset $MyVar
$ echo $MyVar
$
Use the
set command to display a list of environment variables.
Use the
unset command to remove a variable from your shell environment.
$PATH
$ echo $PATH
/usr/kerberos/bin:/usr/local/bin:/bin/:/user/bin:
# Difference btw su and su -
[paul@RHEL3 ~]$ su
Password:
[root@RHEL3 ~]$ echo $PATH
/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin
[root@RHEL3 ~]$ exit
[paul@RHEL3 ~]$ su -
Password:
[root@RHEL3 ~]$ echo $PATH
/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:
The $PATH variable is determines where the shell is looking for commands to execute (unless the command is builtin or aliased). This variable contains a list of directories, separated by colons.
env
$ bash -c 'echo $SHELL $HOME $USER'
/bin/bash /home/paul paul
$ env -i bash -c 'echo $SHELL $HOME $USER'
/bin/bash
env commands without options will display a list of exported variables.
set lists all variables, including those not exported to child shells.
env can also be used to start a clean shell.
export
$ var3=three
$ var4=four
$ export var4
$ echo $var3 $var4
three four
$ bash
$ echo $var3 $var4
four
# But will not export to the parent shell
$ export var5=five
$ echo $var3 $var4 $var5
four five
$ exit
exit
$ echo $var3 $var4 $var5
three four
You can export shell variables to other shells with the export command.
🚀 HANDS-ON
- Use echo to display Hello followed by your username.
- Create a variable
answerwith a value of 42. - Copy the value of
$LANGto$MyLANG. - List all current shell variables.
- List all exported shell variables.
- Do the
envandsetcommands displays your variable ? - Destroy your
answervariable. - Create two variables, and
exportone of them. - Display the exported variable in an interactive child shell.
SHELL EXPANSION
SHELL EMBEDDING AND OPTIONS
shell embedding
$ echo $var1
$ echo $(var1=5; echo $var1)
5
$ echo $var1
$
backticks
$ echo `cd /etc; ls -d * | grep pass`
passwd passwd- passwd.OLD
$
backticks or single quotes
$ echo `var1=5;echo $var1`
5
$ echo 'var1=5;echo $var1'
var1=5;echo $var1
$
shell options
$ echo $var123
$ set -u
$ echo $var123
bash-: var123: unbound variable
$ set +u
$ echo $var123
$
🚀 HANDS-ON
- Find the list of shell options in the man page of
bash. What is the difference btw
set -uandset -onounset ? - Activate
nounsetin your shell. Test that it shows an error message when using non-existing variables. - Desactive nounset.
- Execute
cd /varandlsin an embedded shell - Create the variable embvar in an embedded shell and echo it. Does the variable exists in your current shell now ?
- Explain what "
set -x" does. Can this be useful ?
SHELL EXPANSION
SHELL HISTORY
repeating the XX command(s)
$ echo this will be repeated > file42.txt
$ !!
this will be repeated > file42.txt
$ touch file42
$ cat file42
$ !to
touch file42
$
To repeat the last command in bash, type !!.
history
$ history 10
38 mkdir test
39 cd test
40 touch file1
41 echo hello > file2
42 echo It is very cold day > winter.txt
43 ls
44 ls -l
45 cp winter.txt summer.txt
46 ls -l
47 history 10
To see older commands, use
history.
$HIST...
$ echo $HISTSIZE
500
# Can be changed :
$ HISTSIZE=15000
$ echo $HISTSIZE
15000
$ echo $HISTFILE
/home/paul/.bash_history
$ echo $HISTFILESIZE
15000
$HISTSIZE: number of commands that will be remembered
$HISTFILE: file that contains your history.
$HISTFILESIZE: number of commands kept in your history file
🚀 HANDS-ON
- Issue the command echo The answer to the meaning of life, the universe and everything is 42.
- Repeat the previous command using two characters (there are two solutions!).
- Display at least 5 commands you typed.
- How many commands can be kept in memory for your current shell session ?
SHELL EXPANSION
FILE GLOBBING
* asterisk
$ ls
file1 file2 file3 File4 File55 FileA fileab Fileab FileAB fileabc
$ ls File*
File4 File55 FileA Fileab FileAB
$ ls file*
file1 file2 file3 fileab fileabc
$ ls *ile55
File55
$ ls F*ile55
File55
$ ls F*55
File55
$
? question mark
$ ls
file1 file2 file3 File4 File55 FileA fileab Fileab FileAB fileabc
$ ls File?
File4 FileA
$ ls Fil?4
File4
$ ls Fil??
File4 FileA
$ ls File??
File55 Fileab FileAB
$
[] square brackets
$ ls
file1 file2 file3 File4 File55 FileA fileab Fileab FileAB fileabc
$ ls File[5A]
FileA
$ ls File[A5]
FileA
$ ls File[A5][5b]
File55
$ ls File[a5][5b]
File55 Fileab
$ ls File[a5][5b][abcdefghijklm]
ls: File[a5][5b][abcdefghijklm]: No such file or directory
$ ls file[a5][5b][abcdefghijklm]
fileabc
$
[ is interpreted by the shell as a sign to generate filenames, matching any of the characters between [ and the first subsequent ].
a-z and 0-9 ranges
$ ls
file1 file3 File55 fileab FileAB fileabc
file2 File4 FileA Fileab fileab2
$ ls file[a-z]*
fileab fileab2 fileabc
$ ls file[0-9]
file1 file2 file3
$ ls file[a-z][a-z][0-9]*
fileab2
$
The bash shell will also understand ranges of characters between brackets.
preventing file globbing
$ mkdir test42
$ cd test42
$ echo *
*
$ touch file42 file33
$ echo *
file33 file42
🚀 HANDS-ON
- Create a test directory and enter in it.
- Create the following files : file1, file10, file11, file2, File2, File3, file33, fileAB, filea, fileA, fileAAA, file(, file 2.
- List (with ls) all files starting with :
- file
- File
- file and ending in a number
- File and having a digit as fifth character
- List all files that have exactly five characters.
- List all files that do not start with the letter F.
PIPES AND COMMANDS
PIPES AND COMMANDS
I/O redirection
stdin, stdout and stderr


output redirection
# > stdout
$ echo It is cold today! > winter.txt
$ cat winter.txt
It is cold today !
# Output file is erased !
$ zcho It is cold today! > winter.txt
-bash: zcho: command not found
$ cat winter.txt
$
# >> append
$ echo It is cold today! > winter.txt
$ echo Where is the summer ? >> winter.txt
$ cat winter.txt
It is cold today!
Where is the summer ?
$

error redirection
# Redirecting stderr is done with 2>
$ find / > allfiles.txt 2> /dev/null # Redirect stdout to file and stderr to /dev/null
# 2>&1 Redirect both stdout and stderr to the same file
$ find / > allfiles_and_errors.txt 2>&1
# &> construction will put both stdout and stderr in one stream
$ rm file42 &> out_and_err
$ cat ou_and_err
rm: cannot remove `file42`: No such file or directory
$ echo file42 &> out_and_err
$ cat out_and_err
file42
$

input redirection
# < stdin
$ cat < text.txt
one two
# << here document
$ cat << EOF
> Working dir "$PWD" `pwd`
> EOF
Working dir "/home/paul" /home/paul
# <<< here string
$ base64 <<< institut-de-la-pme # equivalent of echo institut-de-la-pme | base64
aW5zdGl0dXQtZGUtbGEtcG1lCg==
$ base64 -d <<< aW5zdGl0dXQtZGUtbGEtcG1lCg==
institut-de-la-pme
# Quickest way to clear a file
$ >text.txt
🚀 HANDS-ON
- Make sure you have two shells open on the same computer. Create an empty
tailing.txtfile. Then typetail -f tailing.txt. Use the second shell to append a line of text to that file. Verify that the first shell displays this line. - Create a file that contains the names of five people. Use
catand output redirection to create the file and use a here document to end the input.
PIPES AND COMMANDS
FILTERS
cat
$ tac count.txt | cat | cat | cat | cat | cat
five
four
three
two
one
$
When between two pipes, the
cat command does nothing (except putting
stdin on
stdout).
tee
$ tac count.txt | tee temp.txt | tac
one
two
three
four
five
$ cat temp.txt
five
four
three
two
one
$
Puts stdin on stdout and also into a file. Useful on long pipes to store intermediate results.
grep
$ cat tennis.txt
Amelie Mauresmo, Fra
Kim Clijsters, BEL
Justine Henin, Bel
Serena Williams, usa
Venus Williams, USA
$ cat tennis.txt | grep Williams
Serena Williams, usa
Venus Williams, USA
$ grep Bel tennis.txt
Justine Henin, Bel
$ grep -i Bel tennis.txt # insensitive way
Kim Clijsters, BEL
Justine Henin, Bel
$ grep -v Fra tennis.txt # outputs lines not matching the string
Kim Clijsters, BEL
Justine Henin, Bel
Serena Williams, usa
Venus Williams, USA
The most well-known filter!
cut
# Using colon as a delimiter, selecting fields 1 and 3
$ cut -d: -f1,3 /etc/passwd | tail -4
syslog:498
jira:34064
omsagent:497
nxautomation:496
# Using a speace as the delimiter for cut, you have to quote the space
$ cut -d" " -f1 tennis.txt
Amelie
Kim
Justine
Serena
Venus
# Display the second to the seventh character of /etc/passwd
$ cut -c2-7 /etc/passwd | tail -4
yslog:
ira:x:
msagen
xautom
Filter can select columns from files, depending on a delimiter.
tr
$ cat tennis.txt | tr 'e' 'E'
AmEliE MaurEsmo, Fra
Kim ClijstErs, BEL
JustinE HEnin, BEl
SErEna Williams, usa
VEnus Williams, USA
$ cat tennis.txt | tr 'a-z' 'A-Z'
AMELIE MAURESMO, FRA
KIM CLIJSTERS, BEL
JUSTINE HENIN, BEL
SERENA WILLIAMS, USA
VENUS WILLIAMS, USA
$ cat count.txt | tr '\n' ' '
one two three four five
Used for translating characters.
wc
$ wc tennis.txt
5 15 100 tennis.txt
$ wc -l tennis.txt
5 tennis.txt
$ wc -w tennis.txt
15 tennis.txt
$ wc -c tennis.txt
100 tennis.txt
Counting words, lines and characters.
sort
$ cat music.txt
Queen
Brel
Led Zeppelin
Abba
$ sort music.txt
Abba
Brel
Led Zeppelin
Queen
# Sorting on column
$ sort -k1 country.txt
Belgium, Brussels, 10
France, Paris, 60
Germany, Berlin, 100
Iran, Teheran, 70
Italy, Rome, 50
uniq
$ cat music.txt
Queen
Brel
Queen
Abba
$ sort music.txt
Abba
Brel
Queen
Queen
$ sort music.txt |uniq
Abba
Brel
Queen
# uniq can also count occurences
$ sort music.txt |uniq -c
1 Abba
1 Brel
2 Queen
sed
$ echo level5 | sed 's/5/42/'
level42
$ echo level5 | sed 's/level/jump/'
jump5
$ echo level5 level7 | sed 's/level/jump/'
jump5 level7
# Add g for global replacements (all occurences of the string per line)
$ echo level5 level7 | sed 's/level/jump/g'
jump5 jump7
# With d you can remove lines from a stream containing a character.
$ cat tennis.txt
Venus Williams, USA
Martina Hingis, SUI
Justine Henin, BE
$ cat tennis.txt | sed '/BE/d'
Venus Williams, USA
Martina Hingis, SUI
The stream editor sed can perform editing functions in the stream.
pipe examples
# How many users are logged on to this system ?
$ who
root tty1 Jul 25 10:50
paul pts/0 Jul 25 09:29 (laika)
Harry pts/1 Jul 25 12:26 (barry)
paul pts/2 Jul 25 12:26 (pasha)
$ who | wc -l
4
# Display a sorted list of logged on users
$ who | cut -d' ' -f1 | sort
Harry
paul
paul
root
# Display a sorted list of logged on users, but every user only once.
$ who | cut -d' ' -f1 | sort | uniq
Harry
paul
root
🚀 HANDS-ON
- Put a sorted list of all bash users in bashusers.txt
- Put a sorted list of all logged on users in onlineusers.txt
- Make a list of all filenames in
/etcthat contain the string conf in their filename. - Make a sorted like of all files in
/etcthat contain the case insensitive string conf in their filename - Look at the output of
/sbin/ifconfig. Write a line that displays only ip address and the subnet mask. - Write a line that removes all non-letters from a stream.
- Write a line that receives a text file, and outputs all words on a separate line.
PIPES AND COMMANDS
BASIC UNIX TOOLS
find
# Find all files in /etc and put the list in etcfiles.txt
$ find /etc > etcfiles.txt
# Find all files of the entire system and put the list in allfiles.txt
$ find / > allfiles.txt
# Find all files that en in .conf in the current directory (an all subdirs)
$ find . -name "*.conf"
# Find files of type file (not directory, pipe ...) that en in .conf
$ find . -type f -name "*.conf"
# Find files that are newer than file42.txt
$ find . -newer file42.txt
# Find *.odf files in /data and copy them to /backup/
$ find /data -name "*.odf" -exec cp {} /backup/ \;
Can be very useful at the start of a pipe to search for files.
Add
2>/dev/null to avoid cluttering your screen with errors.
locate
$ locate Samba
warning: locate: could not open database: /var/lib/slocate/slocate.db:...
warning: You need to run the 'updatedb' command (as root) to create th...
Please have a look at /etc/updatedb.conf to enable the daily cron job.
$ updatedb
fatal error: updatedb: You are not authorized to create a default sloc...
$ su -
Password:
$ updatedb
$
The
locate tool is very different from
find in that it uses an index to locate files.
This is faster than traversing all the directories but it also means that it's always outdated. If the index does not exist yet, then you have to create it (as root on RHEL).
date
$ date
Thu Jan 11 13:06:40 UTC 2018
$ date +'%A %d-%m-%Y'
Thursday 11-01-2018
$ date +%s
1515676031
# When will this seconds counter read two thousand milion ?
$ date -d '1970-01-01 + 2000000000 seconds'
To display date, time, time zone and more
cal
$ cal
January 2018
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31
$ cal 2 1970
February 1970
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
sleep
$ sleep 5
$
Sometimes used in scripts to wait a number of seconds.
time
$ time date
Thu Jan 11 13:12:30 UTC 2018
real 0m0.002s
user 0m0.001s
sys 0m0.000s
# The sleep 5 command takes five real seconds to execute it
$ time sleep 5
real 0m5.002s
user 0m0.001s
sys 0m0.001s
Can display how long it takes to execute a command.
gzip - gunzip - zcat - zmore
$ ls -lh
total 10M
-rw-r--r-- 1 paul users 10M Jan 11 13:25 10mb.txt
$ gzip 10mb.txt
$ ls -lh
total 12K
-rw-r--r-- 1 paul users 10K Jan 11 13:25 10mb.txt.gz
$ gunzip 10mb.txt.gz
$ ls -lh
total 10M
-rw-r--r-- 1 paul users 10M Jan 11 13:25 10mb.txt
# Files that are compressed with gzip can be viewed with zcat and zmore
Users never have enough disk space so compression comes in handy.
bzip2 - bunzip2 - bzcat - bzmore
🚀 HANDS-ON
- Explain the difference between these two commands
a)find /data -name "*.txt"
b)find /data -name *.txt - Write a find command that finds all files created after January 30th 2010.
- Count the number of
*.conffiles in /etc and all its subdirs. - Create a file called
loctest.txt. Can you find this file with locate ? Why not ? How do you make locate find this file ? - Use
findand-execto rename all .htm files to .html.
vi
Introduction to vi
The vi editor is installed on almost every Unix.
Linux will very often install vim (vi improved) which is similar.
EVERY SYSTEM ADMINISTRATOR SHOULD KNOW VI(M) BECAUSE IT IS AN EASY TOOL TO SOLVE PROBLEMS.
vi cheat sheet
SCRIPTING
SCRIPTING
SCRIPTING INTRODUCTION
hello world
$ echo echo Hello World > hello_world
$ chmod +x hello_world
$ ./hello_world
Hello World
$
she-bang
#!/bin/bash
Hello World
#! is called a she-bang.
Used because you can never be sure which shell a user is running.
comment
#!/bin/bash
#
# Hello World Script
#
echo Hello World
variables
#!/bin/bash
#
# simple variable in script
#
var1=4
echo var1 = $var1
$ echo $var1
$ ./vars
var1 = 4
$ echo $var1
$
sourcing a script
$ source ./vars
var1 = 4
$ echo $var1
4
$
Luckily, you can force a script to run in the same shell; this is called sourcing a script.
🚀 HANDS-ON
Give each script a different name, keep them for later!
- Write a script that outputs the name of a city.
- Make sure the script runs in the bash shell.
- Makes sure the script runs in the Korn shell.
- Create a script that defines two variables, and outputs their value.
- The previous script does not influence your current shell. Now run the script so that it influences your current shell.
- Is there a shorter way to
sourcethe script ? - Comment your scripts so that you know what they're doing.
SCRIPTING
SCRIPTING LOOPS
test []
$ test 10 -gt 55 ; echo $?
1
$ test 56 -gt 55 && echo true || echo false
true
$ [ 6 -gt 55 ] && echo true || echo false
false
# [ -d foo ] Does the directory foo exist ?
# [ -e bar ] Does the file bar exist ?
# [ '/etc' = $PWD ] Is the string /etc equal to the variable $PWD ?
# [ $1 != 'secret' ] Is the first parameter different from secret ?
# [ 55 -lt $bar ] Is 55 less than the value of $bar ?
# [ $foo -ge 1000 ] Is the value of $foo greater or equal to 1000 ?
# [ "abc" < $bar ] Does abc sort before the value of $bar ?
# [ -f foo ] Is foo a regular file ?
# [ -r bar ] Is bar a readable file ?
# [ foo -nt bar ] Is file foo newer than file bar ?
The
test command can test whether something is true or false.
if then else
#!/bin/bash
if [ -f isit.txt ]
then echo isit.txt exists!
else echo isit.txt not found!
fi
If a certain condition is met, then execute something, else execute something else.
$ ./choice
isit.txt not found!
$ touch isit.txt
$ ./choice
isit.txt exists!
$
if then elif
#!/bin/bash
count=42
if [ $count -eq 42 ]
then
echo "42 is correct."
elif [ $count -gt 42 ]
then
echo "Too much."
else
echo "Not enough."
fi
You can nest a new
if inside an
else with
elif.
for loop
for i in 1 2 4
do
echo $i
done
for counter in {1..20}
do
echo counting from 1 to 20, now at $counter
sleep 1
done
while loop
i=100
while [ $i -ge 0 ] ;
do
echo Counting down, from 100 to 0, now at $i;
let i--;
done
until loop
let i=100;
until [ $i -le 0 ];
do
echo Counting down, from 100 to 1, now at $i;
let i--;
done
🚀 HANDS-ON
- Write a script that uses
- A
forloop to count from 3 to 7. - A
whileloop to count from 1 to 17000. - An
untilloop to count down from 8 to 4.
- A
- Write a script that counts the number of file ending in
.txtin the current directory. - Wrap an
if statement around the script so it is also correct when there are zero files ending in
.txt.
SCRIPTING
SCRIPTING PARAMETERS
script parameters
#!/bin/bash
echo The first argument is $1
echo The second argument is $2
echo The third argument is $3
echo \$ $$ PID of the script
echo \# $# count arguments
echo \? $? last return code
echo \* $* all the arguments
$ ./pars one two three
The first argument is one
The second argument is two
The third argument is three
$ 5610 PID of the script
# 3 count arguments
? 0 last return code
* one two three all the arguments
runtime input
#!/bin/bash
echo -n Enter a number:
read number
You can ask the user for input.
sourcing a config file
$ cat myApp.conf
# The config file of myApp
# Enter the path here
myAppPath=/var/myApp
# Enter the number of quines here
quines=5
The
source can be used to source a config file.
$ cat myApp.bash
#!/bin/bash
# ...
. ./myApp.conf
echo There are $quines quines
$ ./myApp.bash
There are 5 quines
$
get script options with getopts
$ cat options.ksh
#!/bin/ksh
while getopts ":afz" option;
do
case $option in
a)
echo received -a
;;
f)
echo received -f
;;
z)
echo received -z
;;
*)
echo "invalid option -$OPTARG"
;;
esac
done
🚀 HANDS-ON
- Write a script that receives four parameters, and outputs them in reverse order.
- Write a script that receives two parameters (two filenames) and outputs whether those files exist.
- Write a script that asks for a filename. Verify existence of the file, then verify that you own the file, and whether it is writable. If not, then make it writable.
- Make a configuration file for the previous script. Put a logging switch in the config file, logging means writing detailed output of everything the script does to a log file in /tmp.
SCRIPTING
SCRIPTING PARAMETERS
eval
# This is horrible but :
$ answer=42
$ word=answer
$ eval x=\$$word ; echo $x
42
eval reads arguments as input to the shell. Allows using the value of a variable as a variable.
(( ))
$ (( 42 > 33 )) && echo true || echo false
true
$ (( 42 > 1201 )) && echo true || echo false
false
$ var42=42
$ (( 42 == var42 )) && echo true || echo false
false
$ (( 42 == $var42 )) && echo true || echo false
true
$ var42=33
$ (( 42 == $var42 )) && echo true || echo false
false
The
(( )) allows for evaluation of numerical expressions.
let
$ let x="3 + 4" ; echo $x
7
$ let x="10 + 100/10" ; echo $x
20
$ let x="10-2+100/10" ; echo $x
18
$ let x="10*2+100/10" ; echo $x
30
#
Care! let is not variable value assignation :
$ dec=15 ; oct=017 ; hex=0x0f
$ echo $dec $oct $hex
15 017 0x0f
$ let dec=15 ; let oct=017 ; let hex=0x0f
$ echo $dec $oct $hex
15 15 15
The
let built-in shell function instructs the shell to perform an evaluation of aritmetic expressions.
case
$ ./help
What animal did you see ? lion
You better start running fast!
$ cat help
#!/bin/bash
#...
echo -n "What animal did you see ? "
read animal
case $animal in
"lion" | "tiger")
echo "You better start running fast!"
;;
"cat")
echo "Let that mouse go..."
;;
"dog")
echo "Don't worry, give it a cookie."
;;
*)
echo "You discovered an unknown animal, name it!"
;;
esac
$
shell functions
$ cat funcs.ksh
#!/bin/ksh
function greetings {
echo Hello World!
echo and hello to $USER to!
}
echo We will now call a function
greetings
echo The end
$ ./funcs.ksh
We will now call a function
Hello World!
and hello to paul to!
The end
🚀 HANDS-ON
- Write a script that asks for two numbers, and outputs the sum and product.
- Improve the previous script to test that the numbers are between 1 and 100, exit with an error if necessary.
- Improve the previous script to congratulate the user is the sum is equals to the product.
LOCAL USER MANAGEMENT
LOCAL USER MANAGEMENT
INTRODUCTION TO USERS
whoami
$ whoami
paul
$
Tells you your username.
who - who am i
$ who
root pts/0 2014-10-10 23:07 (10.104.33.101)
paul pts/1 2014-10-10 23:30 (10.104.33.101)
laura pts/2 2014-10-10 23:34 (10.104.33.96)
tania pts/3 2014-10-10 23:39 (10.104.33.91)
$ who am i
paul pts/1 2014-10-10 23:30 (10.104.33.101)
Give you information about who is logged on the system.
w
$ w
23:34:07 up 31 min, 2 users, load average: 0.00, 0.01, 0.02
USER TTY LOGIN@ IDLE JCPU PCPU WHAT
root pts/0 23:07 15.00s 0.01s 0.01s top
paul pts/1 23:30 7.00s 0.00s 0.00s w
$
Shows you who is logged on and what they are doing
id
$ id
uid=1000(paul) gid=1000(paul) groups=1000(paul)
# On RHEL : you will also get SELinux context information
$ id
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r\
:unconfined_t:s0-s0:c0.c1023
Give you your user id, primary group id, and a list of the groups that you belong to
su
# To another user
laura@centos:~$ su tania
Password:
tania@centos:/home/laura$
# To root
laura@centos:~$ su root
Password:
root@centos:/home/laura#
# As root
root@centos:~# id
uid=0(root) gid=0(root) groups=0(root)
root@centos:~# su - valentina
valentina@centos:~$
# Use su - to get the target user's environment !
$ su - # will assume root is the target
Allow a user to run a shell as another user
run a program as another user
# Example
# User paul received the right to run useraddwith the credentials of root.
# This allows paul to create new users on the system without becoming root and without knowing the root password.
paul@centos:~$ /usr/sbin/useradd -m valentina
useradd: Permission Dernied.
useradd: cannot lock /etc/passwd; try again later.
# But with sudo it works:
paul@centos:~$ sudo /usr/sbin/useradd -m valentina
[sudo] password for paul:
$
- The sudo program allows a user to start a program with the credentials of another user.
- Before this works, the system admin has to set up the /etc/sudoers file.
- This can be useful to delegate administrative tasks to another user.
visudo
$ apropos visudo
visudo (8) - edit the sudoers file
$
Check the man page of visudo before playing with the
/etc/sudoers file !
sudo logging
$ sudo su -
We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:
#1) Respect the privacy of others.
#2) Think before you type.
#3) With great power comes great responsibility.
[sudo] password for paul:
paul is not in the sudoers file. This incident will be reported.
$
Using sudo without autorization will result in a severe warning
Check
/var/log/secure ;-)
🚀 HANDS-ON
- Run a command that displays only your currently logged on user name.
- Display a list of all logged users.
- Display your user name and your unique user identification (userid).
- Use
suto switch to another user account and get back to the previous account. - Now use
su -to switch to another user and notice the differences.
LOCAL USER MANAGEMENT
USER MANAGEMENT
/etc/passwd
$ tail /etc/passwd
apache:x:48:48:Apache:/var/www:/sbin/nologin
BIRT:x:30138:508::/home/BIRT:/bin/bash
postgres:x:26:26:PostgreSQL Server:/var/lib/pgsql:/bin/bash
The local user database on Linux (and most Unixes).
The columns contain the username, an x, the user id, the primary group id, a description, the name of the home directory, and the login shell.
$ man 5 passwd
root
$ head -1 /etc/passwd
root:x:0:0:root:/root:/bin/bash
The root user always has userid 0 (regardless of the name of the account).
useradd - userdel
[root@centos ~]# useradd -m -d /home/alexis -c "alexis plantin" alexis
[root@centos ~]# tail -1 /etc/passwd
alexis:x:529:529:alexis plantin:/home/alexis:/bin/bash
Add users with the
useradd command.
The user named alexis received userid 529 and primary group id 529.
#To delete the user alexis and remove also his home directory
[root@centos ~]# userdel -r alexis
usermod
$ tail -1 /etc/passwd
harry:x:516:520:harry potter:/home/harry:/bin/bash
$ usermod -c 'wizard' harry
$ tail -1 /etc/passwd
harry:x:512:520:wizard:/home/harry:/bin/bash
Modify properties of a user with the
usermod command.
creating home directories
$ mkdir /home/laura
$ chown laura:laura /home/laura
$ chmod 700 /home/laura
- Easiest way : add
-moption withuseradd - Less easy way :
/etc/skel
$ ls -al /etc/skel/
total 32
drwxr-xr-x. 2 root root 4096 Oct 23 14:16 .
drwxr-xr-x. 109 root root 12288 Jan 10 04:30 ..
-rw-r--r-- 1 root root 18 Mar 23 2017 .bash_logout
-rw-r--r-- 1 root root 176 Mar 23 2017 .bash_profile
-rw-r--r-- 1 root root 124 Mar 23 2017 .bashrc
-rw-r--r-- 1 root root 500 Nov 27 2014 .emacs
When
useradd the
-m option is used, the
/etc/skel directory is copied to the newly created home directory.
/etc/skel contains somes usually files that contain profil settings and default values for applications.
deleting home directories
-r option of
userdel will make sure that the home directory is deleted.
chsh
Users can change their login shell with the
chsh command.
$ chsh -l
/bin/sh
/bin/bash
...
...
$ chsh -s /bin/ksh
Changing shell for laura.
Password:
Shell changed.
🚀 HANDS-ON
- Create a user account names serena, including a home directory and a description tha reads Serena Williams. Do all this in one single command.
- Create a user named venus, including home directory, bash shell, a description that reads Venus Williams all in one single command.
- Verify that both users have correct entriers in /etc/passwd, /etc/shadow and /etc/group.
- Verify that their home directory was created.
- Create a user named einstime with /bin/date as default logon shell. What happens when you log on ?
LOCAL USER MANAGEMENT
USER PASSWORDS
passwd
Update users passwords.
[paul@centos ~]$ passwd
Changing password for user paul.
Changing password for paul.
(current) UNIX password:
New password:
BAD PASSWORD: The password is shorter than 8 characters
New password:
BAD PASSWORD: The password is a palindrome
New password:
BAD PASSWORD: The password is too similar to the old one
passwd: Have exhausted maximum number of retries for service
# As root:
root@centos:~# passwd paul
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
shadow file
User passwords are encrypted and kept in /etc/shadow.
[root@centos ~]# tail -4 /etc/shadow
omsagent:!!:17450::::::
apache:!!:17452::::::
nxautomation:!!:17536::::::
f298530:$6$vtRFTYzA$ufKddy.ws6oNfro/aDxND5.ngVxjY50.7ttEfxf7vWRzySx.Mka.CZ1:17546:0:99999:7:::
/etc/login.defs
Contains some default settings for user passwords.
$ grep PASS /etc/login.defs
# PASS_MAX_DAYS Maximum number of days a password may be used.
# PASS_MIN_DAYS Minimum number of days allowed between password changes.
# PASS_MIN_LEN Minimum acceptable password length.
# PASS_WARN_AGE Number of days warning given before a password expires.
PASS_MAX_DAYS 99999
PASS_MIN_DAYS 0
PASS_MIN_LEN 5
PASS_WARN_AGE 7
$
chage
Set an expiration date for a user account (
-E
), minimum (
-m
) and maximum (
-M
) password age ...
$ chage -l paul
Last password change : Jan 15, 2018
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7
disabling a password
Locking, disabling or suspending a user account.
[root@centos ~]# grep paul /etc/shadow | cut -c1-70
paul:$6$vtRFTYzA$ufKddy.ws6oNfro/aDxND5.ngVKDKUBxjY50.7ttEfxf7vWRzy
[root@centos ~]# usermod -L paul # Locking a user
[root@centos ~]# grep paul /etc/shadow | cut -c1-70
paul:!$6$vtRFTYzA$ufKddy.ws6oNfro/aDxND5.ngVKDKUBxjY50.7ttEfxf7vWRz
[root@centos ~]# usermod -U paul # Unlocking a user
[root@centos ~]# grep paul /etc/shadow | cut -c1-70
paul:$6$vtRFTYzA$ufKddy.ws6oNfro/aDxND5.ngVKDKUBxjY50.7ttEfxf7vWRzy
🚀 HANDS-ON
- Set the password for serena to hunter2.
- Also set a password for
venus and then lock the
venus user account with
usermod. Verify the locking in/etc/shadowbefore and after you lock it. - Use
passwd -dto disable the serena password. Verify the serena line in/etc/shadowbefore and after disabling. - What is the difference between locking a user account and disabling a user account's password like we just did with
usermod -Landpasswd -d? - Make sure serena has to change her password in 10 days.
LOCAL USER MANAGEMENT
USER PROFILES
system profile
Both the
bash and the
ksh shell will verify the existence of
/etc/profile and source it if it exists.
~/.bash_profile
When this file exists in home directory, then
bash will source it.
$ cat .bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/.local/bin:$HOME/bin
export PATH
~/.profile
When neither ~/.bash_profile and ~/.bash_login exist, then bash will verify the existence of ~/.profile and execute it.
$ cat .profile
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
alias vi="vim"
~/.bashrc
Leaves room for custom aliases and functions.
# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=
# User specific aliases and functions
~/.bash_logout
When exiting
bash, it can execute
~/.bash_logout.
# ~/.bash_logout
RHEL overview
| script | su | su - | ssh |
|---|---|---|---|
| ~/.bashrc | yes | yes | yes |
| ~/.profile | no | yes | yes |
| /etc/profile | no | yes | yes |
| /etc/bashrc | yes | yes | yes |
🚀 HANDS-ON
- Make a list of all the profile files on your system.
- Read the contents of each of these, often they
sourceextra scripts. - Put a unique variable, alias and function in each of those files.
- When an application depends on a setting in
$HOME/.profile, does it matter whether$HOME/.bash_profileexists or not ?
LOCAL USER MANAGEMENT
GROUPS
groupadd
Groups can be created with the groupadd command.
# groupadd tennis
# groupadd football
# groupadd smoker
# groupadd formula1
# groupadd salsa
group file
Users can be a member of several groups. Group membership is defined by the /etc/group file.
# tail -5 /etc/group
ipme:x:1000:
omiusers:x:993:nxautomation
omsagent:x:992:nxautomation
apache:x:48:
nxautomation:x:991:
Group name ; group's (encrypted) password ; group identification ; list of members
groups
A user can type the groups command to see a list of groups where the user belongs to.
[paul@centos ~]$ groups
harry sports
usermod
Group membership can be modified with the useradd or
usermod command
root@centos:~# usermod -a -G tennis inge
root@centos:~# usermod -a -G tennis katrien
root@centos:~# usermod -a -G salsa katrien
root@centos:~# usermod -a -G snooker sandra
root@centos:~# usermod -a -G formula1 annelies
root@centos:~# tail -5 /etc/group
tennis:x:1006:inge,katrien
football:x:1007:
snooker:x:1008:sandra
formula1:x:1009:annelies
salsa:x:1010:katrien
root@centos:~#
groupmod
You can change the group name with
groupmod command.
# Rename the football group to foot
root@centos:~# groupmod -n foot football
root@centos:~# tail -5 /etc/group
tennis:x:1006:inge,katrien
foot:x:1007:
formula1:x:1009:annelies
salsa:x:1010:katrien
darts:x:1008:sandra
root@centos:~#
groupdel
You can permanently remove a group with the
groupdel command.
root@centos:~# groupdel tennis
root@centos:~#
🚀 HANDS-ON
- Create the groups tennis, football and sports.
- In one command, make venus a member of tennis and sports.
- Rename the football group to foot.
- Use vi to add serena to the tennis group.
- Use the id command to verify that serena is a member of tennis.
FILE SECURITY
FILE SECURITY
STANDARD FILE PERMISSIONS
user & group owner
Every file has a user owner and a group owner.
paul@centos:~/owners$ ls -lh
total 636K
-rw-r--r--. 1 paul snooker 1.1K Apr 8 18:47 data.odt
-rw-r--r--. 1 paul paul 626K Apr 8 18:46 file1
-rw-r--r--. 1 root tennis 185 Apr 8 18:46 file2
-rw-rw-r--. 1 root root 0 Apr 8 18:47 stuff.txt
listing user accounts
You can use the following command to list all local user accounts.
paul@centos:~/owners$ $ cut -d: -f1 /etc/passwd | column
root ftp postfix
bin nobody ntp
daemon systemd-bus-proxy chrony
adm systemd-network tcpdump
lp dbus michelin
sync polkitd omsagent
shutdown libstoragemgmt apache
halt abrt nxautomation
mail rpc ipme
operator sshd
games tss
chgrp
You can change the group owner of a file using the
chgrp command.
root@centos:/home/paul/owners# ls -l file2
-rw-r--r--. 1 root tennis 185 Apr 8 18:46 file2
root@centos:/home/paul/owners# chgrp snooker file2
root@centos:/home/paul/owners# ls -l file2
-rw-r--r--. 1 root snooker 185 Apr 8 18:46 file2
root@centos:/home/paul/owners#
chown
The user owner of a file can be changed with
chown command.
root@centos:/home/paul# ls -l FileForPaul
-rw-r--r-- 1 root paul 0 2008-08-06 14:11 FileForPaul
root@centos:/home/paul# chown paul FileForPaul
root@centos:/home/paul# ls -l FileForPaul
-rw-r--r-- 1 paul paul 0 2008-08-06 14:11 FileForPaul
# You can also use chown to change both the user owner and the group owner.
root@centos:/home/paul# ls -l FileForPaul
-rw-r--r-- 1 paul paul 0 2008-08-06 14:11 FileForPaul
root@centos:/home/paul# chown root:project42 FileForPaul
root@centos:/home/paul# ls -l FileForPaul
-rw-r--r-- 1 root project42 0 2008-08-06 14:11 FileForPaul
list of special files
When you use
ls -l, the first character tells us the type of file.
| first character | file type |
|---|---|
| - | normal file |
| d | directory |
| l | symbolic link |
| p | named pipe |
| b | block device |
| c | character device |
| s | socket |
permissions
The nine characters following the file type denote the permissions in three triplets.
| permission | on a file | on a directory |
|---|---|---|
| r (read) | read file contents (cat) | read directory content (ls) |
| w (write) | change file contents (vi) | create files in (touch) |
| x (execute) | execute the file | enter the directory (cd) |
three sets of rwx
| position | characters | function |
|---|---|---|
| 1 | - | this is a regular file |
| 2-4 | rwx | permissions for the user owner |
| 5-7 | r-x | permissions for the group owner |
| 8-10 | r-- | permissions for others |
root@centos:/home/paul# ls -l
-rwxr-xr-- 1 root paul 0 2008-08-06 14:11 FileForPaul
chmod
[paul@centos]$ ls -l
total 0
-rw-r--r--. 1 paul users 0 Jan 15 14:18 permissions.txt
[paul@centos]$ chmod u+x permissions.txt
[paul@centos]$ ls -l
total 0
-rwxr--r--. 1 paul users 0 Jan 15 14:18 permissions.txt
[paul@centos]$ chmod o-r permissions.txt
[paul@centos]$ ls -l
total 0
-rwxr-----. 1 paul users 0 Jan 15 14:18 permissions.txt
[paul@centos]$ chmod a+w permissions.txt
[paul@centos]$ ls -l
total 0
-rwxrw--w-. 1 paul users 0 Jan 15 14:18 permissions.txt
[paul@centos]$ chmod u=rw,g=rw,o=r permissions.txt
[paul@centos]$ ls -l
total 0
-rw-rw-r--. 1 paul users 0 Jan 15 14:18 permissions.txt
Permissions can be changed with
chmod.
setting octal permissions
[paul@centos]$ ls -l
-rw-r--r--. 1 paul users 0 Jan 15 14:18 permissions.txt
[paul@centos]$ chmod 674 permissions.txt
[paul@centos]$ ls -l
-rw-rwxr--. 1 f298530 users 0 Jan 15 14:18 permissions.txt
| binary | octal | permission |
|---|---|---|
| 000 | 0 | --- |
| 001 | 1 | --x |
| 010 | 2 | -w- |
| 011 | 3 | -wx |
| 100 | 4 | r-- |
| 101 | 5 | r-x |
| 110 | 6 | rw- |
| 111 | 7 | rwx |
🚀 HANDS-ON
- As normal user, create a directory ~/permissions. Create a file owned by yourself in there.
- Copy a file owned by root from /etc/ to your permissions dir, who owns this file now ?
- As root, create a file in the users ~/permissions directory.
- As normal user, look at who owns this file created by root.
- Change the ownership of all files in ~/permissions to yourself.
- Make sure you have all rights to these files, and others can only read.
FILE SECURITY
ADVANCED FILE PERMISSIONS
sticky bit on directory
Prevent users from removing files that they do not own as a user owner.
[paul@centos]$ ls -ld ipme/
drwxr-xr-x. 2 paul users 6 Jan 15 14:33 ipme/
[paul@centos]$ chmod +t ipme/ # Octal syntax : chmod 1775 ipme/
[paul@centos]$ ls -ld ipme/
drwxr-xr-t. 2 paul users 6 Jan 15 14:33 ipme/
You typically find the sticky bit on the /tmp directory.
setgid bit on directory
To make sure that all files inside the directory are owned by the group owner of the directory.
[root@centos]# ll
total 0
drwxr-xr-t. 2 paul users 6 Jan 15 14:33 ipme
[root@centos]# groupadd proj55
[root@centos]# chown root:proj55 ipme/
[root@centos]# chmod 2775 ipme/
[root@centos]# touch ipme/test.txt
[root@centos]# ls -l ipme/
total 0
-rw-r--r--. 1 root proj55 0 Jan 15 14:40 test.txt
set(u/g)id bit on regular file
These two permissions cause an executable file to be executed with the permissions of the file owner instead of the executing owner.
[root@centos]# ls -l /etc/shadow
----------. 1 root root 982 Jan 15 12:40 /etc/shadow
# Changing your password required an update of this file, so how can normal
# non-root users do this ?
[root@centos]# ls -l /usr/bin/passwd
-rwsr-xr-x. 1 root root 27832 Jun 10 2014 /usr/bin/passwd
# When running the passwd program, you are executing it with root credentials.
In most case, setting the setuid bit on executables is sufficient. Setting the setgid bit will result in these programs to run with the credentials of their group owner.
🚀 HANDS-ON
- Set up a directory, owned by the group sports.
- Members of the sports group should be able to create files in this directory.
- All files created in this directory should be group-owned by the sports group.
- Users should be able to delete only their own user-owned files.
- Test that this works!
FILE SECURITY
ACCESS CONTROL LISTS
acl in /etc/fstab
File systems that support access control lists have to be mounted with the acl option listed in /etc/fstab.
root@centos:~# tail -4 /etc/fstab
/dev/sda1 / ext3 acl,relatime 0 1
/dev/sdb2 /home/data auto noacl,defaults 0 0
pasha:/home/r /home/pasha nfs defaults 0 0
wolf:/srv/data /home/wolf nfs defaults 0 0
getfacl
Reading acls can be done with /usr/bin/getfacl.
[paul@centos]$ touch bidule
[paul@centos]$ getfacl bidule
# file: bidule
# owner: paul
# group: users
user::rw-
group::r--
other::r--
setfacl
Writing or changing acls can be done with /usr/bin/setfacl.
# Add user sandra with octal permission 7 to the acl
[paul@centos]$ setfacl -m u:sandra:7 bidule
# Add the group tennis with octal permission 6 to the acl of the same file
[paul@centos]$ setfacl -m g:tennis:6 bidule
# Result is visible with getfacl
[paul@centos]$ getfacl bidule
# file: bidule
# owner: paul
# group: users
user::rw-
user:sandra:rwx
group::r--
group:tennis:rw-
mask::rwx
other::r--
remove an acl entry
# Remove an acl entry
$ setfacl -m u:sandra:7 bidule
$ getfacl bidule | grep sandra
user:sandra:rwx
$ setfacl -x sandra bidule
$ getfacl bidule | grep sandra
# Remove the complete acl
$ setfacl -b bidule
$ getfacl bidule
# file: bidule
# owner: paul
# group: paul
user::rw-
group::r--
other::r--
FILE SECURITY
FILE LINKS
inode contents
An inode is a data structure that contains metadata about a file. When the FS stores a new file on the HD, it stores not only the contents of the file but also extra properties (name, creation date, its permissions, owner ...).
# The ls -l command will display some of the inode contents
root@centos~# ls -ld /home/project42/
drwxr-xr-x 4 root pro42 4.0K Mar 27 14:29 /home/project42/
inode table
The
inode table contains all of the
inodes and is created when you create the FS (with
mkfs).
[paul@centos]$ df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda2 15472128 169689 15302439 2% /
devtmpfs 243306 346 242960 1% /dev
tmpfs 245737 1 245736 1% /dev/shm
tmpfs 245737 396 245341 1% /run
tmpfs 245737 16 245721 1% /sys/fs/cgroup
/dev/sda1 256000 335 255665 1% /boot
/dev/sdb1 655360 12 655348 1% /mnt/resource
tmpfs 245737 1 245736 1% /run/user/1001
inode number
Each
inode has a unique number (the inode number). You can see the
inode numbers with the
ls -li command.
[paul@centos]$ touch file1
[paul@centos]$ touch file2
[paul@centos]$ touch file3
[paul@centos]$ ls -li
total 0
3235 -rw-r--r--. 1 paul paul 0 Jan 15 16:08 file1
18362 -rw-r--r--. 1 paul paul 0 Jan 15 16:08 file2
18372 -rw-r--r--. 1 paul paul 0 Jan 15 16:08 file3
All the information you see with this ls command resides in the inode, except for the filename (contained in the directory).
inode and file contents
Let's put some data in one of the files.
[paul@centos]$ ls -li
total 0
3235 -rw-r--r--. 1 paul paul 0 Jan 15 16:08 file1
18362 -rw-r--r--. 1 paul paul 0 Jan 15 16:08 file2
18372 -rw-r--r--. 1 paul paul 0 Jan 15 16:08 file3
[paul@centos]$ cat file2
It is winter now and it is very cold.
We do not like the cold, we prefer hot summer nights.
[paul@centos]$
The data that is displayed by the cat command is not in the inode, but somewhere else on the disk. The inode contains a pointer to that data.
a directory is a table
A directory is a special kind of file that contains a table which maps filenames to inodes.
[paul@centos]$ ls -al
total 0
drwxr-xr-x. 2 paul users 45 Jan 15 16:08 .
drwx------. 5 paul users 119 Jan 15 14:18 ..
-rw-r--r--. 1 paul paul 0 Jan 15 16:08 file1
-rw-r--r--. 1 paul paul 0 Jan 15 16:08 file2
-rw-r--r--. 1 paul paul 0 Jan 15 16:08 file3
hard links
Create a
hard link to a file with
ln.
[paul@centos]# ln file2 hardlink_to_file2
[paul@centos]# ls -al
total 0
drwxr-xr-x. 2 paul users 69 Jan 15 16:22 .
drwx------. 5 paul users 119 Jan 15 14:18 ..
-rw-r--r--. 1 paul paul 0 Jan 15 16:08 file1
-rw-r--r--. 2 paul paul 0 Jan 15 16:08 file2
-rw-r--r--. 1 paul paul 0 Jan 15 16:08 file3
-rw-r--r--. 2 paul paul 0 Jan 15 16:08 hardlink_to_file
Both files have the same inode => same permissions, owner, content.
Both file are equal, meaning you can safely remove the original file, the hardlinked file will remain.
symbolic links
Sometimes called soft links, they are not linked to inodes.
[paul@poc-dgad-mpo test]# ls -al
total 0
drwxr-xr-x. 2 paul users 70 Jan 15 16:28 .
drwx------. 5 paul users 119 Jan 15 14:18 ..
-rw-r--r--. 1 paul paul 0 Jan 15 16:28 file1
-rw-r--r--. 1 paul paul 0 Jan 15 16:28 file2
-rw-r--r--. 1 paul paul 0 Jan 15 16:28 file3
lrwxrwxrwx. 1 paul paul 5 Jan 15 16:28 symbolic_to_file2 -> file2
Permissions on a symbolic link have no meaning, since the permissions of the target apply.
Hard link are limited to their own partition, symbolic links can link anywhere.
🚀 HANDS-ON
- Create two files named winter.txt and summer.txt, put some text in them.
- Create a hard link to winter.txt named hlwinter.txt.
- Display the inode numbers of these three files.
- Use the find command to list the two hardlinked files.
- Everything about a file is in the inode, except two things : name them!
- Create a symbolic link to summer.txt called slsummer.txt.
- Find all files with inode number 2. What does this information tell you ?
THE END.
Technical Environment & Scripting
By dawlys
Technical Environment & Scripting
- 1,250