Lecture 05: fork and Understanding `execvp`

Principles of Computer Systems

Winter 2021

Stanford University

Computer Science Department

Instructors: Chris Gregg and

Nick Troccoli

Reading: Bryant & O'Hallaron, Chapters 10 and 8

PDF of this presentation

CS110 Topic 2: How can our programs create and interact with other programs?

Learning About Processes

Creating processes and running other programs

Inter-process communication

Signals

Race Conditions

This lecture

Lecture 6

Lecture 7

Lecture 8

Learning Goals

Get more practice with using fork() to create new processes
Understand how to use waitpid() to coordinate between processes
Learn how execvp() lets us execute another program within a process
End Goal: write our first implementation of a shell!

first-shell-soln.c

Lecture Plan

Reintroducing fork()
Practice: Seeing....Quadruple?
waitpid() and waiting for child processes
Demo: Waiting For Children
execvp()
Putting it all together: first-shell

Lecture Plan

Reintroducing fork()
Practice: Seeing....Quadruple?
waitpid() and waiting for child processes
Demo: Waiting For Children
execvp()
Putting it all together: first-shell

fork()

A system call that creates a new child process
The "parent" is the process that creates the other "child" process
From then on, both processes are running the code after the fork
The child process is identical to the parent, except:
- it has a new Process ID (PID)
- for the parent, fork() returns the PID of the child; for the child, fork() returns 0
- fork() is called once, but returns twice

pid_t pidOrZero = fork();
// both parent and child run code here onwards
printf("This is printed by two processes.\n");

fork()

What happens to variables and addresses?

int main(int argc, char *argv[]) {
    char str[128];
    strcpy(str, "Hello");
    printf("str's address is %p\n", str);
    
    pid_t pid = fork();
    
    if (pid == 0) {
        // The child should modify str
        printf("I am the child. str's address is %p\n", str);
        strcpy(str, "Howdy");
        printf("I am the child and I changed str to %s. str's address is still %p\n", str, str);
    } else {
        // The parent should sleep and print out str
        printf("I am the parent. str's address is %p\n", str);
        printf("I am the parent, and I'm going to sleep for 2 seconds.\n");
        sleep(2);
        printf("I am the parent. I just woke up. str's address is %p, and its value is %s\n", str, str);
    }

    return 0;
}

fork-copy.c

fork()

How can the parent and child use the same address to store different data?

Each program thinks it is given all memory addresses to use
The operating system maps these virtual addresses to physical addresses
When a process forks, its virtual address space stays the same
The operating system will map the child's virtual addresses to different physical addresses than for the parent

$ ./fork-copy
str's address is 0x7ffc8cfa9990
I am the parent. str's address is 0x7ffc8cfa9990
I am the parent, and I'm going to sleep for 2 seconds.
I am the child. str's address is 0x7ffc8cfa9990
I am the child and I changed str to Howdy. str's address is still 0x7ffc8cfa9990
I am the parent. I just woke up. str's address is 0x7ffc8cfa9990, and its value is Hello

fork()

Isn't it expensive to make copies of all memory when forking?

The operating system only lazily makes copies.
It will have them share physical addresses until one of them changes its memory contents to be different than the other.
This is called copy on write (only make copies when they are written to).

$ ./fork-copy
str's address is 0x7ffc8cfa9990
I am the parent. str's address is 0x7ffc8cfa9990
I am the parent, and I'm going to sleep for 2 seconds.
I am the child. str's address is 0x7ffc8cfa9990
I am the child and I changed str to Howdy. str's address is still 0x7ffc8cfa9990
I am the parent. I just woke up. str's address is 0x7ffc8cfa9990, and its value is Hello

Example: Loaded Dice

Key Idea: all state is copied from the parent to the child, even the random number generator seed! Both the parent and child will get the same return value from random().

int main(int argc, char *argv[]) {
    // Initialize the random number with a "seed value"
    // this seed state is used to generate future random numbers
    srandom(time(NULL));

    printf("This program will make you question what 'randomness' means...\n");
    pid_t pidOrZero = fork();

    // Parent goes first - both processes *always* get the same roll (why?)
    if (pidOrZero != 0) {
        int diceRoll = (random() % 6) + 1;
        printf("I am the parent and I rolled a %d\n", diceRoll);
        sleep(1);
    } else {
        sleep(1);
        int diceRoll = (random() % 6) + 1;
        printf("I am the child and I'm guessing the parent rolled a %d\n", diceRoll);
    }

    return 0;
}

not-so-random.c

Lecture Plan

Reintroducing fork()
Practice: Seeing....Quadruple?
waitpid() and waiting for child processes
Demo: Waiting For Children
execvp()
Putting it all together: first-shell

It would be nice if there was a function we could call that would "stall" our program until the child is finished.

waitpid()

A function that a parent can call to wait for its child to exit:

pid_t waitpid(pid_t pid, int *status, int options);

pid: the PID of the child to wait on (we'll see other options later)
status: where to put info about the child's termination (or NULL)
options: optional flags to customize behavior (always 0 for now)
the function returns when the specified child process exits
the return value is the PID of the child that exited, or -1 on error (e.g. no child to wait on)
If the child process has already exited, this returns immediately - otherwise, it blocks

waitpid()

int main(int argc, char *argv[]) {
    printf("Before.\n");
    pid_t pidOrZero = fork();
    
    if (pidOrZero == 0) {
        sleep(2);
        printf("I (the child) slept and the parent still waited up for me.\n");
    } else {
        pid_t result = waitpid(pidOrZero, NULL, 0);
        printf("I (the parent) finished waiting for the child.  This always prints last.\n");
    }

    return 0;
}

$ ./waitpid
Before.
I (the child) slept and the parent still waited up for me.
I (the parent) finished waiting for the child.  This always prints last.
$

waitpid.c

waitpid()

We can use WIFEXITED and WEXITSTATUS (among others) to extract info from the status. (full program, with error checking, is right here)
The output will be the same every time! The parent will always wait for the child to finish before continuing.

int main(int argc, char *argv[]) {
    pid_t pid = fork();
    if (pid == 0) {
        printf("I'm the child, and the parent will wait up for me.\n");
        return 110; // contrived exit status (not a bad number, though)
    } else {
        int status;
        int result = waitpid(pid, &status, 0);

        if (WIFEXITED(status)) {
            printf("Child exited with status %d.\n", WEXITSTATUS(status));
        } else {
            printf("Child terminated abnormally.\n");
        }
        return 0;
    }
}

Pass in the address of an integer as the second parameter to get the child's status.

$ ./separate
I am the child, and the parent will wait up for me.
Child exited with status 110.
$

separate.c

A parent process should always wait on its children processes.

A process that finished but was not waited on by its parent is called a zombie 🧟‍♂️.
Zombies take up system resources (until they are ultimately cleaned up later by the OS)
Calling waitpid in the parent "reaps" the child process (cleans it up)
- If a child is still running, waitpid in the parent will block until it finishes, and then clean it up
- If a child process is a zombie, waitpid will return immediately and clean it up
Orphaned child processes get "adopted" by the init process (PID 1)

waitpid()

Make sure to reap your zombie children.

(Wait, what?)

Lecture Plan

Reintroducing fork()
Practice: Seeing....Quadruple?
waitpid() and waiting for child processes
Demo: Waiting For Children
execvp()
Putting it all together: first-shell

A parent can call fork multiple times, but must reap all the child processes.

A parent can use waitpid to wait on any of its children by passing in -1 as the PID.
Key Idea: The children may terminate in any order!
If waitpid returns -1 and sets errno to ECHILD, this means there are no more children.

Demo: Let's see how we might use this (reap-as-they-exit.c)

Waiting On Multiple Children, No Order

reap-as-they-exit.c

What if we want to wait for children in the order in which they were created?

Check out the abbreviated program below (full program with error checking right here):

int main(int argc, char *argv[]) {
    pid_t children[kNumChildren];

    for (size_t i = 0; i < kNumChildren; i++) {
        children[i] = fork();
        if (children[i] == 0) exit(110 + i);
    }

    for (size_t i = 0; i < kNumChildren; i++) {
        int status;
        pid_t pid = waitpid(children[i], &status, 0);
        assert(WIFEXITED(status));
        printf("Child with pid %d accounted for (return status of %d).\n", children[i], WEXITSTATUS(status));
    }

    return 0;
}

Waiting On Multiple Children, In Order

reap-in-fork-order.c

This program reaps processes in the order they were spawned.
Child processes may not finish in this order, but they are reaped in this order.
- E.g. first child could finish last, holding up first loop iteration
Sample run below - the pids change between runs, but even those are guaranteed to be published in increasing order.

Waiting On Multiple Children, In Order

$ ./reap-in-fork-order 
Child with pid 12649 accounted for (return status of 110).
Child with pid 12650 accounted for (return status of 111).
Child with pid 12651 accounted for (return status of 112).
Child with pid 12652 accounted for (return status of 113).
Child with pid 12653 accounted for (return status of 114).
Child with pid 12654 accounted for (return status of 115).
Child with pid 12655 accounted for (return status of 116).
Child with pid 12656 accounted for (return status of 117).
$

Lecture Plan

Reintroducing fork()
Practice: Seeing....Quadruple?
waitpid() and waiting for child processes
Demo: Waiting For Children
execvp()
Putting it all together: first-shell

The most common use for fork is not to spawn multiple processes to split up work, but instead to run a completely separate program under your control and communicate with it.

This is what a shell is; it is a program that prompts you for commands, and it executes those commands in separate processes. Let's take a look.

execvp()

execvp is a function that lets us run another program in the current process.

int execvp(const char *path, char *argv[]);

execvp()

It runs the executable at the specified path, completely cannibalizing the current process.

If successful, execvp never returns in the calling process
If unsuccessful, execvp returns -1

To run another executable, we must specify the (NULL-terminated) arguments to be passed into its main function, via the argv parameter.

For our programs, path and argv[0] will be the same

execvp has many variants (execle, execlp, and so forth. Type man execvp for more). We rely on execvp in CS110.

int main(int argc, char *argv[]) {
    char *args[] = {"/bin/ls", "-l", "/usr/class/cs110/lecture-examples", NULL};
    execvp(args[0], args);
    printf("This only prints if an error occurred.\n");
    return 0;
}

$ ./execvp-demo 
total 26
drwx------ 2 troccoli operator 2048 Jan 11 21:03 cpp-primer
drwx------ 3 troccoli operator 2048 Jan 15 12:43 cs107review
drwx------ 2 troccoli operator 2048 Jan 13 14:15 filesystems
drwx------ 2 troccoli operator 2048 Jan 13 14:14 lambda
drwxr-xr-x 3 poohbear root     2048 Nov 19 13:24 map-reduce
drwx------ 2 poohbear root     4096 Nov 19 13:25 networking
drwxr-xr-x 2 poohbear root     6144 Jan 22 08:58 processes
drwxr-xr-x 2 poohbear root     2048 Oct 29 06:57 threads-c
drwxr-xr-x 2 poohbear root     4096 Oct 29 06:57 threads-cpp
$

execvp()

execvp is a function that lets us run another program in the current process.

int execvp(const char *path, char *argv[]);

execvp-demo.c

Lecture Plan

Reintroducing fork()
Practice: Seeing....Quadruple?
waitpid() and waiting for child processes
Demo: Waiting For Children
execvp()
Putting it all together: first-shell

A shell is essentially a program that repeats asking the user for a command and running that command (Demo: first-shell-soln.c)

Component 1: loop for asking for user input
Component 2: way to run an arbitrary command

What Is A Shell?

first-shell-soln.c

system()

The built-in system function can execute a given shell command.

int system(const char *command);

command is a shell command (like you would type in the terminal); e.g. "ls" or "./myProgram"
system forks off a child process that executes the given shell command, and waits for it
on success, system returns the termination status of the child

int main(int argc, char *argv[]) {
    int status = system(argv[1]);
    printf("system returned %d\n", status);
    return 0;
}

$ ./system-demo "ls -l"
total 26
drwx------ 2 troccoli operator 2048 Jan 11 21:03 cpp-primer
drwx------ 3 troccoli operator 2048 Jan 15 12:43 cs107review
drwx------ 2 troccoli operator 2048 Jan 13 14:15 filesystems
drwx------ 2 troccoli operator 2048 Jan 13 14:14 lambda
drwxr-xr-x 3 poohbear root     2048 Nov 19 13:24 map-reduce
drwx------ 2 poohbear root     4096 Nov 19 13:25 networking
drwxr-xr-x 2 poohbear root     6144 Jan 21 19:38 processes
drwxr-xr-x 2 poohbear root     2048 Oct 29 06:57 threads-c
drwxr-xr-x 2 poohbear root     4096 Oct 29 06:57 threads-cpp
system returned 0
$

system-demo.c

mysystem()

We can implement our own version of system with fork(), waitpid() and execvp()!

int mysystem(const char *command);

call fork to create a child process
In the child, call execvp with the command to execute
In the parent, wait for the child with waitpid and then return exit status info

One twist; not all shell commands are executable programs, and some need parsing.

We can't just pass the command to execvp
Solution: there is a program called sh that runs any shell command
- e.g. /bin/sh -c "ls -a" runs the command "ls -a"
- We can call execvp to run /bin/sh with -c and the command as arguments

If execvp returns at all, an error occurred
Why not call execvp inside parent and forgo the child process altogether? Because
execvp would consume the calling process, and that's not what we want.

static int mysystem(char *command) {
    pid_t pidOrZero = fork();
    if (pidOrZero == 0) {
        char *arguments[] = {"/bin/sh", "-c", command, NULL};
        execvp(arguments[0], arguments);
        // If the child gets here, there was an error
        exitIf(true, kExecFailed, stderr, "execvp failed to invoke this: %s.\n", command);
    }

    // If we are the parent, wait for the child
    int status;
    waitpid(pidOrZero, &status, 0);
    return WIFEXITED(status) ? WEXITSTATUS(status) : -WTERMSIG(status);
}

Here's the implementation, with minimal error checking (the full version is right here):

mysystem()

first-shell-soln.c

Lecture Recap

Reintroducing fork()
Practice: Seeing....Quadruple?
waitpid() and waiting for child processes
Demo: Waiting For Children
execvp()
Putting it all together: first-shell

Next time: Interprocess communication

Practice Problems

Practice: fork()

int main(int argc, char *argv[]) {
    printf("Starting the program\n");
    pid_t pidOrZero1 = fork();
    pid_t pidOrZero2 = fork();
    
    if (pidOrZero1 != 0 && pidOrZero2 != 0) {
        printf("Hello\n");
    }

    if (pidOrZero2 != 0) {
        printf("Hi there\n");
    }

    return 0;
}

How many processes run in total?

a) 1 b) 2 c) 3 d) 4

How many times is "Hello" printed?

a) 1 b) 2 c) 3 d) 4

How many times is "Hi there" printed?

a) 1 b) 2 c) 3 d) 4

Parent

pidOrZero1 = nonzero

pidOrZero2 = nonzero

int main(int argc, char *argv[]) {
    printf("Starting the program\n");
    pid_t pidOrZero1 = fork();
    pid_t pidOrZero2 = fork();
    
    if (pidOrZero1 != 0 && pidOrZero2 != 0) {
        printf("Hello\n");
    }

    if (pidOrZero2 != 0) {
        printf("Hi there\n");
    }

    return 0;
}

Practice: fork()

First Child

pidOrZero1 = 0

pidOrZero2 = nonzero

Grandchild

pidOrZero1 = 0

pidOrZero2 = 0

Second Child

pidOrZero1 = nonzero

pidOrZero2 = 0

What if we want to spawn a single child and wait for that child before spawning another child?

static const int kNumChildren = 8;
int main(int argc, char *argv[]) {
    for (size_t i = 0; i < kNumChildren; i++) {
        pid_t pidOrZero = fork();
        if (pidOrZero == 0) {
            printf("Hello from child %d!\n", getpid());
            return 110 + i;
        }

        int status;
        pid_t pid = waitpid(pidOrZero, &status, 0);        
        if (WIFEXITED(status)) {
            printf("Child with pid %d exited normally with status %d\n", pid, WEXITSTATUS(status));
        } else {
            printf("Child with pid %d exited abnormally\n", pid);
        }
    }

    return 0;
}

Waiting On Children

spawn-and-reap.c

Check out the abbreviated program below (full program with error checking right here):

Lecture 05: fork and Understanding execvp

CS110 Topic 2: How can our programs create and interact with other programs?

Learning About Processes

This lecture

Lecture 6

Lecture 7

Lecture 8

Learning Goals

Lecture Plan

Lecture Plan

fork()

fork()

fork()

fork()

Example: Loaded Dice

Lecture Plan

It would be nice if there was a function we could call that would "stall" our program until the child is finished.

waitpid()

waitpid()

waitpid()

waitpid()

Make sure to reap your zombie children.

Lecture Plan

Waiting On Multiple Children, No Order

Waiting On Multiple Children, In Order

Waiting On Multiple Children, In Order

Lecture Plan

execvp()

execvp()

execvp()

Lecture Plan

What Is A Shell?

system()

mysystem()

mysystem()

Lecture Recap

Practice Problems

Practice: fork()

Practice: fork()

Waiting On Children

Lecture 05: fork and Understanding `execvp`