CS110: Principles of Computer Systems
Winter 2021-2022
Stanford University
Instructors: Nick Troccoli and Jerry Cain
Illustration courtesy of Roz Cyrus.
Creating processes and running other programs
Inter-process communication and Pipes
Signals
Race Conditions
assign3: implement multiprocessing programs like "trace" (to trace another program's behavior) and "farm" (parallelize tasks)
assign4: implement your own shell!
fork()
pid_t pidOrZero = fork();
// both parent and child run code here onwards
printf("This is printed by two processes.\n");
waitpid()
A function that a parent can call to wait for its child to exit:
pid_t waitpid(pid_t pid, int *status, int options);
The function returns when the specified child process exits.
execvp
is a function that lets us run another program in the current process.
int execvp(const char *path, char *argv[]);
execvp()
It runs the executable at the specified path, completely cannibalizing the current process.
To run another executable, we must specify the (NULL-terminated) arguments to be passed into its main function, via the argv parameter.
execvp
has many variants (execle
, execlp
, and so forth. Type man
execvp
for more). We rely on execvp
in CS110.
mysystem
mysystem is our own version of the built-in function system.
first-shell
int main(int argc, char *argv[]) {
char command[kMaxLineLength];
while (true) {
printf("> ");
fgets(command, sizeof(command), stdin);
// If the user entered Ctl-d, stop
if (feof(stdin)) {
break;
}
// Remove the \n that fgets puts at the end
command[strlen(command) - 1] = '\0';
int commandReturnCode = mysystem(command);
printf("return code = %d\n", commandReturnCode);
}
printf("\n");
return 0;
}
Our first-shell program is a loop in main that parses the user input and passes it to mysystem.
static int mysystem(char *command) {
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
char *arguments[] = {"/bin/sh", "-c", command, NULL};
execvp(arguments[0], arguments);
// If the child gets here, there was an error
exitIf(true, kExecFailed, stderr, "execvp failed to invoke this: %s.\n", command);
}
// If we are the parent, wait for the child
int status;
waitpid(pidOrZero, &status, 0);
return WIFEXITED(status) ? WEXITSTATUS(status) : -WTERMSIG(status);
}
first-shell
first-shell
Takeaways
fork
, execvp
, and waitpid
to do this!stsh
("Stanford shell") with much of the functionality of real Unix shells.Shells usually also let you run a command in the background by adding "&" at the end:
Let's make an updated version of mysystem called executeCommand.
static void executeCommand(char *command, bool inBackground) {
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// If we are the child, execute the shell command
char *arguments[] = {"/bin/sh", "-c", command, NULL};
execvp(arguments[0], arguments);
// If the child gets here, there was an error
exitIf(true, kExecFailed, stderr, "execvp failed to invoke this: %s.\n", command);
}
// If we are the parent, either wait or return immediately
if (inBackground) {
printf("%d %s\n", pidOrZero, command);
} else {
waitpid(pidOrZero, NULL, 0);
}
}
static void executeCommand(char *command, bool inBackground) {
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// If we are the child, execute the shell command
char *arguments[] = {"/bin/sh", "-c", command, NULL};
execvp(arguments[0], arguments);
// If the child gets here, there was an error
exitIf(true, kExecFailed, stderr, "execvp failed to invoke this: %s.\n", command);
}
// If we are the parent, either wait or return immediately
if (inBackground) {
printf("%d %s\n", pidOrZero, command);
} else {
waitpid(pidOrZero, NULL, 0);
}
}
Line 1: Now, the caller can optionally run the command in the background.
static void executeCommand(char *command, bool inBackground) {
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// If we are the child, execute the shell command
char *arguments[] = {"/bin/sh", "-c", command, NULL};
execvp(arguments[0], arguments);
// If the child gets here, there was an error
exitIf(true, kExecFailed, stderr, "execvp failed to invoke this: %s.\n", command);
}
// If we are the parent, either wait or return immediately
if (inBackground) {
printf("%d %s\n", pidOrZero, command);
} else {
waitpid(pidOrZero, NULL, 0);
}
}
Lines 11-16: The parent waits on a foreground child, but not a background child.
int main(int argc, char *argv[]) {
char command[kMaxLineLength];
while (true) {
printf("> ");
fgets(command, sizeof(command), stdin);
// If the user entered Ctl-d, stop
if (feof(stdin)) {
break;
}
// Remove the \n that fgets puts at the end
command[strlen(command) - 1] = '\0';
if (strcmp(command, "quit") == 0) break;
bool isbg = command[strlen(command) - 1] == '&';
if (isbg) {
command[strlen(command) - 1] = '\0';
}
executeCommand(command, isbg);
}
printf("\n");
return 0;
}
In main, we add two additional things:
Note that a background child isn't reaped! This is a problem - one we'll learn how to fix soon.
The pipe system call populates the 2-element array fds with two file descriptors such that everything written to fds[1]
can be read from fds[0]
. Returns 0 on success, or -1 on error.
int pipe(int fds[]);
pipe()
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
int result = pipe(fds);
// Write message to pipe (assuming here all bytes written immediately)
write(fds[1], kPipeMessage, strlen(kPipeMessage) + 1);
close(fds[1]);
// Read message from pipe
char receivedMessage[strlen(kPipeMessage) + 1];
read(fds[0], receivedMessage, sizeof(receivedMessage));
close(fds[0]);
printf("Message read: %s\n", receivedMessage);
return 0;
}
Tip: you learn to read before you learn to write (read = fds[0], write = fds[1]).
$ ./pipe-demo
Message read: Hello, this message is coming through a pipe.
Tip: you learn to read before you learn to write (read = fds[0], write = fds[1]).
pipe()
pipe
can allow processes to communicate!
Illustration courtesy of Roz Cyrus.
Illustration courtesy of Roz Cyrus.
Here's an example program showing how pipe works across processes (full program link at bottom).
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
Make a pipe just like before.
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
The parent must close all its open FDs. It never uses the Read FD so we can close it here.
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
Write to the Write FD to send a message to the child.
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
We are now done with the Write FD so we can close it here.
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
We wait for the child to terminate.
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
Key Idea: when we call fork, the child gets a copy of the parent's file descriptor table. Any open FDs in the parent at the time fork is called must be closed in both the parent and the child.
This duplication means the child's file descriptor table entries point to the same open file table entries as the parent. Thus, the open file table entries for the two pipe FDs both have reference counts of 2.
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
The child must close all its open FDs. It never uses the Write FD so we can close it here.
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
Read from the Read FD to read the message from the parent.
Key Idea: read() blocks until the bytes are available or there is no more to read (e.g. end of file or pipe write end closed). If the parent hasn't written yet, the child's call to read() will wait.
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
We are now done with the Read FD so we can close it here. Also print the received message.
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
Key Idea: the child gets a copy of the parent's file descriptor table. Any open FDs in the parent at the time fork is called must be closed in both the parent and the child.
Here, right before the fork call, the parent has 2 open file descriptors (besides 0-2): the pipe Read FD and Write FD.
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
Key Idea: the child gets a copy of the parent's file descriptor table. Any open FDs in the parent at the time fork is called must be closed in both the parent and the child.
Therefore, when the child is spawned, it also has the same 2 open file descriptors (besides 0-2): the pipe Read FD and Write FD.
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
Key Idea: the child gets a copy of the parent's file descriptor table. Any open FDs in the parent at the time fork is called must be closed in both the parent and the child.
We should close FDs when we are done with them. The parent closes them here.
static const char * kPipeMessage = "Hello, this message is coming through a pipe.";
int main(int argc, char *argv[]) {
int fds[2];
pipe(fds);
size_t bytesSent = strlen(kPipeMessage) + 1;
pid_t pidOrZero = fork();
if (pidOrZero == 0) {
// In the child, we only read from the pipe
close(fds[1]);
char buffer[bytesSent];
read(fds[0], buffer, sizeof(buffer));
close(fds[0]);
printf("Message from parent: %s\n", buffer);
return 0;
}
// In the parent, we only write to the pipe (assume everything is written)
close(fds[0]);
write(fds[1], kPipeMessage, bytesSent);
close(fds[1]);
waitpid(pidOrZero, NULL, 0);
return 0;
}
Key Idea: the child gets a copy of the parent's file descriptor table. Any open FDs in the parent at the time fork is called must be closed in both the parent and the child.
We should close FDs when we are done with them. The child closes them here.
Illustrations courtesy of Roz Cyrus.
continued...
Illustrations courtesy of Roz Cyrus.
continued...
Illustrations courtesy of Roz Cyrus.
continued...
This method of communication between processes relies on the fact that file descriptors are duplicated when forking.
This is the core idea behind how a shell can support piping between processes
(e.g. cat file.txt | uniq | sort). Let's see how this works in a shell.
Next time: more practice with pipes