CS111: Operating Systems Principles, Section 1

Spring 2022
Jerry Cain

Section 1: fork, execvp, and Redirection

Section 1: fork, execvp, and Redirection

  • In C, a file can be opened for reading and/or writing using open system call, and you can set the permissions at that time, as well. The open function comes with the following signatures:
int open(const char *pathname, int flags);
int open(const char *pathname, int flags, mode_t mode);

  • The first argument names the file you'd like to interact with, e.g. "sh111.cc"

  • The second argument is a bitwise or'ed collection of flags that specifies how you'd like to interact with the file. The argument must include exactly one of the following:

    • O_RDONLY: read only

    • O_WRONLY: write only

    • O_RDWR: read and write (this one won't come up in Project 1)

  • Include one or more of the following when opening a file for writing:
    • O_CREAT: create the file if it doesn't exist
    • O_EXCL: mandate the file be created from scratch
    • O_TRUNC: clear out file contents if it already exists
  • When creating a new file, you include a third argument to specify what the new file's permissions should be (often 0644 for "rw-r--r--")
  • Successful calls to open return file descriptors, which are integer identifiers used with other I/O-related functions to read from and write to files.

Section 1: fork, execvp, and Redirection

Section 1: fork, execvp, and Redirection

  • Here's a more complete list of the I/O-specific functions provided by the OS.  In fact, the FILE * abstraction you're familiar with from C and the stream abstractions you're familiar with from C++ are all implemented in terms of these functions.





     
  • The read function reads in up to len characters from the file session identified by fd and plants what gets read into the supplied buffer. The return value is the number of bytes actually read, or 0 if the end of the file has been reached.
  • The write function assumes len meaningful bytes reside in the supplied buffer, and writes those characters out to the file accessible through fd.  You can assume that the return value always equals len, as it will be for reasonably small len values.
  • close does what you'd expect.  It shuts down the file session and frees the associated descriptor up so it can be reused again.
  • You won't need to use read and write for your Project 1, though you'll certainly use open, close, and a few others we'll soon discuss.
int open(const char *pathname, int flags);
int open(const char *pathname, int flags, mode_t mode);

ssize_t read(int fd, char buffer[], size_t len);
ssize_t write(int fd, char buffer[], size_t len);

int close(int fd);
  • The tee program that ships with Linux copies everything from standard input to standard output, making zero or more extra copies in the named files supplied as user program arguments.
  • For example, if the file contains 27 bytes—the 26 letters of the English alphabet followed by a newline character—then the following would print the alphabet to standard output and to three files named one.txt, two.txt, and three.txt.

  • If vowels.txt contains the five vowels and a newline, and tee is invoked as follows, one.txt would be overwritten to contain just the English vowels.

$ cat vowels.txt | ./tee one.txt
aeiou
$ cat one.txt 
aeiou
$ cat alphabet.txt | ./tee one.txt two.txt three.txt
abcdefghijklmnopqrstuvwxyz
$ cat one.txt 
abcdefghijklmnopqrstuvwxyz
$ cat two.txt
abcdefghijklmnopqrstuvwxyz
$ diff one.txt two.txt
$ diff one.txt three.txt

Section 1: fork, execvp, and Redirection

  • We'll work through an implementation of our tee program during the review session, but that implementation is on the next slide.

int main(int argc, char *argv[]) {
  int fds[argc];
  fds[0] = STDOUT_FILENO;
  for (size_t i = 1; i < argc; i++)
    fds[i] = open(argv[i], O_WRONLY | O_CREAT | O_TRUNC, 0644);

  char buffer[2048];
  while (true) {
    ssize_t numRead = read(STDIN_FILENO, buffer, sizeof(buffer));
    if (numRead == 0) break;
    for (size_t i = 0; i < argc; i++) write(fds[i], buffer, numRead);
  }

  for (size_t i = 1; i < argc; i++) close(fds[i]);
  return 0;
}
  • Note that argc incidentally equals the number of descriptors we need to write to. That's why we declare an int array (or rather, a descriptor array) of length argc.
  • STDIN_FILENO is a built-in constant for the number 0, which is the descriptor normally linked to standard input. STDOUT_FILENO is a constant for the number 1, which is the default descriptor bound to standard output.
  • Note the back and forth between the read call on line 9 and the repeated write calls on line 11.  We read in a collection of bytes and write those same bytes to multiple destinations.
  • Note that we close all of the descriptors (except for fds[0]) at the end.

Section 1: fork, execvp, and Redirection

Section 1: fork, execvp, and Redirection

  • Creating new processes: fork
    • Here's a simple program that knows how to spawn new processes. It uses system calls named fork, getpid, and getppid. The full program can be viewed right here.






       
    • Here's the output of two consecutive runs of the above program.
int main(int argc, char *argv[]) {
   std::cout << "Greetings from process " << getpid() 
             << " (with parent " << getppid() << ")" << std::endl;
   pid_t pid = fork();
   assert(pid >= 0);
   std::cout << "Bye-bye from process " << getpid() 
             << " (with parent " << getppid() << ")" << std::endl;
   return 0;
}
myth60$ ./basic-fork 
Greetings from process 29686! (parent 29351)
Bye-bye from process 29686! (parent 29351)
Bye-bye from process 29687! (parent 29686)
myth60$ ./basic-fork 
Greetings from process 29688! (parent 29351)
Bye-bye from process 29688! (parent 29351)
Bye-bye from process 29689! (parent 29688)
  • fork is called once, but it returns twice.
    • getpid and getppid return the process id of the caller and the process id of the caller's parent, respectively.
    • As a result, the output of our program is the output of two processes.
      • We should expect to see a single greeting but two separate bye-byes.
      • Each bye-bye is printed to the console by two different processes. The operating system decides whether the child or the parent gets to print its bye-bye first.
    • fork knows how to clone the calling process, synthesize a nearly identical copy of it, and schedule the copy to run as if it’s been running all along.
      • Think of it as a form of process mitosis, where one process becomes twins.
      • All segments (stack, data, code, etc) are faithfully replicated.
      • All open descriptors are replicated, and these copies are donated to the clone.

Section 1: fork, execvp, and Redirection

  • Reasonable question: Why does the child process insert its text into the same terminal that the parent does? And how?
    • Answer: The parent process’s file descriptors are cloned on fork.
    • Among those descriptors? File descriptors 0, 1, and 2.  The parent and child each print to their own file descriptor 1, and both are linked to the same terminal.

Section 1: fork, execvp, and Redirection

  • Next idea: Synchronizing between parent and child using waitpid
    • Synchronization between parent and child can be managed using yet another system call called waitpid. It can be used to temporarily block a process until that process terminates or stops.

       
    • The first argument supplies the pid of the child process that should complete before waitpid can return, or -1 if you'd like to wait for any one of many child processes.
    • The second argument supplies the address of an integer where process termination information can be placed (or we can pass NULL if we don't need the information).
    • The third argument is a collection of bitwise-or'ed flags we'll study later. For the moment, we'll just go with 0 as the required parameter value, which means that waitpid should only return when the process with the given pid exits.
    • The return value is the pid of the process that successfully exited, or -1 if waitpid fails (perhaps because the pid is invalid, or you passed in other bogus arguments).
pid_t waitpid(pid_t pid, int *status, int options);

Section 1: fork, execvp, and Redirection

  • Third example: Synchronizing between parent and child using waitpid
    • Consider the following program, which is more representative of how fork really gets used in practice (full program, with error checking, is right here).  
    • The parent process correctly waits for the child to complete using waitpid.
int main(int argc, char *argv[]) {
    std::cout << "Before." << std::endl;
    pid_t pid = fork();
    std::cout << "After." << std::endl;
    if (pid == 0) {
        std::cout << "I'm taking CS111!" << std::endl;
        return 111;
    }
    
    int status;
    waitpid(pid, &status, 0);
    assert(WIFEXITED(status) && WEXITSTATUS(status) == 111);
    std::cout << "Student completed CS111 and aced it!" 
              << std::endl;
    return 0;
}
  • The parent lifts child exit information out of the waitpid call, and uses the WIFEXITED macro to examine some high-order bits of its argument to confirm the process exited normally, and it uses the WEXITSTATUS macro to extract the lower eight bits of its argument to produce the child return value (111 as expected).
  • The waitpid call also donates child process-oriented resources back to the system.
  • Rounding out the trio: execvp

     

    • execvp effectively cannibalizes a process to run a different program from scratch.

      • ​path is relative or absolute pathname of the executable to be invoked.
      • argv is the argument vector that should be funneled through to the new executable's main function.
      • path and argv[0]generally end up being the same exact string.
      • If execvp fails to cannibalize the process and install a new executable image within it, it returns -1 to express failure.
      • If execvp succeeds, it 😱 never returns 😱 (to the original executable, anyway)
    • execvp has many variants (execle, execlp, and so forth. Type man execvp to see all of them).
int execvp(const char *path, char *argv[]);

Section 1: fork, execvp, and Redirection

  • Let's leverage fork, waitpid, and execvp to implement an executable called timeout, invoked like this:

     
    • timeout launches the provided command with all of the arguments that follow and allows it to run for up to n seconds before terminating it.
    • If the process running command finishes before time is up, timeout itself returns the exit code of that process without waiting any additional time.
    • If the process running command doesn’t finish before time is up, timeout kills it and returns an exit code of 124.
    • You can programmatically
      terminate a process by
      calling kill(pid, SIGKILL).
      (though you won't need
      to use kill for Project 1).













 

myth62:~$ ./timeout <n> <command> [<arg1> [<argv2 [...]]]
myth62:~$ ./timeout 5 sleep 3
myth62:~$ echo $? # this prints return value of last command
0
myth62:~$ ./timeout 5 sleep 10
myth62:~$ echo $?
124
myth62:~$ ./timeout 1 factor 1234 2345 3456
1234: 2 617
2345: 5 7 67
3456: 2 2 2 2 2 2 2 3 3 3
myth62:~$ echo $?
0
myth62:~$ ./timeout 0 factor 3125250912230709951372256510
myth62:~$ echo $?
124
myth62:~$

Section 1: fork, execvp, and Redirection

  • Check this out!
    ​










     
    • Overarching idea: spawn two processes.
      • The first process self-cannibalizes itself to run the intended command.
      • The second brute-force sleeps for the supplied number of seconds.
      • We rely on waitpid to identify which of the two processes finished first and then just terminate the other. Clever!
      • Provided the command of interest ran to completion, we return its exit status.  If time expires, we return 124.
int main(int argc, char *argv[]) {
    pid_t timed = fork();
    if (timed == 0) { execvp(argv[2], argv + 2); exit(0); }
  
    pid_t timer = fork();
    if (timer == 0) { sleep(atoi(argv[1])); return 0; }
  
    int status;
    pid_t gold = waitpid(-1, &status, 0);
    pid_t silver = gold == timed ? timer : timed;
    kill(silver, SIGKILL);
    waitpid(silver, NULL, 0);
    if (gold == timed) {
        return WEXITSTATUS(status);
    } else {
        return 124;
    }
}

Section 1: fork, execvp, and Redirection

int pipe(int fds[]);
  • Introducing the pipe system call.
    • The pipe system call takes an uninitialized array of two integers—we'll call it fds—and populates it with two file descriptors such that everything written to fds[1] can be read from fds[0].
    • Here's the prototype:

       
    • pipe is particularly useful for allowing parent processes to communicate with spawned child processes and establish communication channels between child processes.
      • Recall that the file descriptor table of the parent is cloned across fork boundaries and preserved by execvp calls.
      • That means resources referenced by the parent's pipe endpoints are also referenced by the child's copies of them. Neat!

Section 1: fork, execvp, and Redirection

  • How does pipe work?
    • To illustrate how pipe works and how messages can be passed from one process to a second, let's consider the following program (available for play right here):
int main(int argc, char *argv[]) {
    int fds[2];
    pipe(fds);
    pid_t pid = fork();
    if (pid == 0) {
        close(fds[1]);  // close is the fclose of descriptors
        char buffer[6];
        read(fds[0], buffer, sizeof(buffer)); // read is the scanf of descriptors
        std::cout << "Read the following from the pid " << getpid() << ": \"" 
                  << buffer << "\"." << std::endl;
        close(fds[0]);
        return 0;
    }
    close(fds[0]);
    std::cout << "Printing \"hello\" from pid " << getpid() << "." << std::endl;
    write(fds[1], "hello", 6); // write is the printf of descriptors
    close(fds[1]);    
    waitpid(pid, NULL, 0);
    return 0;
}

Section 1: fork, execvp, and Redirection

  • How do pipe and fork work together in this example?
    • The base address of a small integer array called fds is shared with the call to pipe.
    • pipe allocates two descriptors, setting the first to read from a resource and the second to write to that same resource.  Think of this resource as an unnamed file that only the OS knows about.
    • pipe then plants copies of those two descriptors into indices 0 and 1 of the supplied array before it returns.
    • The fork call creates a child process, which itself inherits a shallow copy of the parent's fds array.
      • Immediately after the fork call, anything printed to fds[1] is readable from the parent's fds[0] and the child's fds[0].
      • Both the parent and child are capable of publishing text to the same resource via their copies of fds[1].
    • The parent closes fds[0] before it writes to anything to fds[1] to emphasize the fact that the parent has no need to read anything from the pipe.
    • The child closes fds[1] before it reads from fds[0] to be clear it has zero interest in printing anything to the pipe.

Section 1: fork, execvp, and Redirection

  • On fork, a child process inherits a properly wired file descriptor 1—you know, stdout—that's linked to the terminal.
  • If the child process calls execvp, then the freshly installed executable still directs anything published to descriptor 1 to your screen.
  • Descriptors are duplicated across fork boundaries, allowing pipes to bridge multiple processes. These bridges between processes enable an ability to pass arbitrary data from one process to a second.
     
  • Pipes are typically used to allow one process to pass data to a second. However, a process may want to rewrite a particular descriptor—say, 0, 1, or 2—so that its bound not to the keyboard or the terminal, but rather to one of the pipe endpoints.
  • This is where the dup2 system call comes into play, and this is its prototype.
     
     
  • dup2 rewires the target descriptor so that it references the same I/O that source does.  So, after dup2 returns, source and target are bound to the same resource. If target referenced something else prior, it is first closed before any rewiring is done.
  • The return value is simply the value of target, provided dup2 manages the rewiring without drama. Otherwise, it returns -1 like any unhappy system call would.

 

int dup2(int source, int target);

Section 1: fork, execvp, and Redirection

Section 1: fork, execvp, and Redirection

  • Here's a sample call to dup2 and close that comes up quite a bit in practice:

     
  • Here, the child's STDIN_FILENO—
    presumably open and linked to the
    screen—is closed, thereby detaching it from
    the terminal it's typically attached to.
    STDIN_FILENO is then bound to the same
    resource that fds[0] is, which, as per our
    illustration, is the read end of the pipe.
  • The subsequent close call shuts down the
    read-end descriptor allocated by the original
    pipe call, leaving the child's descriptor 0 as
    the only descriptor through which material
    residing in this pipe can be read.
  • This has the dramatic effect that all reads
    from file descriptor 0—a number which is
    effectively hardcoded into all scanf, getline, and getc calls—pull in bytes from a pipe instead of the terminal. The fact that the 0 descriptor was rewired under the hood is completely hidden from the implementation of all functions that read from it.
dup2(fds[0], STDIN_FILENO); // STDIN_FILENO is a #define constant for 0
close(fds[0]);

Section 1: fork, execvp, and Redirection

  • Here are a small number of tips for Project 1:
    • The code we provide for Project 1 builds argument vectors to be C++ vector's of C++ strings, i.e. std::vector<std::string>.
      • You can for loop over the vector much as you would a traditional array.
      • All C++ vectors return their length if you call its size method.
      • Should you need to build an array of C strings from a vector of C++ strings (which you will), then you can do something like this:




         
      • Your calls to execvp require you pass through NULL-terminated arrays of old-school C strings, and the above code snippet will prove crucial to building an argument vector of C strings from a literal vector of C++ strings.
    • You might type ulimit -u 100 in at the prompt when you log in, just in case you accidentally implant a fork bomb into your Project 1 implementation.  And please, don't experiment with fork bombs now that you know about them. 😊
std::vector<std::string> arguments({"make", "clean"});
char *argv[arguments.size() + 1];
for (size_t i = 0; i < arguments.size(); i++) {
    argv[i] = (char *) arguments[i].c_str();
}
argv[arguments.size()] = NULL;

Project 1 Discussion

By Jerry Cain

Made with Slides.com

Project 1 Discussion

  • 2,084

More from Jerry Cain