Lecture 01: Welcome to CS110: Principles of Computer Systems

Principles of Computer Systems

Winter 2020

Stanford University

Computer Science Department

Instructors:

Chris Gregg and Nick Troccoli

PDF of this presentation

• What is this class all about?
  • Principles of System Design: CS110 touches on seven big principles
    • Abstraction
    • Modularity and Layering
    • Naming and Name Resolution
    • Caching
    • Virtualization
    • Concurrency
    • Client-server request-and-response

Lecture 01: Welcome to CS110: Principles of Computer Systems

• Let's take a look a the first three of these and jump right in
  • Principles of System Design: CS110 touches on seven big principles
    • Abstraction
    • Modularity and Layering
    • Naming and Name Resolution

Lecture 01: Welcome to CS110: Principles of Computer Systems

• ​Abstraction separates behavior from implementation.

• Take a look at the result of the linux file list command, ls -1 to the left.
• There is a file list, but this is simply an abstraction
  • How are files stored on the computer?
  • If everything is a 0 or a 1 to a computer, there must be some translation, and abstraction.
  • There are an infinite number of ways to store files (your second assignment will investigate one!), but the behavior of a file system is well-defined.

• Let's take a look a the first three of these and jump right in
  • Principles of System Design: CS110 touches on seven big principles
    • Abstraction
    • Modularity and Layering
    • Naming and Name Resolution

Lecture 01: Welcome to CS110: Principles of Computer Systems

• ​Abstraction separates behavior from implementation.

• What kinds of things does an operating system designer need to think about to design a file system?
  • How are the files stored? Assuming that it is in non-volatile memory (e.g., a hard drive, or SSD), what is the actual low-level form of the file storage. Keep in mind that the files must be located when required!
  • What is the relationship between a file's name location and its data location? This can be very different!
  • Are small files stored differently than large files?
  • How are files deleted so that the space doesn't go to waste?
  • Can two filenames point to the same file?
  • Does file data share the same space on the disk as metadata? (think: Heap Allocator...)

• These are just some of the questions that must be answered. But, no matter how they are answered, the behavior of the system to the user should remain constant. There are many different varieties of Unix file systems, but this should be transparent to the user through the use of abstraction.

• Let's take a look a the first three of these and jump right in
  • Principles of System Design: CS110 touches on seven big principles
    • Abstraction
    • Modularity and Layering
    • Naming and Name Resolution

Lecture 01: Welcome to CS110: Principles of Computer Systems

• ​Abstraction separates behavior from implementation.

• If we add the -i flag for our directory list, we can dig a bit more.
• It turns out that files on a Linux system have an associated inode, which is a form of modularity called layering. For a computer, it is easier to keep track of a file with a number, but for a human, the textual name is better.
• This was a decision that the Linux file system designers made! It actually bridges the abstraction layer a bit because the user can access these inodes.

• Let's take a look a the first three of these and jump right in
  • Principles of System Design: CS110 touches on seven big principles
    • Abstraction
    • Modularity and Layering
    • Naming and Name Resolution

Lecture 01: Welcome to CS110: Principles of Computer Systems

• ​Abstraction separates behavior from implementation.

• The distinction between inodes and filenames is also an example of naming and name resolution.
• Given a file's name (and, more concretely, its path), there has to be code that can figure out what the inode is that is associated with that particular path.
• This is non-trivial, especially if you want the lookup to be fast (which you do!)

• Chris Gregg (cgregg@stanford.edu)
  • Electrical Engineering undergrad Johns Hopkins, Master's of Education, Harvard, Ph.D. in Computer Engineering, University of Virginia
  • Lecturer in CS, teaching CS 106B/X, CS 107/107E, CS 110, CS208E, CS 298.
  • At Stanford since 2016, at Tufts prior, and high school teaching prior to that.
  • I love the CS 110 material!
    • It is challenging, yet interesting, and it is a new window into systems that you haven't yet seen in the CS curriculum. I guarantee that you will write programs of the sort you have not written before.
  • I love to tinker
    • Stop by my office (Gates 201) some time to see my musical typewriter project.
    • I'm always happy to chat about Arduino / Raspberry Pi / iOS apps you are working on

Lecture 01: Welcome to CS110: Principles of Computer Systems: Instructors

• Nick Troccoli (troccoli@stanford.edu)
  • ​Computer Science undergrad and grad at Stanford
  • Lecturer in CS, teaching CS106, 107, 110
  • Looking forward to meeting all of you!

Lecture 01: Welcome to CS110: Principles of Computer Systems: Instructors

• Staff and Students
  • ​​​227 students as of January 6, 2020
  • Each of you should know C and C++ reasonably well so that you can...
    • write moderately complex programs
    • read and understand portions of large code bases
    • trace memory diagrams
  • Each of you should be fluent with Unix, gcc, valgrind, and make to the extent they're covered in CS107 or its equivalent.
  • graduate student CAs
    • Feross, Andrew, Caroline, Aleksander, Kristine, Shrey, Robert, Wilhem, Eric, Clara
    • The CAs will hold office hours, lead lab sections, and grade your work

Lecture 01: Welcome to CS110: Principles of Computer Systems

• Course Web Site: https://cs110.stanford.edu
  • Check the website for information about upcoming lectures, assignment handouts, discussion sections, and links to lecture slides like the one you're working through right now
• Online Student Support
  • Peer-collaborative forum I: Piazza (for the questions that require staff response)
• Office Hours
  • Chris's office hours are Tuesdays from 9:00 until 11:00am, or by appointment, in Gates 201
  • Nick's office hours: Mon. 3-5PM and Thurs. 1-3PM, in Gates 193
  • CA's will provide a full matrix of office hours, soon to be determined
  • Office hours are not for debugging your assignments, and the CA's have been instructed to not look at code. Ever (though this is relaxed a bit for the first assignment, only)
• Contacting the instructors
  • Please use this email to contact both Chris and Nick for course-related information: cs110-win20-instructors@lists.stanford.edu

Lecture 01: Welcome to CS110: Principles of Computer Systems

• Two Textbooks
  • First textbook is other half of CS107 textbook
    • "Computer Systems: A Programmer's Perspective", by Bryant and O'Hallaron
    • Stanford Bookstore stocks custom version of just the four chapters needed for CS110
  • Second textbook is more about systems-in-the-large, less about implementation details
    • "Principles of Computer System Design: An Introduction", by Jerome H. Saltzer and M. Frans Kaashoek
    • Provided free-of-charge online, chapter by chapter. Not stocked at Stanford Bookstore by design. You can buy a copy of it from Amazon if you want.

Lecture 01: Welcome to CS110: Principles of Computer Systems

• Lecture Examples
  • Lectures will be driven by slides and coding examples, and all coding examples can be copied/cloned into local space so you can play and confirm they work properly
  • Code examples will be developed and tested on the myth machines, which is where you'll complete all of your CS110 assignments
  • The accumulation of all lecture examples will be housed in a git repository at /usr/class/cs110/lecture-examples, which you can initially git clone, and then subsequently git pull to get the newer examples as we check them in

Lecture 01: Welcome to CS110: Principles of Computer Systems

• Lecture Slides
  • We'll try to make the slides as comprehensive as possible, but working with the code yourself is going to teach you more.
  • They are not a substitute for attending lecture
    • We go off script quite a bit and discuss high-level concepts, and you're responsible for anything that comes up in lecture
    • Exams include short answer questions in addition to coding questions, so all aspects of the course are tested

Lecture 01: Welcome to CS110: Principles of Computer Systems

• CS 110 -- more specifically
  • Five main topics (more detail in a few slides):
    1. Unix Filesystems
    2. Multiprocessing (multiple processes running simultaneously)
    3. Signal Handling (sending a signal to a process)
    4. Multithreading (multiple threads in a single process running simultaneously)
    5. Networking Servers and Clients
  • There will be eight assignments, with each assignment at least one week in duration
    1. C and C++ refresher
    2. Unix Filesystems
    3. Multiprocessing Warmup
    4. Multiprocessing: Stanford Shell
    5. Multithreading I
    6. Multithreading II: ThreadPool
    7. Networking
    8. Networking, Threading, Multiprocessing: MapReduce

Lecture 01: Welcome to CS110: Principles of Computer Systems

• Overview of Linux Filesystems
  • Linux and C libraries for file manipulation: stat, struct stat, open, close, read, write, readdir, struct dirent, file descriptors, regular files, directories, soft and hard links, programmatic manipulation of them, implementation of ls, cp, find, and other core Unix utilities you probably never realized were plain old C programs
  • Naming, abstraction and layering concepts in systems as a means for managing complexity, blocks, inodes, inode pointer structure, inode as abstraction over blocks, direct blocks, indirect blocks, doubly indirect blocks, design and implementation of a file system
• Multiprocessing and Exceptional Control Flow
  • Introduction to multiprocessing, fork, waitpid, execvp, process ids, interprocess communication, context switches, user versus kernel mode, system calls and how their calling convention differs from those of normal functions
  • Protected address spaces, virtual memory, virtual to physical address mapping, scheduling
  • Concurrency versus parallelism, multiple cores versus multiple processors, concurrency issues with multiprocessing, signal masks

Course Syllabus

• Threading and Concurrency
  • Sequential programming, desire to emulate the real world within a single process using parallel threads, free-of-charge exploitation of multiple cores (two per myth machine, 12-16 per wheat machine, 16 per oat machine), pros and cons of threading versus forking
  • C++ threads, thread construction using function pointers, blocks, functors, join, detach, race conditions, mutex, IA32 implementation of lock and unlock, spinlock, busy waiting, preemptive versus cooperative multithreading, yield, sleep_for
  • Condition variables, condition_variable_any, rendezvous and thread communication, wait, notify_one, notify_all, deadlock, thread starvation
  • Semaphore concept and semaphore implementation, generalized counters, pros and cons of semaphore versus exposed condition_variable_any, thread pools, cost of threads versus processes
  • Active threads, blocked threads, ready threads, high-level implementation details of a thread manager, mutex, and condition_variable_any
  • Pure C alternatives via pthreads, pros and cons of pthreads versus C++'s thread package

Course Syllabus

• Networking and Distributed Systems
  • Client-server model, peer-to-peer model, telnet, protocols, request, response, stateless versus keep-alive connections, latency and throughput issues, gethostbyname, gethostbyaddr, IPv4 versus IPv6, struct sockaddr hierarchy of records, network-byte order
  • Ports, sockets, socket descriptors, socket, connect, bind, accept, read, read, simple echo server, time server, concurrency issues, spawning threads to isolate and manage single conversations
  • C++ layer over raw C I/O file descriptors, introduction to sockbuf and sockstream C++ classes (via socket++ open source project)
  • HTTP 1.0 and 1.1, header fields, GET, HEAD, POST, response codes, caching
  • MapReduce programming model, implementation strategies using multiple threads and multiprocessing
  • Nonblocking I/O, where normally slow system calls like accept, read, and write return immediately instead of blocking
    • select, epoll, and libev libraries all provide nonblocking I/O alternatives to maximize CPU time using a single thread of execution within a single process

Course Syllabus

• Programming Assignments
  • 60% of final grade, with eight assignments
  • Some assignments are single file, others are significant code bases to which you'll contribute. You should always become familiar with the header files and the assignment handout before you start writing a single line of code.
  • Late policy is different than it is for many other CS classes
    • Every late day potentially costs you (read below why it's potentially)
      • If you submit on time, you can get 100% of the points.
      • If you can't meet the deadline, you can still submit up to 24 hours later, but your overall score is capped at 90%
      • If you need more than 24 additional hours to submit, you can submit up to 48 hours later, but overall score is capped at 60%
      • No assignments are ever accepted more than 48 hours after the deadline
    • Exception: first assignment must be submitted on time, no late days allowed
    • Requests for extensions are routinely denied, save for extenuating circumstances (e.g. family emergency, illness requiring medical intervention, and so forth)

Course Expectations

• Discussion Sections
  • In addition to our MW lectures, you'll also sign up for an 80-minute section to meet each week
  • We introduced the CS110 discussion section for the first time almost two years ago, and the general consensus is that they've substantially improved the course
  • If you have a laptop, bring it to discussion section. Section will be a mix of theoretical work, coding exercises, and advanced software engineering etudes using gdb and valgrind
  • Discussion section signups will go live later this week
  • 5% of final grade, provided you attend all of them
    • Everyone's discussion section grade is 100%
    • Every time you miss a discussion section, your discussion section grade counts a little less, and your final exam score counts a little more
    • Exact policy details are spelled out in the Course Information handout
• Discussion Section signup will start at noon on Sunday, January 12th at Noon. Go to https://web.stanford.edu/class/cs110/templates/labs to sign up.

Course Expectations

• Midterm
  • The midterm date will be February 14th, in class.
  • We will use the laptop-based program BlueBook to take the exams. You may have used BlueBook in CS 106A/B or CS 107, but if not, it is an in-house program that allows you to type your answers for questions. It has a similar feel to taking a paper exam (e.g., no code compiling, etc.). If you do not have a laptop to use for the exam, please email Chris and we can set you up with one to use.
  • 15% of final grade, material drawn from first five or so weeks of lecture, mix of implementation and short answer questions
  • Closed-book, closed-notes, closed-electronics, one double-sided cheat sheet that you can prepare ahead of time
  • You must pass the midterm in order to pass the class
    • Passing score will be revealed on midterm solution set, which will be posted well before the withdrawal deadline
  • Multiple practice midterms will be provided
  • If you have a competing class and would prefer to take the midterm another time, we will try to accommodate this, provided you email Chris or Nick ahead of time and explain why you need to take the exam at an alternate time
  • If you have testing accommodations, please email Chris or Nick as soon as possible.

Course Expectations

• Final Exam
  • Three-hour final is Wednesday, March 18th at 3:30pm - 6:30pm, also using BlueBook.
    • 20% of final grade, cumulative, mix of implementation and short answer questions
      • Counts even more with each discussion section absence
    • Closed-book, closed-notes, closed-electronics, two double-sided cheat sheets that you can prepare ahead of time
    • You must pass the final in order to pass the class
    • Multiple practice finals will be provided
  • We may be able to accommodate an earlier exam on the 18th for students with conflicts, but we will not be able to offer the exam on another day, nor remotely. Please make sure you can attend the final on that day.
    • Email Chris and Nick directly if you need to take the final exam during an alternate time slot on Wednesday because of a competing final

Course Expectations

• Please take the honor code seriously, because the CS Department does
  • Everything you submit for a grade is expected to be original work
  • Provide detailed citations of all sources and collaborations
  • The following are clear no-no's
    • Looking at another student's code
    • Showing another student your code
    • Discussing assignments in such detail that you duplicate a portion of someone else's code in your own program
    • Uploading your code to a public repository (e.g. github) so others can find it
      • If you'd like to upload your code to a private repository, you can do so on github or some other hosting service that provides free-of-charge private hosting

Honor Code

• You should already be familiar with the Linux filesystem as a user. The filesystem uses a tree-based model to store files and directories of files. You can get details of a file in a particular directory with the ls command

Introduction to UNIX Filesystems

• You can get a more detailed listing  with the ls -al command:

• With this listing, there are two files listed as directories (d), "." and "..".  These stand for:
  • "." is the current directory
  • ".." is the parent directory
• The "rwx------" designates the permissions for a file or directory, with "r" for read permission, "w" for write permission, and "x" for execute permission (for runnable files).

Introduction to UNIX Filesystems

• There are actually three parts to the permissions line, each with the three permission types available:
  • rwx r-x r-x

 owner

               group 

                             other

Introduction to UNIX Filesystems

Introduction to UNIX Filesystems

Introduction to UNIX Filesystems

Introduction to UNIX Filesystems

UNIX Filesystem APIs

Implementing copy to emulate cp

Implementing copy to emulate cp

Implementing copy to emulate cp

Pros and cons of file descriptors over FILE pointers and C++ iostreams

Implementing t to emulate tee

Source: https://commons.wikimedia.org/wiki/File:Tee.svg

Implementing t to emulate tee