CS110 Lecture 13: Introduction to Multithreading

CS110: Principles of Computer Systems

Winter 2021-2022

Stanford University

Instructors: Nick Troccoli and Jerry Cain

The Stanford University logo

PDF of this presentation

A graphic "words that will never be the same after CS110". It has various drawings below it of: fork (labeled "fork"), needle and thread (labeled "thread"), a bundled baby (labeled "child"), a shovel (labeled "farm"), a seashell (labeled "shell"), an open padlock (labeled "lock"), a knife (labeled "kill"), a paper document (labeled "permit"), a dead face with two Xs for eyes and a stuck-out tongue (labeled "zombie"), a pipe (labeled "pipe") and a hat (labeled "workers"). Below the diagram is the author's name and a smiley face: "Ecy :)"

Illustration courtesy of Ecy King, CS110 Champion, Spring 2021

CS110 Topic 3: How can we have concurrency within a single process?

Learning About Multithreading

Introduction to Threads

Mutexes and Condition Variables

Semaphores

Multithreading Patterns

Today

Lectures 14/15

Lecture 16

Lectures 17/18

Today's Learning Goals

  • Learn about how threads allow for concurrency within a single process
  • Understand the differences between threads and processes
  • Discover some of the pitfalls of threads sharing the same virtual address space

Plan For Today

  • Introducing multithreading
  • Example: greeting friends
  • Race conditions
  • Threads share memory
  • Completing tasks in parallel
  • Example: selling tickets

Plan For Today

  • Introducing multithreading
  • Example: greeting friends
  • Race conditions
  • Threads share memory
  • Completing tasks in parallel
  • Example: selling tickets

From Processes to Threads

  • Multiprocessing has allowed us to spawn other processes to do tasks or run programs
  • Powerful; can execute/ wait on other programs, secure (separate memory space), communicate with pipes and signals
  • But limited; interprocess communication is cumbersome, hard to share data/coordinate
  • Is there another way we can have concurrency beyond multiprocessing that handles these tradeoffs differently?

We can have concurrency within a single process using threads: independent execution sequences within a single process.

  • Threads let us run multiple functions in our program concurrently
  • Multithreading is very common to parallelize tasks, especially on multiple cores
  • In C++: spawn a thread using thread() and the thread variable type and specify what function you want the thread to execute (optionally passing parameters!)
  • Thread manager switches between executing threads like the OS scheduler switches between executing processes
  • Each thread operates within the same process, so they share a virtual address space (!) (globals, text, data, and heap segments)
  • The processes's stack segment is divided into a "ministack" for each thread.
  • Many similarities between threads and processes; in fact, threads are often called lightweight processes.

Multithreading

Processes:

  • isolate virtual address spaces (good: security and stability, bad: harder to share info)
  • can run external programs easily (fork-exec) (good)
  • harder to coordinate multiple tasks within the same program (bad)

Threads:

  • share virtual address space (bad: security and stability, good: easier to share info)
  • can't run external programs easily (bad)
  • easier to coordinate multiple tasks within the same program (good)

Threads vs. Processes

C++ thread

A thread object can be spawned to run the specified function with the given arguments.

thread myThread(myFunc, arg1, arg2, ...);
  • myFunc: the function the thread should execute asynchronously
  • args: a list of arguments (any length, or none) to pass to the function upon execution
  • Once initialized with this constructor, the thread may execute at any time!
  • Thread function's return value is ignored (can pass by reference instead)

C++ thread

For multiple threads, we must wait on a specific thread one at a time:

thread friends[5];

...

for (size_t i = 0; i < 5; i++) {
	friends[i].join();
}

To wait on a thread to finish, use the .join() method:

thread myThread(myFunc, arg1, arg2);

... // do some work

// Wait for thread to finish (blocks)
myThread.join();

Plan For Today

  • Introducing multithreading
  • Example: greeting friends
  • Race conditions
  • Threads share memory
  • Completing tasks in parallel
  • Example: selling tickets

Our First Threads Program

static const size_t kNumFriends = 6;

static void greeting() {
    cout << "Hello, world!" << endl;
}

int main(int argc, char *argv[]) {
  cout << "Let's hear from " << kNumFriends << " threads." << endl;  

   // declare array of empty thread handles
  thread friends[kNumFriends];

  // Spawn threads
  for (size_t i = 0; i < kNumFriends; i++) {
      friends[i] = thread(greeting); 
  }

  // Wait for threads
  for (size_t i = 0; i < kNumFriends; i++) {
     friends[i].join();    
  }

  cout << "Everyone's said hello!" << endl;
  return 0;
}

Our First Threads Program

Our First Threads Program

static const size_t kNumFriends = 6;

static void greeting(size_t i) {
    cout << "Hello, world! I am thread " << i << endl;
}

int main(int argc, char *argv[]) {
  cout << "Let's hear from " << kNumFriends << " threads." << endl;  

   // declare array of empty thread handles
  thread friends[kNumFriends];

  // Spawn threads
  for (size_t i = 0; i < kNumFriends; i++) {
      friends[i] = thread(greeting, i); 
  }

  // Wait for threads
  for (size_t i = 0; i < kNumFriends; i++) {
     friends[i].join();    
  }

  cout << "Everyone's said hello!" << endl;
  return 0;
}

Our First Threads Program

C++ thread

We can also initialize an array of threads as follows (note the loop by reference):

thread friends[5];
for (thread& currFriend : friends) {
    currFriend = thread(myFunc, arg1, arg2);
}   
// declare array of empty thread handles
thread friends[5];

// Spawn threads
for (size_t i = 0; i < 5; i++) {
	friends[i] = thread(myFunc, arg1, arg2); 
}

We can make an array of threads as follows:

Plan For Today

  • Introducing multithreading
  • Example: greeting friends
  • Race conditions
  • Threads share memory
  • Completing tasks in parallel
  • Example: selling tickets

Race Conditions

  • Like with processes, threads can execute in unpredictable orderings.
  • A race condition is an unpredictable ordering of events where some orderings may cause undesired behavior.
  • A thread-safe function is one that will always execute correctly, even when called concurrently from multiple threads.
  • printf is thread-safe, but operator<< is not.  This means e.g. cout statements could get interleaved!
  • To avoid this, use ​oslock and osunlock (custom CS110 functions - #include "ostreamlock.h") around streams.  They ensure at most one thread has permission to write into a stream at any one time.
cout << oslock << "Hello, world!" << endl << osunlock;

Our First Threads Program

static const size_t kNumFriends = 6;

static void greeting(size_t i) {
    cout << oslock << "Hello, world! I am thread " << i << endl << osunlock;
}

int main(int argc, char *argv[]) {
  cout << "Let's hear from " << kNumFriends << " threads." << endl;  

   // declare array of empty thread handles
  thread friends[kNumFriends];

  // Spawn threads
  for (size_t i = 0; i < kNumFriends; i++) {
      friends[i] = thread(greeting, i); 
  }

  // Wait for threads
  for (size_t i = 0; i < kNumFriends; i++) {
     friends[i].join();    
  }

  cout << "Everyone's said hello!" << endl;
  return 0;
}
friends.cc

Plan For Today

  • Introducing multithreading
  • Example: greeting friends
  • Race conditions
  • Threads share memory
  • Completing tasks in parallel
  • Example: selling tickets

Threads Share Memory

  • Unlike parent/child processes, threads execute in the same virtual address space
  • This means we can e.g. pass parameters by reference and have all threads access/modify them!
  • To pass by reference with thread(), we must use the special ref() function around any reference parameters:
static void greeting(size_t& i) {
	...
}

for (size_t i = 0; i < kNumFriends; i++) {
    friends[i] = thread(greeting, ref(i)); 
}

Threads Share Memory

for (size_t i = 0; i < kNumFriends; i++) {
    friends[i] = thread(greeting, ref(i)); 
}

_start

greeting

main

argc

argv

i

args

args

args

args

args

args

created thread stacks

main stack

Here, we can just pass by copy instead.  But keep an eye out for consequences of shared memory!

Threads Share Memory

Plan For Today

  • Introducing multithreading
  • Example: greeting friends
  • Race conditions
  • Threads share memory
  • Completing tasks in parallel
  • Example: selling tickets
  • Threads allow a process to parallelize a problem across multiple cores
  • Consider a scenario where we want to sell 250 tickets and have 10 cores
  • Simulation: let each thread help sell tickets until none are left
int main(int argc, const char *argv[]) {
    thread ticketAgents[kNumTicketAgents];
    size_t remainingTickets = 250;
    
    for (size_t i = 0; i < kNumTicketAgents; i++) {
        ticketAgents[i] = thread(sellTickets, i, ref(remainingTickets));
    }
    for (thread& ticketAgent: ticketAgents) {
        ticketAgent.join();
    }
    
    cout << "Ticket selling done!" << endl;
    return 0;
}

Thread-Level Parallelism

confused-ticket-agents.cc

Demo: confused-ticket-agents.cc

confused-ticket-agents.cc
  • There is a race condition in this code caused by multiple threads accessing remainingTickets.
static void sellTickets(size_t id, size_t& remainingTickets) {
    while (remainingTickets > 0) {
        sleep_for(500);  // simulate "selling a ticket"
        remainingTickets--;
        cout << oslock << "Thread #" << id << " sold a ticket (" << remainingTickets
             << " remain)." << endl << osunlock;
    }
    cout << oslock << "Thread #" << id << " sees no remaining tickets to sell and exits."
         << endl << osunlock;
}

Overselling Tickets

thread #1

thread #2

thread #3

remainingTickets = 1

  • There is a race condition in this code caused by multiple threads accessing remainingTickets.
static void sellTickets(size_t id, size_t& remainingTickets) {
    while (remainingTickets > 0) {
        sleep_for(500);  // simulate "selling a ticket"
        remainingTickets--;
        cout << oslock << "Thread #" << id << " sold a ticket (" << remainingTickets
             << " remain)." << endl << osunlock;
    }
    cout << oslock << "Thread #" << id << " sees no remaining tickets to sell and exits."
         << endl << osunlock;
}

Overselling Tickets

thread #1

thread #2

thread #3

Line 2: checking if there are tickets left.  Yep!

remainingTickets = 1

  • There is a race condition in this code caused by multiple threads accessing remainingTickets.
static void sellTickets(size_t id, size_t& remainingTickets) {
    while (remainingTickets > 0) {
        sleep_for(500);  // simulate "selling a ticket"
        remainingTickets--;
        cout << oslock << "Thread #" << id << " sold a ticket (" << remainingTickets
             << " remain)." << endl << osunlock;
    }
    cout << oslock << "Thread #" << id << " sees no remaining tickets to sell and exits."
         << endl << osunlock;
}

Overselling Tickets

thread #1

thread #2

thread #3

Line 2: checking if there are tickets left.  Yep!

remainingTickets = 1

          z

        z

    z

  • There is a race condition in this code caused by multiple threads accessing remainingTickets.
static void sellTickets(size_t id, size_t& remainingTickets) {
    while (remainingTickets > 0) {
        sleep_for(500);  // simulate "selling a ticket"
        remainingTickets--;
        cout << oslock << "Thread #" << id << " sold a ticket (" << remainingTickets
             << " remain)." << endl << osunlock;
    }
    cout << oslock << "Thread #" << id << " sees no remaining tickets to sell and exits."
         << endl << osunlock;
}

Overselling Tickets

thread #1

thread #2

thread #3

Line 2: checking if there are tickets left.  Yep!

remainingTickets = 1

          z

        z

    z

          z

        z

    z

  • There is a race condition in this code caused by multiple threads accessing remainingTickets.
static void sellTickets(size_t id, size_t& remainingTickets) {
    while (remainingTickets > 0) {
        sleep_for(500);  // simulate "selling a ticket"
        remainingTickets--;
        cout << oslock << "Thread #" << id << " sold a ticket (" << remainingTickets
             << " remain)." << endl << osunlock;
    }
    cout << oslock << "Thread #" << id << " sees no remaining tickets to sell and exits."
         << endl << osunlock;
}

Overselling Tickets

thread #1

thread #2

thread #3

Line 4: Selling ticket!

remainingTickets = 0

          z

        z

    z

          z

        z

    z

  • There is a race condition in this code caused by multiple threads accessing remainingTickets.
static void sellTickets(size_t id, size_t& remainingTickets) {
    while (remainingTickets > 0) {
        sleep_for(500);  // simulate "selling a ticket"
        remainingTickets--;
        cout << oslock << "Thread #" << id << " sold a ticket (" << remainingTickets
             << " remain)." << endl << osunlock;
    }
    cout << oslock << "Thread #" << id << " sees no remaining tickets to sell and exits."
         << endl << osunlock;
}

Overselling Tickets

thread #1

thread #2

thread #3

Line 4: Selling ticket!

remainingTickets = <really large number>

          z

        z

    z

  • There is a race condition in this code caused by multiple threads accessing remainingTickets.
static void sellTickets(size_t id, size_t& remainingTickets) {
    while (remainingTickets > 0) {
        sleep_for(500);  // simulate "selling a ticket"
        remainingTickets--;
        cout << oslock << "Thread #" << id << " sold a ticket (" << remainingTickets
             << " remain)." << endl << osunlock;
    }
    cout << oslock << "Thread #" << id << " sees no remaining tickets to sell and exits."
         << endl << osunlock;
}

Overselling Tickets

thread #1

thread #2

thread #3

Line 4: Selling ticket!

remainingTickets = <really large number - 1>

There is a race condition here!

  • Problem: threads could interrupt each other in between checking tickets and selling them.

 

 

​

 

  • If a thread evaluates remainingTickets > 0 to be true and commits to selling a ticket, another thread could come in and sell that same ticket before this thread does.
  • This can happen because remainingImages > 0 test and remainingImages-- aren't atomic.
  • Atomicity: externally, the code has either executed or not; external observers do not see any intermediate states mid-execution.
  • We want a thread to do the entire check-and-sell operation uninterrupted.

Overselling Tickets

static void sellTickets(size_t id, size_t& remainingTickets) {
    while (remainingTickets > 0) {
        sleep_for(500);  // simulate "selling a ticket"
        remainingTickets--;
        ...
    }
  • C++ statements aren't inherently atomic. 
  • We assume that assembly instructions are atomic; but even single C++ statements like remainingTickets-- take multiple assembly instructions.

 

 

Atomicity

  • Even if we altered the code to be something like this, it still wouldn't fix the problem:
static void sellTickets(size_t id, size_t& remainingTickets) {
    while (remainingTickets-- > 0) {
        sleep_for(500);  // simulate "selling a ticket"
        ...
    }
// gets remainingTickets
0x0000000000401a9b <+36>:    mov    -0x20(%rbp),%rax
0x0000000000401a9f <+40>:    mov    (%rax),%eax

// Decrements by 1
0x0000000000401aa1 <+42>:    lea    -0x1(%rax),%edx

// Saves updated value
0x0000000000401aa4 <+45>:    mov    -0x20(%rbp),%rax
0x0000000000401aa8 <+49>:    mov    %edx,(%rax)
  • Each core has its own registers that it has to read from
  • Each thread makes a local copy of the variable before operating on it
  • Problem: What if multiple threads do this simultaneously?  They all think there's only 128 tickets remaining and process #128 at the same time!

Atomicity

// gets remainingImages
0x0000000000401a9b <+36>:    mov    -0x20(%rbp),%rax
0x0000000000401a9f <+40>:    mov    (%rax),%eax

// Decrements by 1
0x0000000000401aa1 <+42>:    lea    -0x1(%rax),%edx

// Saves updated value
0x0000000000401aa4 <+45>:    mov    -0x20(%rbp),%rax
0x0000000000401aa8 <+49>:    mov    %edx,(%rax)

It would be nice if we could put the check-and-sell operation behind a "locked door" and say "only one thread may enter at a time to do this block of code".

Recap

  • Introducing multithreading
  • Example: greeting friends
  • Race conditions
  • Threads share memory
  • Completing tasks in parallel
  • Example: selling tickets

 

Next time:  introducing mutexes

Made with Slides.com