CS110: Principles of Computer Systems
Winter 2021-2022
Stanford University
Instructors: Nick Troccoli and Jerry Cain
https://commons.wikimedia.org/wiki/File:An_illustration_of_the_dining_philosophers_problem.png
Introduction to Threads
Mutexes and Condition Variables
Semaphores
Multithreading Patterns
assign5: implement your own multithreaded news aggregator to quickly fetch news from the web!
https://commons.wikimedia.org/wiki/File:An_illustration_of_the_dining_philosophers_problem.png
static void philosopher(size_t id, mutex& left, mutex& right) {
...
}
int main(int argc, const char *argv[]) {
mutex forks[kNumForks];
thread philosophers[kNumPhilosophers];
for (size_t i = 0; i < kNumPhilosophers; i++) {
mutex& left = forks[i];
mutex& right = forks[(i + 1) % kNumPhilosophers];
philosophers[i] = thread(philosopher, i, ref(left), ref(right));
}
for (thread& p: philosophers) p.join();
return 0;
}
Goal: we must encode resource constraints into our program.
Example: how many philosophers can hold the same fork at the same time?
How can we encode this into our program?
One.
Let's make a mutex for each fork.
A philosopher thinks, then eats, and repeats this three times.
static void eat(size_t id, mutex& left, mutex& right) {
left.lock();
right.lock();
cout << oslock << id << " starts eating om nom nom nom." << endl << osunlock;
sleep_for(getEatTime());
cout << oslock << id << " all done eating." << endl << osunlock;
left.unlock();
right.unlock();
}
Spoiler: there is a race condition here that leads to deadlock.
What if: all philosophers grab their left fork and then go off the CPU?
static void eat(size_t id, mutex& left, mutex& right) {
left.lock();
sleep_for(5000); // artificially force off the processor
right.lock();
cout << oslock << id << " starts eating om nom nom nom." << endl << osunlock;
sleep_for(getEatTime());
cout << oslock << id << " all done eating." << endl << osunlock;
left.unlock();
right.unlock();
}
Goal: we must encode resource constraints into our program.
Example: how many philosophers can try to eat at the same time?
Four.
How can we encode this into our program?
int main(int argc, const char *argv[]) {
// NEW
size_t permits = 4;
mutex permitsLock;
mutex forks[5];
thread philosophers[5];
for (size_t i = 0; i < 5; i++) {
mutex& left = forks[i];
mutex& right = forks[(i + 1) % 5];
philosophers[i] = thread(philosopher, i, ref(left), ref(right), ref(permits), ref(permitsLock));
}
for (thread& p: philosophers) p.join();
return 0;
}
static void eat(size_t id, mutex& left, mutex& right, size_t& permits, mutex& permitsLock) {
// NEW
waitForPermission(permits, permitsLock);
left.lock();
right.lock();
cout << oslock << id << " starts eating om nom nom nom." << endl << osunlock;
sleep_for(getEatTime());
cout << oslock << id << " all done eating." << endl << osunlock;
// NEW
grantPermission(permits, permitsLock);
left.unlock();
right.unlock();
}
static void grantPermission(size_t& permits, mutex& permitsLock) {
permitsLock.lock();
permits++;
permitsLock.unlock();
}
static void waitForPermission(size_t& permits, mutex& permitsLock) {
while (true) {
permitsLock.lock();
if (permits > 0) break;
permitsLock.unlock();
sleep_for(10);
}
permits--;
permitsLock.unlock();
}
Problem: this is busy waiting! We are unnecessarily and arbitrarily using CPU time to check when a permit is available.
A condition variable is a variable that can be shared across threads and used for one thread to notify other threads when something happens. Conversely, a thread can also use this to wait until it is notified by another thread.
condition_variable_any myConditionVariable;
...
// thread A waits until another thread signals
myConditionVariable.wait();
...
// thread B signals, waking up thread A
myConditionVariable.notify_all();
*not final prototype for wait
How do we implement waitForPermission? Recall:
Idea:
Now we must create a condition variable to pass by reference to all threads.
int main(int argc, const char *argv[]) {
mutex forks[kNumForks];
size_t permits = kNumForks - 1;
mutex permitsLock;
// NEW
condition_variable_any permitsCV;
thread philosophers[kNumPhilosophers];
for (size_t i = 0; i < kNumPhilosophers; i++) {
mutex& left = forks[i];
mutex& right = forks[(i + 1) % kNumForks];
philosophers[i] = thread(philosopher, i, ref(left), ref(right),
ref(permits), ref(permitsCV), ref(permitsLock));
}
for (thread& p: philosophers) p.join();
return 0;
}
For grantPermission, we must signal when we make permits go from 0 to 1.
static void grantPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
permits++;
if (permits == 1) permitsCV.notify_all();
permitsLock.unlock();
}
For waitForPermission, if no permits are available we must wait until one becomes available.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
Key Idea: we must give up ownership of the lock when we wait, so that someone else can put a permit back.
Spoiler: there is a race condition here that could lead to deadlock. (HINT: notifications aren't queued)
thread #1
thread #2
permits = 0
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
PERMIT
🍝
😋
thread #1
thread #2
permits = 0
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
🍝
😋
I need to wait for a permit in order to eat.
PERMIT
thread #2
permits = 0
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
thread #1
🍝
😋
All done eating! I will return my permit.
PERMIT
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
thread #1
🙂
All done eating! I will return my permit.
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
thread #1
😮
Oh! I should notify that there is a permit now.
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
thread #1
😲
"attention waiting threads, a permit is available!"
💤
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
thread #1
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
thread #1
*100 years later*
🤨
For waitForPermission, if no permits are available we must wait until one becomes available.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
Key Idea: we must give up ownership of the lock when we wait, so that someone else can put a permit back.
Key Problem: if we give up the lock before calling wait(), someone could notify before we are ready, because notifications aren't queued! If that is the last notification, we may wait forever.
For waitForPermission, if no permits are available we must wait until one becomes available.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
Solution: condition variables are meant for these situations. They are able to unlock the lock for us after we are put to sleep.
For waitForPermission, if no permits are available we must wait until one becomes available.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
Solution: a condition variable is meant for these situations.
For waitForPermission, if no permits are available we must wait until one becomes available.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
Here's what cv.wait does:
For waitForPermission, if no permits are available we must wait until one becomes available.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
Spoiler: there is a race condition here that could lead to negative permits. (HINT: consider when one permit becomes available for multiple waiting threads.)
thread #1
thread #2
permits = 0
🍝
😋
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
PERMIT
thread #1
thread #2
permits = 0
🍝
😋
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
We need to wait for a permit in order to eat.
PERMIT
thread #1
thread #2
permits = 0
🍝
😋
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
All done eating! I will return my permit.
PERMIT
thread #1
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
All done eating! I will return my permit.
🙂
thread #1
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
😮
Oh! I should notify that there is a permit now.
thread #1
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
😲
"attention waiting threads, a permit is available!"
thread #1
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
🤩
thread #1
thread #2
permits = 0
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
🍝
😋
PERMIT
thread #1
thread #2
permits = 0
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
🍝
😋
🤩
PERMIT
thread #1
thread #2
permits = <very large #>
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
🍝
😋
🤔
PERMIT
??
PERMIT
For waitForPermission, if no permits are available we must wait until one becomes available.
Key Problem: if multiple threads are woken up for one new permit, it's possible that some of them may have to continue waiting for a permit.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
For waitForPermission, if no permits are available we must wait until one becomes available.
Key Problem: if multiple threads are woken up for one new permit, it's possible that some of them may have to continue waiting for a permit.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
while (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
Solution: we must call wait() in a loop, in case we must call it again to wait longer.
There is a notify_one() method to notify just one thread instead of all.
Key Idea: In our approach of notifying just when we go from 0 -> 1, others may return further permits after us; therefore, we should wake up everyone just in case. Perhaps this approach results in fewer notifications.
Other note: we still always need a loop around calls to wait - because of spurious wakeups even when a notification wasn't sent! Or if another philosopher grabs the permit before us.
A condition variable is a variable that can be shared across threads and used for one thread to notify other threads when something happens. Conversely, a thread can also use this to wait until it is notified by another thread.
This while loop pattern is so common that there is another convenience form of wait that also includes the loop.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
while (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
permitsCV.wait(permitsLock, [&permits] { return permits > 0; });
permits--;
permitsLock.unlock();
}
// possible implementation of 2-arg wait
template <Predicate pred>
void condition_variable_any::wait(mutex& m, Pred pred) {
while (!pred()) wait(m);
}
This "permission slip" pattern with signaling is a very common pattern:
A semaphore is a variable type that lets you manage a count of finite resources.
class semaphore {
public:
semaphore(int value = 0);
void wait();
void signal();
private:
int value;
std::mutex m;
std::condition_variable_any cv;
}
A semaphore is a variable type that lets you manage a count of finite resources.
wait
s for the permit, and then signal
s when it is done using a permit:
semaphore permits(5); // this will allow five permits
permits.wait(); // if five other threads currently hold permits, this will block
// only five threads can be here at once
permits.signal(); // if other threads are waiting, a permit will be available
A semaphore is a variable type that lets you manage a count of finite resources.
void semaphore::signal() {
m.lock();
value++;
if (value == 1) cv.notify_all();
m.unlock();
}
class semaphore {
public:
semaphore(int value = 0);
void wait();
void signal();
private:
int value;
std::mutex m;
std::condition_variable_any cv;
}
A semaphore is a variable type that lets you manage a count of finite resources.
void semaphore::wait() {
m.lock();
cv.wait(m, [this] { return value > 0; })
value--;
m.unlock();
}
class semaphore {
public:
semaphore(int value = 0);
void wait();
void signal();
private:
int value;
std::mutex m;
std::condition_variable_any cv;
}
Why [this]? To access instance variables in a lambda, we must capture the current object.
Here's our final version of the dining-philosophers, replacing size_t, mutex, and condition_variable_any with a single semaphore.
static void philosopher(size_t id, mutex& left, mutex& right, semaphore& permits) {
for (size_t i = 0; i < kNumMeals; i++) {
think(id);
eat(id, left, right, permits);
}
}
int main(int argc, const char *argv[]) {
// NEW
semaphore permits(kNumForks - 1);
mutex forks[kNumForks];
thread philosophers[kNumPhilosophers];
for (size_t i = 0; i < kNumPhilosophers; i++) {
mutex& left = forks[i];
mutex& right = forks[(i + 1) % kNumPhilosophers];
philosophers[i] = thread(philosopher, i, ref(left), ref(right), ref(permits));
}
for (thread& p: philosophers) p.join();
return 0;
}
Next time: multithreading patterns with mutexes, condition variables and semaphores