CS110 Lecture 16: Condition Variables and Semaphores
CS110: Principles of Computer Systems
Winter 2021-2022
Stanford University
Instructors: Nick Troccoli and Jerry Cain
https://commons.wikimedia.org/wiki/File:An_illustration_of_the_dining_philosophers_problem.png
CS110 Topic 3: How can we have concurrency within a single process?
Learning About Multithreading
Introduction to Threads
Mutexes and Condition Variables
Semaphores
Multithreading Patterns
Lecture 13
Lectures 14/15
This Lecture
Lectures 17/18
assign5: implement your own multithreaded news aggregator to quickly fetch news from the web!
Learning Goals
- Learn how condition variables can let threads signal to each other
- Get practice with the "available permits" resource model
- Understand how semaphores combine a mutex, counter and condition variable
Plan For Today
- Recap: Dining With Philosophers
- Condition Variables
- Semaphores
Plan For Today
- Recap: Dining With Philosophers
- Condition Variables
- Semaphores
- This is a canonical multithreading example of the potential for deadlock and how to avoid it.
- Five philosophers sit around a circular table, eating spaghetti
- There is one fork for each of them
- Each philosopher thinks, then eats, and repeats this three times for their three daily meals.
- To eat, a philosopher must grab the fork on their left and the fork on their right. Then they chow on spaghetti to nourish their big, philosophizing brain. When they're full, they put down the forks in the same order they picked them up and return to thinking for a while.
- To think, the a philosopher keeps to themselves for some amount of time. Sometimes they think for a long time, and sometimes they barely think at all.
Dining Philosophers Problem
https://commons.wikimedia.org/wiki/File:An_illustration_of_the_dining_philosophers_problem.png
- Each philosopher either holds a fork or doesn't.
- A philosopher grabs a fork by locking that mutex. If the fork is available, the philosopher continues. Otherwise, it blocks until the fork becomes available and it can have it.
- A philosopher puts down a fork by unlocking that mutex.
static void philosopher(size_t id, mutex& left, mutex& right) {
...
}
int main(int argc, const char *argv[]) {
mutex forks[kNumForks];
thread philosophers[kNumPhilosophers];
for (size_t i = 0; i < kNumPhilosophers; i++) {
mutex& left = forks[i];
mutex& right = forks[(i + 1) % kNumPhilosophers];
philosophers[i] = thread(philosopher, i, ref(left), ref(right));
}
for (thread& p: philosophers) p.join();
return 0;
}
Dining Philosophers Problem
Goal: we must encode resource constraints into our program.
Example: how many philosophers can hold the same fork at the same time?
How can we encode this into our program?
One.
Let's make a mutex for each fork.
A philosopher thinks, then eats, and repeats this three times.
- eat is modeled as grabbing the two forks, sleeping for some amount of time, and putting the forks down.
static void eat(size_t id, mutex& left, mutex& right) {
left.lock();
right.lock();
cout << oslock << id << " starts eating om nom nom nom." << endl << osunlock;
sleep_for(getEatTime());
cout << oslock << id << " all done eating." << endl << osunlock;
left.unlock();
right.unlock();
}
Dining Philosophers Problem
Spoiler: there is a race condition here that leads to deadlock.
What if: all philosophers grab their left fork and then go off the CPU?
- deadlock! All philosophers will wait on their right fork, which will never become available.
- Testing our hypothesis: insert a sleep_for call on line 3, between getting left fork and right fork
- We should be able to insert a sleep_for call anywhere in a thread routine and have no concurrency issues.
static void eat(size_t id, mutex& left, mutex& right) {
left.lock();
sleep_for(5000); // artificially force off the processor
right.lock();
cout << oslock << id << " starts eating om nom nom nom." << endl << osunlock;
sleep_for(getEatTime());
cout << oslock << id << " all done eating." << endl << osunlock;
left.unlock();
right.unlock();
}
Food For Thought
Goal: we must encode resource constraints into our program.
Example: how many philosophers can try to eat at the same time?
- Alternative: how many philosophers can eat at the same time? Two.
- Why might the first one be better? Imposes less bottlenecking while still solving the issue.
- let's add another shared variable representing a count of "permits" or "tickets" available.
- In order to try to eat (aka grab forks at all) a philosopher must get a permit
- Once done eating, a philosopher must return their permit
Race Conditions and Deadlock
Four.
How can we encode this into our program?
- Let's add a new variable in main called permits, and a lock for it called permitsLock, so that we can update it without race conditions.
- We pass these to each philosopher by reference.
int main(int argc, const char *argv[]) {
// NEW
size_t permits = 4;
mutex permitsLock;
mutex forks[5];
thread philosophers[5];
for (size_t i = 0; i < 5; i++) {
mutex& left = forks[i];
mutex& right = forks[(i + 1) % 5];
philosophers[i] = thread(philosopher, i, ref(left), ref(right), ref(permits), ref(permitsLock));
}
for (thread& p: philosophers) p.join();
return 0;
}
Tickets, Please...
- The implementation of eat changes:
- Before eating, the philosopher must get a permit
- After eating, the philosopher must return their permit.
static void eat(size_t id, mutex& left, mutex& right, size_t& permits, mutex& permitsLock) {
// NEW
waitForPermission(permits, permitsLock);
left.lock();
right.lock();
cout << oslock << id << " starts eating om nom nom nom." << endl << osunlock;
sleep_for(getEatTime());
cout << oslock << id << " all done eating." << endl << osunlock;
// NEW
grantPermission(permits, permitsLock);
left.unlock();
right.unlock();
}
Tickets, Please...
- How do we implement grantPermission?
- Recall: "To return a permit, increment by 1 and continue"
static void grantPermission(size_t& permits, mutex& permitsLock) {
permitsLock.lock();
permits++;
permitsLock.unlock();
}
grantPermission
- How do we implement waitForPermission?
- Recall:
- "If there are permits available (count > 0) then decrement by 1 and continue"
- "If there are no permits available (count == 0) then block until a permit is available"
static void waitForPermission(size_t& permits, mutex& permitsLock) {
while (true) {
permitsLock.lock();
if (permits > 0) break;
permitsLock.unlock();
sleep_for(10);
}
permits--;
permitsLock.unlock();
}
waitForPermission
Problem: this is busy waiting! We are unnecessarily and arbitrarily using CPU time to check when a permit is available.
It would be nice if someone could let us know when they return their permit. Then, we can sleep until this happens.
Plan For Today
- Recap: Dining With Philosophers
- Condition Variables
- Semaphores
Condition Variables
A condition variable is a variable that can be shared across threads and used for one thread to notify other threads when something happens. Conversely, a thread can also use this to wait until it is notified by another thread.
- We can call wait() to sleep until another thread signals this condition variable.
- We can call notify_all() to send a signal to waiting threads.
condition_variable_any myConditionVariable;
...
// thread A waits until another thread signals
myConditionVariable.wait();
...
// thread B signals, waking up thread A
myConditionVariable.notify_all();
*not final prototype for wait
How do we implement waitForPermission? Recall:
- "If there are permits available (count > 0) then decrement by 1 and continue"
- "If there are no permits available (count == 0) then block until a permit is available"
Idea:
- when someone returns a permit and it is the only one now available, notify all.
- if we need a permit but there are none available, wait.
waitForPermission
Now we must create a condition variable to pass by reference to all threads.
int main(int argc, const char *argv[]) {
mutex forks[kNumForks];
size_t permits = kNumForks - 1;
mutex permitsLock;
// NEW
condition_variable_any permitsCV;
thread philosophers[kNumPhilosophers];
for (size_t i = 0; i < kNumPhilosophers; i++) {
mutex& left = forks[i];
mutex& right = forks[(i + 1) % kNumForks];
philosophers[i] = thread(philosopher, i, ref(left), ref(right),
ref(permits), ref(permitsCV), ref(permitsLock));
}
for (thread& p: philosophers) p.join();
return 0;
}
Condition Variables
For grantPermission, we must signal when we make permits go from 0 to 1.
static void grantPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
permits++;
if (permits == 1) permitsCV.notify_all();
permitsLock.unlock();
}
grantPermission
For waitForPermission, if no permits are available we must wait until one becomes available.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
waitForPermission
Key Idea: we must give up ownership of the lock when we wait, so that someone else can put a permit back.
Spoiler: there is a race condition here that could lead to deadlock. (HINT: notifications aren't queued)
waitForPermission
thread #1
thread #2
permits = 0
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
PERMIT
🍝
😋
waitForPermission
thread #1
thread #2
permits = 0
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
🍝
😋
I need to wait for a permit in order to eat.
PERMIT
waitForPermission
thread #2
permits = 0
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
thread #1
🍝
😋
All done eating! I will return my permit.
PERMIT
waitForPermission
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
thread #1
🙂
All done eating! I will return my permit.
waitForPermission
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
thread #1
😮
Oh! I should notify that there is a permit now.
waitForPermission
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
thread #1
😲
"attention waiting threads, a permit is available!"
💤
waitForPermission
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
thread #1
waitForPermission
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
thread #1
*100 years later*
🤨
For waitForPermission, if no permits are available we must wait until one becomes available.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
waitForPermission
Key Idea: we must give up ownership of the lock when we wait, so that someone else can put a permit back.
Key Problem: if we give up the lock before calling wait(), someone could notify before we are ready, because notifications aren't queued! If that is the last notification, we may wait forever.
For waitForPermission, if no permits are available we must wait until one becomes available.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsLock.unlock();
permitsCV.wait();
permitsLock.lock();
}
permits--;
permitsLock.unlock();
}
waitForPermission
Solution: condition variables are meant for these situations. They are able to unlock the lock for us after we are put to sleep.
For waitForPermission, if no permits are available we must wait until one becomes available.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
waitForPermission
Solution: a condition variable is meant for these situations.
- It will unlock the lock for us after we are put to sleep.
- When we are notified, it will only return once it has reacquired the lock for us.
For waitForPermission, if no permits are available we must wait until one becomes available.
waitForPermission
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
Here's what cv.wait does:
- it puts the caller to sleep and unlocks the given lock, all atomically
- it wakes up when the cv is signaled
- upon waking up, it tries to acquire the given lock (and blocks until it's able to do so)
- then, cv.wait returns
For waitForPermission, if no permits are available we must wait until one becomes available.
waitForPermission
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
Spoiler: there is a race condition here that could lead to negative permits. (HINT: consider when one permit becomes available for multiple waiting threads.)
waitForPermission
thread #1
thread #2
permits = 0
🍝
😋
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
PERMIT
waitForPermission
thread #1
thread #2
permits = 0
🍝
😋
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
We need to wait for a permit in order to eat.
PERMIT
waitForPermission
thread #1
thread #2
permits = 0
🍝
😋
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
All done eating! I will return my permit.
PERMIT
waitForPermission
thread #1
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
All done eating! I will return my permit.
🙂
waitForPermission
thread #1
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
😮
Oh! I should notify that there is a permit now.
waitForPermission
thread #1
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
😲
"attention waiting threads, a permit is available!"
waitForPermission
thread #1
thread #2
permits = 1
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
🤩
waitForPermission
thread #1
thread #2
permits = 0
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
🍝
😋
PERMIT
waitForPermission
thread #1
thread #2
permits = 0
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
🍝
😋
🤩
PERMIT
waitForPermission
thread #1
thread #2
permits = <very large #>
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
thread #3
🍝
😋
🤔
PERMIT
??
PERMIT
For waitForPermission, if no permits are available we must wait until one becomes available.
waitForPermission
Key Problem: if multiple threads are woken up for one new permit, it's possible that some of them may have to continue waiting for a permit.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
if (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
For waitForPermission, if no permits are available we must wait until one becomes available.
waitForPermission
Key Problem: if multiple threads are woken up for one new permit, it's possible that some of them may have to continue waiting for a permit.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
while (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
Solution: we must call wait() in a loop, in case we must call it again to wait longer.
There is a notify_one() method to notify just one thread instead of all.
- however, here, we would then have to notify every time we put a permit back (why?)
Condition Variables
- imagine we notify just when permits goes from 0 to 1, but only notify one thread
- threads A and B have permits and are eating
- threads C and D are waiting for permits (none currently available)
- first, thread A returns their permit (0 -> 1) and signals to one thread (e.g. C)
- then, thread B returns their permit (1 -> 2), but doesn't signal
- C wakes up and claims the permit
- D is still waiting! :(
Key Idea: In our approach of notifying just when we go from 0 -> 1, others may return further permits after us; therefore, we should wake up everyone just in case. Perhaps this approach results in fewer notifications.
Other note: we still always need a loop around calls to wait - because of spurious wakeups even when a notification wasn't sent! Or if another philosopher grabs the permit before us.
Condition Variables
A condition variable is a variable that can be shared across threads and used for one thread to notify other threads when something happens. Conversely, a thread can also use this to wait until it is notified by another thread.
- We can call wait(lock) to sleep until another thread signals this condition variable. The condition variable will unlock and re-lock the specified lock for us.
- We can call notify_all() to send a signal to waiting threads.
- We call wait(lock) in a loop in case we are woken up but must wait longer.
Demo: Dining Philosophers with Condition Variables
Condition Variables
This while loop pattern is so common that there is another convenience form of wait that also includes the loop.
- There is a second parameter which is a lambda function: it should return true when we wish to stop repeatedly waiting for a notification.
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
while (permits == 0) {
permitsCV.wait(permitsLock);
}
permits--;
permitsLock.unlock();
}
Condition Variables
static void waitForPermission(size_t& permits, condition_variable_any& permitsCV, mutex& permitsLock) {
permitsLock.lock();
permitsCV.wait(permitsLock, [&permits] { return permits > 0; });
permits--;
permitsLock.unlock();
}
// possible implementation of 2-arg wait
template <Predicate pred>
void condition_variable_any::wait(mutex& m, Pred pred) {
while (!pred()) wait(m);
}
Semaphore
This "permission slip" pattern with signaling is a very common pattern:
- Have a counter, mutex and condition_variable_any to track some permission slips
- Thread-safe way to grant permission and to wait for permission (aka sleep)
- But, it's cumbersome to need 3 variables to implement this - is there a better way?
- A semaphore is a single variable type that encapsulates all this functionality
Plan For Today
- Recap: Dining With Philosophers
- Condition Variables
- Semaphores
Semaphore
A semaphore is a variable type that lets you manage a count of finite resources.
- You initialize the semaphore with the count of resources to start with
- You can request permission via semaphore::wait() - aka waitForPermission
- You can grant permission via semaphore::signal() - aka grantPermission
- Note: count can be negative! This allows for some interesting use cases (more later).
class semaphore {
public:
semaphore(int value = 0);
void wait();
void signal();
private:
int value;
std::mutex m;
std::condition_variable_any cv;
}
Semaphore
A semaphore is a variable type that lets you manage a count of finite resources.
- You initialize the semaphore with the count of resources to start with
- When a thread wants to use a permit, it first
wait
s for the permit, and thensignal
s when it is done using a permit:
- A mutex is kind of like a special case of a semaphore with one permit, but you should use a mutex in that case as it is simpler and more efficient. Additionally, the benefit of a mutex is that it can only be released by the lock-holder.
semaphore permits(5); // this will allow five permits
permits.wait(); // if five other threads currently hold permits, this will block
// only five threads can be here at once
permits.signal(); // if other threads are waiting, a permit will be available
Semaphore - signal
A semaphore is a variable type that lets you manage a count of finite resources.
- You can grant permission via semaphore::signal() - aka grantPermission
void semaphore::signal() {
m.lock();
value++;
if (value == 1) cv.notify_all();
m.unlock();
}
class semaphore {
public:
semaphore(int value = 0);
void wait();
void signal();
private:
int value;
std::mutex m;
std::condition_variable_any cv;
}
Semaphore - wait
A semaphore is a variable type that lets you manage a count of finite resources.
- You can request permission via semaphore::wait() - aka waitForPermission
void semaphore::wait() {
m.lock();
cv.wait(m, [this] { return value > 0; })
value--;
m.unlock();
}
class semaphore {
public:
semaphore(int value = 0);
void wait();
void signal();
private:
int value;
std::mutex m;
std::condition_variable_any cv;
}
Why [this]? To access instance variables in a lambda, we must capture the current object.
Here's our final version of the dining-philosophers, replacing size_t, mutex, and condition_variable_any with a single semaphore.
static void philosopher(size_t id, mutex& left, mutex& right, semaphore& permits) {
for (size_t i = 0; i < kNumMeals; i++) {
think(id);
eat(id, left, right, permits);
}
}
int main(int argc, const char *argv[]) {
// NEW
semaphore permits(kNumForks - 1);
mutex forks[kNumForks];
thread philosophers[kNumPhilosophers];
for (size_t i = 0; i < kNumPhilosophers; i++) {
mutex& left = forks[i];
mutex& right = forks[(i + 1) % kNumPhilosophers];
philosophers[i] = thread(philosopher, i, ref(left), ref(right), ref(permits));
}
for (thread& p: philosophers) p.join();
return 0;
}
And Now....We Eat!
Recap
- Recap: Dining With Philosophers
- Condition Variables
- Semaphores
Next time: multithreading patterns with mutexes, condition variables and semaphores
CS110 Lecture 16: Condition Variables and Semaphores
By Nick Troccoli
CS110 Lecture 16: Condition Variables and Semaphores
- 2,384