ENPM809V

Race Conditions

Sources

Some of the information of the information has come from pwn.college's lecture's as this was the resource that helped me the most learning about race conditions

History of Computers

  • Originally was a single CPU

Initalize()

do_action_1()

do_action_2()

do_action_3()

finalize()

History of Computers

  • Later added multiple cores per processor 
    • More computation power

Initalize()

do_action_1()

do_action_2()

do_action_3()

finalize()

pInitalize()

do_action_1()

do_action_2()

do_action_3()

finalize()

History of Computers

  • What it actually means is this...

p2_Initalize()

p2_do_action_1()

p2_do_action_2()

p2_finalize()

p1_Initalize()

p1_do_action_1()

p1_do_action_2()

p1_finalize()

History of Computers

  • What it actually means is this...

p2_Initalize()

p2_do_action_1()

p2_do_action_2()

p2_finalize()

p1_Initalize()

p1_do_action_1()

p1_do_action_2()

p1_finalize()

Order of Execution

p2_Initalize()

p2_do_action_1()

p2_do_action_2()

p2_finalize()

p1_Initalize()

p1_do_action_1()

p1_do_action_2()

p1_finalize()

p2_Initalize()

p2_do_action_1()

p2_do_action_2()

p2_finalize()

p1_Initalize()

p1_do_action_1()

p1_do_action_2()

p1_finalize()

p2_Initalize()

p2_do_action_1()

p2_do_action_2()

p2_finalize()

p1_Initalize()

p1_do_action_1()

p1_do_action_2()

p1_finalize()

What implications does it have?

As a result...

  • More cores to compute process
  • But they can execute in any order
    • Programs can execute concurrently
    • No guarantee about which process completes first
  • A race condition is formed!

Before We Start...

Lets Learn about Processes and Threads!

What's a Process?

  • It is a currently running program
  • They are in complete isolation of each other
    • Their own Virtual Memory
    • Registers
    • File Descriptors
    • Process IDs
    • Security Properties

What's a Thread?

  • A thread is a single sequential flow of control within a program
  • Threads Share Virtual Memory (excluding the stack) and File Descriptors 
  • They are independent in every other way. 

Every process has atleast one thread

Various Implementations

Threads:

  • System V
  • Posix - This is what we will focus on today

 

Example

void *threadFunc(int arg)
{
	printf("Thread %d, PID %d, TID %d, UID %d\n", arg, getpid(), gettid(), getuid());
}

void main()
{
	pthread_t thread1, thread2;
    pthread_create(&thread1, NULL, threadFunc, 1);
    pthread_create(&thread1, NULL, threadFunc, 2);
    printf("Main thread PID %d, TID %d, UID %d", getpid(), gettid(), getuid());
    pthread_join(thread1, NULL); 
    pthread_join(thread2, NULL); 
}

Underneath the hood

clone(
  child_stack = 0x7f81afdc7fb0,
  flags = CLONE_VM|		// parent and child will run in the same memory space
          CLONE_FS|		// parent and child will share filesystem info (chroot(), chdir(), and similar effects will be shared between parent and child)
          CLONE_FILES|	// parent and child will share file descriptors
          CLONE_SIGHAND|	// parent and child will share signal handlers
          CLONE_THREAD|	// signify that the child is a fellow thread of the parent
          CLONE_SYSVSEM|	// parent and child will share semaphore information
          CLONE_SETTLS|	// set a unique "Thread Local Storage" area for the new thread
          CLONE_PARENT_SETTID|	// store the new thread ID at the memory location pointed to by the tid arg below
          CLONE_CHILD_CLEARTID,	// zero out the thread ID at the location pointed to by the tid arg below when the child exits
  parent_tid=[1926535],
  tls=0x7f81afdc8700,
  child_tidptr=0x7f81afdc89d0
)
  • pthread_create uses the clone system call to create its thread.
    • What it's actually doing is creating a child_process that shares the same memory space as the parent process.

Some things to note

  • Although libc functions (like libpthread) and the system call interface are generally similar, there are a couple of discrepancies
    • setuid - the libc syscall wrapper will set the UID of all thread; however the system call itself only sets UID of the caller
    • exit  - The libc wrapper for exit() calls exit_group() meaning it will terminate all threads. The exit() system call only terminates the caller thread.

Some things to note

  • Threads run until they are finished executing!
  • The only way to exit a thread early is to call the pthread_kill function (not recommended).
  • Some common ways of handling this:
    • Global variables (though this could be unsafe if not careful).
    • Using signal handlers
    • Calling exit() (though better to let threads terminate before calling it).

A race condition is...

the ability to change the state of the program by an external context, making behavior nondeterministic.

Race Conditions in The File System

How can they be caused?

  • Multiple processes accessing the same file
    • Reading & Writing a file at the same time
  • A file opened twice
  • Having a symlink to a file so that an access can occur twice.
  • How can we achieve this?
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

int main(int argc, char **argv)
{
    int fd = open(
    	argv[1], 
    	O_WRONLY | O_CREAT | O_TRUNC,
        0755);
    
    write(fd, "#!/bin/sh\necho SAFE\n", 20); 
    close(fd);
    
    execl("/bin/sh", "/bin/sh", argv[1], NULL);
    return 0; 
}

Example Race Condition

while /bin/true; do cp -v ./catflag asdf; done
wittsend2@wittsend2-virtual-machine:[~/Documents/race_condition_test]
$ ./main asdf
SAFE
wittsend2@wittsend2-virtual-machine:[~/Documents/race_condition_test]
$ ./main asdf
SAFE
wittsend2@wittsend2-virtual-machine:[~/Documents/race_condition_test]
$ ./main asdf
SAFE
wittsend2@wittsend2-virtual-machine:[~/Documents/race_condition_test]
$ ./main asdf
SAFE
wittsend2@wittsend2-virtual-machine:[~/Documents/race_condition_test]
$ ./main asdf
FLAG

Source pwn.college

#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

int main(int argc, char **argv)
{
    int echo_fd = open("/bin/echo", O_RDONLY);

    int fd = open(
    	argv[1],
    	O_WRONLY | O_CREAT,
        0755);

    sendfile(fd, echo_fd, 0, 1024*1024);
    close(fd);

    execl(argv[1], argv[1], "SAFE", NULL);
    return 0;
}

Less Winnable Race Condition

Source pwn.college

Time of Checck to Time of Use (TOCTOU)

  • A race condition where between checking if a file exists and the usage exist, the state of the file changes.
  • How it works:
    • A condition where it checks a value occurs
    • In another thread/context, the value gets modified.
    • The program continues with new value

Strategies for Being More Winnable

if (access("file", W_OK) != 0) {
    exit(1);
}

fd = open("file", O_WRONLY);
write(fd, buffer, sizeof(buffer));

What is wrong here?

Strategies for Being More Winnable

if (access("file", W_OK) != 0) {
    exit(1);
}

fd = open("file", O_WRONLY);
write(fd, buffer, sizeof(buffer));
symlink("/etc/passwd", "file");

Victim

Attacker

Strategies for Being More Winnable

Scheduling Priorities:

  • System Calls like Nice can slow programs down by lowering scheduling class and priority
    • Check man nice and man ionice to see how it works

Path Complexity:

  • Shorter paths tend to be faster than longer paths (with many directories)
  • Kernel needs to process each directory
  • Symbolic Links can increase that complexity too
  • Example:
    • cat a/b/c/d/e/f/g/h/i/1/2/3/4/5/6/7/8/9/target_file is slower than cat target_file (remember, max path size is 4096)

Races in Threads and Memory

How are memory race conditions possible?

  • Memory is a shared resource between processes and threads
    • Shared memory API (POSIX and System V)
    • Passing pointers around
      • Global variable that is a pointer to some location in memory (must NOT BE STACK)

How are thread race conditions possible

  • Developer Mistakes 
  • Threads accessing memory unsafely 
  • Kernel Space not checking memory correctly
  • Not utilizing locking mechanisms correctly

How can They Happen

unsigned int size = 128;
void read_data() 
{
    char buffer[16];
    if (buffer < 16) 
    {
    	printf("Valid size!");
        read(0, buffer, 16); 
        printf("%s\n", buffer); 
    }
}

void *start_thread()
{
    while (true)
    {
    	read_data();
    }
}

void main()
{
    pthread_t thread; 
    pthread_create(&thread, NULL, thread_allocator, 0);
    
    while(size != 0)
    {
    	read(0, &size, 1);
    }
    
 	return; 
}
#define SECRET "123456" 

bool canAccess = false;
char readData[64]; 
void *accessSensitiveInfo()
{
	char password[64];
    printf("Enter password: ");
    scanf("%s", buffer); 
    if (password == SECRET)
    {
    	canAccess = true;
    }
	if (canAccess) 
    {
    	int fd = open("/path/to/sensitiveFile", O_RDONLY);
        read(fd, readData, 64);
        close(fd);
    }
}

void main()
{
	pthread_t user1, user2, user3; 
    pthread_create(&user1, NULL, accessSensitiveInfo, 0);
    pthread_create(&user2, NULL, accessSensitiveInfo, 0);
    pthread_create(&user3, NULL, accessSensitiveInfo, 0);
    pthread_join(&user1, NULL);
    pthread_join(&user2, NULL);
    pthread_join(&user3, NULL);
    exit(0); 
}

How can They Happen

int check_safety(char *user_buffer, int maximum_size) {
    int size;
    copy_from_user(&size, user_buffer, sizeof(size));
    return size <= maximum_size;
}

static long device_ioctl(struct file *file, unsigned int cmd, unsigned long user_buffer) {
    int size;
    char buffer[16];
    if (!check_safety(user_buffer, 16)) return;
    copy_from_user(&size, user_buffer, sizeof(size));
    copy_from_user(buffer, user_buffer+sizeof(size), size);
}
  • copy_from_user and copy_to_user are functions within kernel space to copy data from/to the user from/to kernel space.

How can They Happen

int check_safety(char *user_buffer, int maximum_size) {
    int size;
    copy_from_user(&size, user_buffer, sizeof(size));
    return size <= maximum_size;
}

static long device_ioctl(struct file *file, unsigned int cmd, unsigned long user_buffer) {
    int size;
    char buffer[16];
    if (!check_safety(user_buffer, 16)) return;
    copy_from_user(&size, user_buffer, sizeof(size));
    copy_from_user(buffer, user_buffer+sizeof(size), size);
}

Example of Time of Check to Time of Use race condition in the Linux kernel

How to prevent them

#include <stdio.h>
#include <pthread.h>

unsigned int n = 0; 
void *thread_function()
{
    while(1)
    {
        n++; 
        n--;
        printf("Num = %d\n", n);
    }
    
}

int main()
{
    pthread_t thread1;
    pthread_t thread2;
    pthread_t thread3;
    
    pthread_create(&thread1, NULL, thread_function, 0);
    pthread_create(&thread2, NULL, thread_function, 0);
    pthread_create(&thread3, NULL, thread_function, 0); 

    getchar();    
    return 0; 
}

Without Mutex

How to prevent them

unsigned int n = 0; 
pthread_mutex_t mutex l;

void *thread_function()
{
    while(1)
    {
        pthread_mutex_lock(&l);
        n++; 
        n--;
        printf("Num = %d\n", n);
        pthread_mutext_unlock(&l); 
    }
}

int main()
{
    pthread_t thread1;
    pthread_t thread2;
    pthread_t thread3;
    
    pthread_create(&thread1, NULL, thread_function, 0);
    pthread_create(&thread2, NULL, thread_function, 0);
    pthread_create(&thread3, NULL, thread_function, 0); 

    getchar();    
    return 0; 
}

With Mutex

Tools To Help

  • Valgrind - keeps track of memory and ensures proper access
    • Need to have proper test cases to trigger it
    • Also good for checking for memory leaks
  • Clang thread-sanitizer
    • Built in thread sanitizer for debugging
    • Utilize -fsanitize-thread when compiling with clang

Races in Signals and Reentrancy

What are signals?

  • Signal is a kind of interrupt used to announce asynchronous events that happen on the system.
    • Think of things like when you press ctrl+c to send a SIGINT
    • There is a limited list of possible signals that we do not create on our own.

Handling Signals

  • Processes can register signal handlers to perform an action when a signal is sent to it.
  • What happens? Signal pauses process execution and calls into signal handler. After completion:
    • Process resumeds
    • Process terminates via an exit() call.
  • Global Variables are the mechanism for signals to send data back to the process
  • You can send Any Sginal to Any Process that has the same ruid as you (even if their eUID is root)

 

Triggering Signals

  • Time Slices: the kernel gives a process a limited time to run before it hijacks the process to see if signals are thrown (or execute another process's code).
    • It actually hijacks the CPU from your process at the end of the time slice
    • If a system call is made near the end of your time slice, the kernel might just confiscate it early.
  • Signal checking: the kernel will check for recieve signals to your process. It then triggers the signal handler
    • Then Schedules your process for execution (after a system call or the beginning of the next time slice

 

This means any program can suddenly and unexpectedly divert execution to the signal handler!

Something Crazy Signal Handlers

bool adminUser = false;
void signal_handler(int signum) {
    	adminUser = true;
}

int main() {
    signal(SIGUSR1, signal_handler);
    while (1) {
		if (adminUser) printf("Key is abcd1234\n");
	}
}

Something Crazy Signal Handlers

int hold;
void swap(int *x, int *y)
{
	hold = *x; 
    *x = *y;
    *y = tmp;
}
int call_swap()
{
	int x = 1; y = 2;
    swap(x, y);
}
int main()
{
	signal(SIGUSER1, call_swap);
    call_swap();
}

Reentrant function are functions that would operate properly even when interrupted with an instance of themselves.

Are any of these functions reentrant?

No! All of these functions call swap(), which is a non-reentrant function. The signal handler is call_swap too!

Something Crazy Signal Handlers

void swap(int *x, int *y)
{
	int hold = *x; 
    *x = *y;
    *y = tmp;
}
int call_swap()
{
	int x = 1; y = 2;
    swap(x, y);
}
int main()
{
	signal(SIGUSER1, call_swap);
    call_swap();
}

Reentrant function are functions that would operate properly even when interrupted with an instance of themselves.

Are any of these functions reentrant?

Yes they are! All variables are non-global and no global variables are modified by the signal handler

How do we call signals?

Preventing Race Conditions via Signals

  • Do not call non-reentrant functions in your signal handler
    • They might have been internrupted mid-execution when sending your signal
    • Other signals can interrupt the signal handler

Note: malloc and free are non reentrantn

man signal-safety

Takeaways

Protect your code!

  • Good coding practices will go a long way. if you have asynchronous code, make sure it does it intended behavior.
  • Never assume asynchronous code will be executed in the same order.
    • If it needs the same result every time, make sure you are using memory, locking, and re entrant functions properly.

Homework

Exploit a Threaded Race Condition

  • The program has a memory/thread based race condition.
  • You will exploit it so that you can get the flag.
  • Will be released next week.