Vladislav Shpilevoy PRO
Database C developer at Tarantool. Backend C++ developer at VirtualMinds.
Lecture 1:
The kernel. Structure. Process schedulers.
Version: 3
System programming
Education
Lecture plan
- "Operating system" overview
- Linux kernel components
- Process
- Work with hardware
- Time
- File system
- IPC - interprocess communication
- Network
- Users
- Data structures
- Process scheduler
- Homework
Operating system [1]
Kernel
Drivers
Console
Bootloader
This IS operating system
Player
Browser
Compiler
This IS NOT operating system
OS - minimal subset of programs
Operating system [2]
Kernel
Console
Kernel space
Player
Browser
Compiler
User space
System calls
Kernel - hardware manager
read write open close fork ...
Interrupts [1]
User
Buttons
Electrical signal
Interrupt
Kernel
Device controller initiates an electrical signal
A button is pressed
Processor got the signal and stopped the current task
A handler in the kernel is called
Handler knows interrupt number, plans further processing
Interrupts [2]
Interrupts
Kernel
Processing ...
Bootloader [1]
Who starts the kernel?
Bootloader
Who starts the bootloader?
Where is BIOS?
God ... what is ROM now?
Bootloader [2]
Why does not BIOS start the kernel directly? Why is a bootloader needed?
It does not know what a filesystem there is, and can not find kernel files in there
Bootloader [3]
1 point
The lamp on a slide means a scored question
Linux kernel
Processes
Hardware
Time
Filesystem
IPC
Network
Users
Data structures
Virtualization
Linux kernel. Process [1]
/**
* 30.09.2018
* #include/linux/sched.h
* 618 lines.
*/
struct task_struct {
struct thread_info thread_info;
long state;
void *stack;
atomic_t usage;
unsigned int cpu;
int prio;
struct mm_struct *mm;
int exit_state;
int exit_code;
int exit_signal;
pid_t pid;
struct task_struct *parent;
struct list_head children;
u64 start_time;
const struct cred *cred;
struct files_struct *files;
struct thread_struct thread;
};
long state;
atomic_t usage;
unsigned int cpu;
int prio;struct mm_struct *mm;
int exit_state;
int exit_code;
int exit_signal;pid_t pid;
struct task_struct *parent;
struct list_head children;const struct cred *cred;
struct files_struct *files;pid_t
getpid(void);
pid_t
getppid(void);vladislav$> cat /proc/sys/kernel/pid_max 32768
Linux kernel. Process [2]
fork()
Ready to run
Running
schedule()
Terminated
do_exit()
preemtion
Waiting an event
Wait an event
The event or signal
Linux kernel. Process [3]
Thread
Process
Linux kernel. Hardware
Linux kernel. Time [1]
Periodical and onetime tasks
Hardware time source
Oscillator
Oscillator inside
Polished crystal
Raw crystal
Oscillator inside
Polished crystal
HZ = #
1s
Too large HZ
Damn ... when is the next nanosecond?
vladislav$> grep 'CONFIG_HZ=' /boot/config-$(uname -r) CONFIG_HZ=250
Linux kernel. Time [2]
Real Time Clock
Detected 2400.131 MHz processor. Calibrating delay loop... 4799.56 BogoMIPS
Linux kernel. Time [3]
Pulse Per Second
Source of signal
Atomic clock
Precision - picoseconds
Linux kernel. FS
/**
* 30.09.2018
* 33 virtual functions, 149 lines.
*/
struct file_operations {
loff_t (*llseek) (struct file *, loff_t, int);
ssize_t (*read) (struct file *, char __user *, size_t, loff_t *);
ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *);
int (*mmap) (struct file *, struct vm_area_struct *);
int (*open) (struct inode *, struct file *);
int (*flush) (struct file *, fl_owner_t id);
int (*fsync) (struct file *, loff_t, loff_t, int datasync);
int (*flock) (struct file *, int, struct file_lock *);
long (*fallocate)(struct file *file, int mode, loff_t offset,
loff_t len);
};
struct super_block {
struct file_system_type *s_type;
const struct super_operations *s_op;
int s_count;
struct list_head s_mounts;
};read(fd)
syscall
struct inode
inode->read()
ext4_read()
Linux kernel. IPC
Mutexes, semaphores
Message queues
Shared memory
Pipe
Domain sockets
Linux kernel. Network
Application
Transport
Internet
Link
Level:
Model IP + TCP/UDP
Linux kernel. Users
/**
* 30.09.2018.
* 39 lines.
*/
struct cred {
kuid_t uid; /* real UID of the task */
kgid_t gid; /* real GID of the task */
kuid_t suid; /* saved UID of the task */
kgid_t sgid; /* saved GID of the task */
kuid_t euid; /* effective UID of the task */
kgid_t egid; /* effective GID of the task */
struct user_struct *user; /* real user ID subscription */
};Linux kernel. Data structures [1]. Naive list.
struct unit {
int hp;
int damage;
struct unit *next, *prev;
char name[0];
};
struct unit_list {
struct unit *first;
struct unit *last;
};
void
unit_list_push_back(struct unit_list *list,
struct unit *u);
void
unit_list_push_front(struct unit_list *list,
struct unit *u);
void
unit_list_remove(struct unit_list *list,
struct unit *u);
bool
unit_list_is_empty(const struct unit_list *list);
void
unit_list_create(struct unit_list *list);struct building {
int hp;
struct unit_list units;
struct building *next, *prev;
char name[0];
};
struct building_list {
struct building *first;
struct building *last;
};
void
building_list_push_back(struct building_list *list,
struct building *b);
void
building_list_push_front(struct building_list *list,
struct building *b);
void
building_list_remove(struct building_list *list,
struct building *b);
bool
building_list_is_empty(const struct building_list *list);
void
building_list_create(struct building_list *list);Need two lists - code duplication. Can C++ help?
template<class parent>
struct list_element {
parent *next, *prev;
char name[0];
};
template<class parent>
struct list {
parent *first;
parent *last;
};
void
list_push_back(struct list *list,
struct list_element *e);
void
list_push_front(struct list *list,
struct list_element *e);
void
list_remove(struct list *list,
struct list_element *e);
bool
list_is_empty(const struct list *list);
void
list_create(struct list *list);struct building : public list_element<struct building> {
int hp;
struct unit_list units;
char name[0];
};
struct unit : public list_element<struct unit> {
int hp;
int damage;
char name[0];
};Can not store in multiple lists because of inheritance. Only one list_element parent is possible. Lets try STL?
Linux kernel. Data structures [2]. Naive list.
template<class object>
struct list_element {
object *obj;
struct list_element<object> *next, *prev;
};
template<class object>
struct list {
struct list_element<object> *first, *last;
};
void
list_push_back(struct list *list,
struct object e);
void
list_push_front(struct list *list,
struct object e);
void
list_remove(struct list *list,
struct object e);
bool
list_is_empty(const struct list *list);
void
list_create(struct list *list);struct building {
int hp;
struct unit_list units;
char name[0];
};
struct unit {
int hp;
int damage;
char name[0];
};Pointer on pointer - double lookup into the memory - twice longer access
Linux kernel. Data structures [3]. Naive list.
struct list_element {
struct list_element *next, *prev;
};
struct list {
struct list_element *first, *last;
};
void
list_push_back(struct list *list,
struct list_element *e);
void
list_push_front(struct list *list,
struct list_element *e);
void
list_remove(struct list *list,
struct list_element *e);
bool
list_is_empty(const struct list *list);
void
list_create(struct list *list);struct unit {
int hp;
int damage;
struct list_element in_army;
struct list_element in_building;
char name[0];
};
struct building {
int hp;
struct list units;
struct list_element in_city;
char name[0];
};
A no-data list. Works with no-data elements.
When a structure wants to participate in a list, it adds the empty list element as an attribute.
Linux kernel. Data structures [4]. Intrusive list.
The list stores pointers on struct unit attributes
struct list army;
struct building house;
struct unit unit;
list_insert(&army, &unit.in_army);
list_insert(&house.units, &unit.in_building);
/* ... */
struct list_element *e;
struct unit *u;
e = list_first(&army);
u = container_of(e, struct unit, in_army);
e = list_first(&house.units);
u = container_of(e, struct unit, in_building);Knowing an attribute offset, it is possible to get the parent structure pointer
#define container_of(ptr, type, member) ({ \
const typeof( ((type *)0)->member ) *__mptr = (ptr); \
(type *)( (char *)__mptr - offsetof(type,member) );})Linux kernel. Data structures [5]. Intrusive list.
Linux kernel. Data structures [6]
struct list_head {
struct list_head *next, *prev;
};Kernel list
struct my_struct {
struct list_head base;
int a;
int b;
const char *c;
};
struct my_struct ms1, ms2;
INIT_LIST_HEAD(&ms1);
list_add(&ms1, &ms2);struct rb_node {
unsigned long __rb_parent_color;
struct rb_node *rb_right;
struct rb_node *rb_left;
};
struct rb_root {
struct rb_node *rb_node;
};Kernel red-black tree
struct rb_root tree = RB_ROOT;
struct my_struct {
struct rb_node node;
int a;
int b;
const char *c;
};
struct my_struct ms1, ms2;
rb_insert_color(&tree, &ms1);
rb_insert_color(&tree, &ms2);When the list element is the first attribute, it is not even necessary to access it directly. Works like C 'inheritance'.
Linux kernel. Scheduler [1]
Cooperative multitasking -
voluntary yield
Preemptive multitasking -
mandatory preemtion
Linux kernel. Scheduler [2]
dumb scheduler
O(1) scheduler
Completely Fair Scheduler
I/O Bound
CPU Bound
Time
Disk
Interactive applications
Network
Transaction processing
Calculations
What are two types of multitasking?
Cooperative and preemptive
Linux kernel. Scheduler [3]
1 point
Linux kernel. Scheduler [4]
int
getpriority(int which, id_t who);
int
setpriority(int which, id_t who, int prio);
int
nice(int inc);vladislav$> ps -el
UID PID PRI NI TTY CMD
0 1 37 0 ?? /sbin/launchd
Real-time priority
Nice
Timeslice / quantum
start
interrupt
nice -- execute a utility with an altered scheduling priority renice -- alter priority of running processes chrt -- manipulate the real-time attributes of a process
Linux kernel. Scheduler [5]
Example
Video conversion
Chess with AI
Will end earlier
Low interactivity
Priority
Priority
Cache losses, will end later
Fast reaction
Linux kernel. Scheduler [6]
Time quantum in Linux - relative value
10 мс
10 мс
50 %
50 %
Process can consume time quantum partially, and share the rest with others
Linux kernel. Scheduler [7]
Completely Fair Scheduler
N - process count,
perfect processor sharing - 1/N -> 0, infinitely often switching
Perfect scheduler
/**
* 30.09.2018.
* 36 lines.
*/
struct sched_entity {
struct load_weight load;
unsigned long runnable_weight;
struct rb_node run_node;
struct list_head group_node;
u64 exec_start;
u64 sum_exec_runtime;
u64 vruntime;
u64 nr_migrations;
struct sched_entity *parent;
};struct load_weight load;
unsigned long runnable_weight;u64 exec_start;
u64 sum_exec_runtime;
u64 vruntime;W_i = \frac{1024}{1.25^{nice}}; P_j = \frac{W_j}{\sum\limits_{i=1}^n W_i}\\
Example\colon\\
nice1 = 1, nice2 = 0\\
weight1 = 820, weight2 = 1024\\
P_1 = 55%, P2 = 45%
/*
* Pick the next process, keeping these things in mind, in this order:
* 1) keep things fair between processes/task groups
* 2) pick the "next" process, since someone really wants that to run
* 3) pick the "last" process, for cache locality
* 4) do not run the "skip" process, if something else is available
*/
static struct sched_entity *
pick_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *curr)
{
struct sched_entity *left = __pick_first_entity(cfs_rq);
/* .... 47 lines of code. */
}struct rb_node run_node;Linux kernel. Scheduler [8]
Summary
Kernel != OS. The operating system is built on top of the kernel.
There are things below the kernel like BIOS, bootloader, hypervisor.
Kernel has many components like drivers, time tracking, process scheduling, networking, own data structure libraries.
Process/thread scheduler makes the system look like all runs in parallel, but in fact the programs just work in turns for very small durations which we don't notice normally. The scheduler's main job is to make those turns fair.
Practice [1]
Merge sort using C/C++ coroutines
Files are stored on disk. They contain ASCII encoded numbers in arbitrary order, unsorted. You need to sort each file and then merge them into a big one. In other words, just do the merge sort.
Implement C coroutines, a scheduler, and use them. More information by the link:
Points: 15 - 25.
Deadline: 2 weeks.
Penalty: -1 for each day after deadline, max -10
Publish your solution on Github and give me the link. Assessment: any way you want - messengers, calls, emails.
Conclusion
Process. Work modes, memory, resources. Interrupts. Communication with the kernel. System calls.
Press on the heart, if like the lecture
Lectures: slides.com/gerold103/decks/sysprog_eng
Next time:
System programming 1
By Vladislav Shpilevoy
System programming 1
The kernel. Process and its representation in the kernel. Process states, lifecycle. Types of process scheduling: preemptive and cooperative. Process schedulers, and IO Bound vs Processor Bound processes, details of scheduling. 'Nice' value and priorities. Timeslice - scheduling atom. CFS scheduler.
