Lecture 1:

The kernel. Structure. Process schedulers.

slides.com/gerold103/sysprog_eng1

Version: 3

System programming

Education

Lecture plan

  • "Operating system" overview
  • Linux kernel components
    • Process
    • Work with hardware
    • Time
    • File system
    • IPC - interprocess communication
    • Network
    • Users
    • Data structures
    • Process scheduler
  • Homework

Operating system [1]

Kernel

Drivers 

Console

Bootloader

This IS operating system

Player

Browser

Compiler

Email

This IS NOT operating system

OS - minimal subset of programs

Operating system [2]

Kernel

Console

Kernel space

Player

Browser

Compiler

Email

User space

System calls

Kernel - hardware manager

read
write
open
close
fork
...

Interrupts [1]

User

Buttons

Electrical signal

Interrupt

Kernel

Device controller initiates an electrical signal

A button is pressed

Processor got the signal and stopped the current task

A handler in the kernel is called

Handler knows interrupt number, plans further processing

Interrupts [2]

Interrupts

Kernel

Processing ...

Bootloader [1]

Who starts the kernel?

Bootloader

Who starts the bootloader?

Where is BIOS?

God ... what is ROM now?

Bootloader [2]

Why does not BIOS start the kernel directly? Why is a bootloader needed?

It does not know what a filesystem there is, and can not find kernel files in there

Bootloader [3]

1 point

The lamp on a slide means a scored question

Linux kernel

Processes

Hardware

Time

Filesystem

IPC

Network

Users

Data structures

Virtualization

Linux kernel. Process [1]

/**
 * 30.09.2018
 * #include/linux/sched.h
 * 618 lines.
 */
struct task_struct {
	struct thread_info		thread_info;
	long	        		state;
	void				*stack;
	atomic_t			usage;
	unsigned int			cpu;
	int				prio;
	struct mm_struct		*mm;
	int				exit_state;
	int				exit_code;
	int				exit_signal;
	pid_t				pid;
	struct task_struct              *parent;
	struct list_head		children;
	u64				start_time;
	const struct cred		*cred;
	struct files_struct		*files;
	struct thread_struct		thread;
};
long          state;
atomic_t      usage;
unsigned int  cpu;
int           prio;
struct mm_struct *mm;
int              exit_state;
int              exit_code;
int              exit_signal;
pid_t              pid;
struct task_struct *parent;
struct list_head   children;
const struct cred    *cred;
struct files_struct  *files;
pid_t
getpid(void);

pid_t
getppid(void);
vladislav$> cat /proc/sys/kernel/pid_max
32768

Linux kernel. Process [2]

fork()

Ready to run

Running

schedule()

Terminated

do_exit()

preemtion

Waiting an event

Wait an event

The event or signal

Linux kernel. Process [3]

Thread

Process

Linux kernel. Hardware

Linux kernel. Time [1]

Periodical and onetime tasks

Hardware time source

Oscillator

Oscillator inside

Polished crystal

Raw crystal

Oscillator inside

Polished crystal

HZ = #

1s

Too large HZ

Damn ... when is the next nanosecond?

vladislav$> grep 'CONFIG_HZ=' /boot/config-$(uname -r)
CONFIG_HZ=250

Linux kernel. Time [2]

Real Time Clock

Detected 2400.131 MHz processor.
Calibrating delay loop... 4799.56 BogoMIPS

Linux kernel. Time [3]

Pulse Per Second

Source of signal

Atomic clock

Precision - picoseconds

Linux kernel. FS

/**
 * 30.09.2018
 * 33 virtual functions, 149 lines.
 */
struct file_operations {
	loff_t (*llseek) (struct file *, loff_t, int);
	ssize_t (*read) (struct file *, char __user *, size_t, loff_t *);
	ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *);
	int (*mmap) (struct file *, struct vm_area_struct *);
	int (*open) (struct inode *, struct file *);
	int (*flush) (struct file *, fl_owner_t id);
	int (*fsync) (struct file *, loff_t, loff_t, int datasync);
	int (*flock) (struct file *, int, struct file_lock *);
	long (*fallocate)(struct file *file, int mode, loff_t offset,
			  loff_t len);
};

struct super_block {
	struct file_system_type	*s_type;
	const struct super_operations	*s_op;
	int			s_count;
	struct list_head	s_mounts;
};

read(fd)

syscall

struct inode

inode->read()

ext4_read()

Linux kernel. IPC

Mutexes, semaphores

Message queues

Shared memory

Pipe

Domain sockets

Linux kernel. Network

Application

Transport

Internet

Link

Level:

Model IP + TCP/UDP

Linux kernel. Users

/**
 * 30.09.2018.
 * 39 lines.
 */
struct cred {
	kuid_t		uid;		/* real UID of the task */
	kgid_t		gid;		/* real GID of the task */
	kuid_t		suid;		/* saved UID of the task */
	kgid_t		sgid;		/* saved GID of the task */
	kuid_t		euid;		/* effective UID of the task */
	kgid_t		egid;		/* effective GID of the task */
	struct user_struct *user;	/* real user ID subscription */
};

Linux kernel. Data structures [1]. Naive list.

struct unit {
        int hp;
        int damage;
        struct unit *next, *prev;
        char name[0];
};

struct unit_list {
        struct unit *first;
        struct unit *last;
};

void
unit_list_push_back(struct unit_list *list,
                    struct unit *u);

void
unit_list_push_front(struct unit_list *list,
                     struct unit *u);

void
unit_list_remove(struct unit_list *list,
                 struct unit *u);

bool
unit_list_is_empty(const struct unit_list *list);

void
unit_list_create(struct unit_list *list);
struct building {
        int hp;
        struct unit_list units;
        struct building *next, *prev;
        char name[0];
};

struct building_list {
        struct building *first;
        struct building *last;
};

void
building_list_push_back(struct building_list *list,
                        struct building *b);

void
building_list_push_front(struct building_list *list,
                         struct building *b);

void
building_list_remove(struct building_list *list,
                     struct building *b);

bool
building_list_is_empty(const struct building_list *list);

void
building_list_create(struct building_list *list);

Need two lists - code duplication. Can C++ help?

template<class parent>
struct list_element {
        parent *next, *prev;
        char name[0];
};

template<class parent>
struct list {
        parent *first;
        parent *last;
};

void
list_push_back(struct list *list,
               struct list_element *e);

void
list_push_front(struct list *list,
                struct list_element *e);

void
list_remove(struct list *list,
            struct list_element *e);

bool
list_is_empty(const struct list *list);

void
list_create(struct list *list);
struct building : public list_element<struct building> {
        int hp;
        struct unit_list units;
        char name[0];
};

struct unit : public list_element<struct unit> {
        int hp;
        int damage;
        char name[0];
};

Can not store in multiple lists because of inheritance. Only one list_element parent is possible. Lets try STL?

Linux kernel. Data structures [2]. Naive list.

template<class object>
struct list_element {
        object *obj;
        struct list_element<object> *next, *prev;
};

template<class object>
struct list {
        struct list_element<object> *first, *last;
};

void
list_push_back(struct list *list,
               struct object e);

void
list_push_front(struct list *list,
                struct object e);

void
list_remove(struct list *list,
            struct object e);

bool
list_is_empty(const struct list *list);

void
list_create(struct list *list);
struct building {
        int hp;
        struct unit_list units;
        char name[0];
};

struct unit {
        int hp;
        int damage;
        char name[0];
};

Pointer on pointer - double lookup into the memory - twice longer access

Linux kernel. Data structures [3]. Naive list.

struct list_element {
        struct list_element *next, *prev;
};

struct list {
        struct list_element *first, *last;
};

void
list_push_back(struct list *list,
               struct list_element *e);

void
list_push_front(struct list *list,
                struct list_element *e);

void
list_remove(struct list *list,
            struct list_element *e);

bool
list_is_empty(const struct list *list);

void
list_create(struct list *list);
struct unit {
        int hp;
        int damage;
        struct list_element in_army;
        struct list_element in_building;
        char name[0];
};

struct building {
        int hp;
        struct list units;
        struct list_element in_city;
        char name[0];
};

A no-data list. Works with no-data elements.

When a structure wants to participate in a list, it adds the empty list element as an attribute.

Linux kernel. Data structures [4]. Intrusive list.

The list stores pointers on struct unit attributes

struct list army;
struct building house;
struct unit unit;

list_insert(&army, &unit.in_army);
list_insert(&house.units, &unit.in_building);

/* ... */

struct list_element *e;
struct unit *u;

e = list_first(&army);
u = container_of(e, struct unit, in_army);

e = list_first(&house.units);
u = container_of(e, struct unit, in_building);

Knowing an attribute offset, it is possible to get the parent structure pointer

#define container_of(ptr, type, member) ({                      \
        const typeof( ((type *)0)->member ) *__mptr = (ptr);    \
        (type *)( (char *)__mptr - offsetof(type,member) );})

Linux kernel. Data structures [5]. Intrusive list.

Linux kernel. Data structures [6]

struct list_head {
	struct list_head *next, *prev;
};

Kernel list

struct my_struct {
        struct list_head base;
        int a;
        int b;
        const char *c;
};

struct my_struct ms1, ms2;
INIT_LIST_HEAD(&ms1);
list_add(&ms1, &ms2);
struct rb_node {
	unsigned long  __rb_parent_color;
	struct rb_node *rb_right;
	struct rb_node *rb_left;
};

struct rb_root {
	struct rb_node *rb_node;
};

Kernel red-black tree

struct rb_root tree = RB_ROOT;

struct my_struct {
        struct rb_node node;
        int a;
        int b;
        const char *c;
};

struct my_struct ms1, ms2;
rb_insert_color(&tree, &ms1);
rb_insert_color(&tree, &ms2);

When the list element is the first attribute, it is not even necessary to access it directly. Works like C 'inheritance'.

Linux kernel. Scheduler [1]

Cooperative multitasking -

voluntary yield

Preemptive multitasking -

mandatory preemtion

Linux kernel. Scheduler [2]

dumb scheduler

O(1) scheduler

Completely Fair Scheduler

I/O Bound

CPU Bound

Time

Disk

Interactive applications

Network

Transaction processing

Calculations

What are two types of multitasking?

Cooperative and preemptive

Linux kernel. Scheduler [3]

1 point

Linux kernel. Scheduler [4]

int
getpriority(int which, id_t who);

int
setpriority(int which, id_t who, int prio);

int
nice(int inc);
vladislav$> ps -el
UID   PID PRI NI TTY CMD
  0     1  37  0 ??  /sbin/launchd

Real-time priority

Nice

Timeslice / quantum

start

interrupt

nice -- execute a utility with an altered scheduling priority
renice -- alter priority of running processes
chrt -- manipulate the real-time attributes of a process

Linux kernel. Scheduler [5]

Example

Video conversion

Chess with AI

Will end earlier

Low interactivity

Priority

Priority

Cache losses, will end later

Fast reaction

Linux kernel. Scheduler [6]

Time quantum in Linux - relative value

10 мс

10 мс

50 %

50 %

Process can consume time quantum partially, and share the rest with others

Linux kernel. Scheduler [7]

Completely Fair Scheduler

N  - process count,

perfect processor sharing - 1/N -> 0, infinitely often switching

Perfect scheduler 

/**
 * 30.09.2018.
 * 36 lines.
 */
struct sched_entity {
	struct load_weight  load;
	unsigned long       runnable_weight;
	struct rb_node      run_node;
	struct list_head    group_node;
	u64                 exec_start;
	u64                 sum_exec_runtime;
	u64                 vruntime;
	u64                 nr_migrations;
	struct sched_entity *parent;
};
struct load_weight load;
unsigned long      runnable_weight;
u64 exec_start;
u64 sum_exec_runtime;
u64 vruntime;
W_i = \frac{1024}{1.25^{nice}}; P_j = \frac{W_j}{\sum\limits_{i=1}^n W_i}\\ Example\colon\\ nice1 = 1, nice2 = 0\\ weight1 = 820, weight2 = 1024\\ P_1 = 55%, P2 = 45%
/*
 * Pick the next process, keeping these things in mind, in this order:
 * 1) keep things fair between processes/task groups
 * 2) pick the "next" process, since someone really wants that to run
 * 3) pick the "last" process, for cache locality
 * 4) do not run the "skip" process, if something else is available
 */
static struct sched_entity *
pick_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *curr)
{
	struct sched_entity *left = __pick_first_entity(cfs_rq);
        /* .... 47 lines of code. */
}
struct rb_node run_node;

Linux kernel. Scheduler [8]

Summary

Kernel != OS. The operating system is built on top of the kernel.

There are things below the kernel like BIOS, bootloader, hypervisor.

Kernel has many components like drivers, time tracking, process scheduling, networking, own data structure libraries.

Process/thread scheduler makes the system look like all runs in parallel, but in fact the programs just work in turns for very small durations which we don't notice normally. The scheduler's main job is to make those turns fair.

Practice [1]

Merge sort using C/C++ coroutines

Files are stored on disk. They contain ASCII encoded numbers in arbitrary order, unsorted. You need to sort each file and then merge them into a big one. In other words, just do the merge sort.

 

Implement C coroutines, a scheduler, and use them. More information by the link:

github.com/sysprogio/sysprog/tree/master/1

Points: 15 - 25.

Deadline: 2 weeks.

Penalty: -1 for each day after deadline, max -10

Publish your solution on Github and give me the link. Assessment: any way you want - messengers, calls, emails.

Conclusion

Process. Work modes, memory, resources. Interrupts. Communication with the kernel. System calls.


Press on the heart, if like the lecture

Lectures: slides.com/gerold103/decks/sysprog_eng

Next time:

System programming 1

By Vladislav Shpilevoy

System programming 1

The kernel. Process and its representation in the kernel. Process states, lifecycle. Types of process scheduling: preemptive and cooperative. Process schedulers, and IO Bound vs Processor Bound processes, details of scheduling. 'Nice' value and priorities. Timeslice - scheduling atom. CFS scheduler.

  • 2,747

More from Vladislav Shpilevoy