Processes and Resource Virtualization in Operating Systems

Processes David Ferry, Chris Gill, Brian Kocoloski, Marion Sudvarg, James Orr CSE 422S - Operating Systems Organization Washington University in St. Louis St. Louis, MO 63143 1

Resource Virtualization What resources do you use when you write programs? The processor(s) Memory Files Data (global variables, things like hardcoded strings) What do you not have to worry about generally? Existence of other processes State of hardware resources CSE 422S Operating Systems Organization 2

Resource Virtualization The operating system virtualizes physical hardware resources into virtual resources in the form of processes Physical Memory Physical Processors CSE 422S Operating Systems Organization 3

Resource Virtualization Process 2 Process 1 Processor Virtualization: Two (or more) processes may share the same set of physical CPUs How to share them? CPU scheduling Memory Virtualization: Translate private memory addresses to physical addresses Virtual memory Physical Memory Physical Processors CSE 422S Operating Systems Organization 4

Source Code -> Program -> Process Generate program $ gcc program.c o a.out Create process ./a.out CSE 422S Operating Systems Organization 5

Processes in Linux Fundamental process abstraction: Original: Each process behaves as though it has total control over the system Modern: Threads still execute as though they monopolize the system, but the process abstraction also facilitates multi-threading, signals, and inter- process communication More features: Scheduling Virtual Memory Process accounting Signals Process groups Synchronization Multi-threading Shared memory Sockets etc. CSE 422S Operating Systems Organization 6

Tasks in Linux A task/process is an execution stack together with data that describes the process state. (also describes threads) User side: Kernel side: User Stack Two stacks: Kernel mode User mode Kernel stack .bss thread_info .data Data: thread_info: small, efficient task_struct: large, statically allocated by slab allocator, points to other data structs task_struct .text CSE 422S Operating Systems Organization 7

The fork() System Call Creates a new process by duplicating the calling process Duplicates file descriptors, signal handlers, address space, namespace, etc. Uses copy-on-write (COW), a lazy optimization of memory copy for new process Example: #include <sys/types.h> #include <unistd.h> pid_t pid = fork(); if (pid > 0) printf ("I am the parent of pid=%d!\n", pid); else if (!pid) printf ("I am the child!\n"); else if (pid == -1) perror ("fork"); Cannot predict whether parent or child will run first! 8 CSE 422S Operating Systems Organization

The exec() System Call Family Loads and executes a new program image into an already running process Often called after fork() creates a new process Returns -1 on error, otherwise jumps to entry point of new program and does not return Keeps pid, parent pid, priority, owning user and group Resets most attributes, e.g.: signals, memory locks, thread attributes, process statistics, address space 9 CSE 422S Operating Systems Organization

Examples of exec() Several versions: execl, execlp, execle, execv, execvp, execve l versus v: arguments provided via list or vector p: user s path is searched for the specified file e: A new environment is supplied #include <unistd.h> //Edit file /home/pi/program.c with arguments as list int ret = execlp ("vi", "vi", "/home/pi/program.c", NULL); if (ret == -1) perror ("execlp"); //Edit file /home/pi/program.c with arguments as vector const char * args[] = { "vi", "/home/pi/program.c", NULL } int ret = execvp ("vi", args); if (ret == -1) perror ("execvp"); 10 CSE 422S Operating Systems Organization

Threads Process 1 A process may utilize multiple threadsto divide work and enable intra-task parallelism Physical Memory Physical Processors CSE 422S Operating Systems Organization 11

One Size Fits All Processes, threads, and kernel threads are all implemented with the same data structures (thread_info and task_struct) and functions. Threads are like user processes but they share their parent s address space. Kernel threads don t have a user-side address space (i.e. a task_struct smm_struct pointer is NULL) and not all data values are used. CSE 422S Operating Systems Organization 12

Processes in the kernel: struct task_struct CSE 422S Operating Systems Organization 13

Possible states of a process CSE 422S Operating Systems Organization 14

Threads, Processes, and Kernel Threads Address space thread_info Kernel level Kernel stack .bss (static variables) .text (executable code) .data (heap) User level User stack CSE 422S Operating Systems Organization 15

Threads, Processes, and Kernel Threads <set stack pointer to kernel address> <set page table to kernel memory> Address space thread_info Kernel level Kernel stack .bss (static variables) .text (executable code) .data (heap) User level main() syscall() User stack CSE 422S Operating Systems Organization 16

Threads, Processes, and Kernel Threads // current is a user-level process Address space struct task_struct * task = current; struct mm_struct * mm = current->mm; thread_info Kernel level Kernel stack .bss (static variables) .text (executable code) .data (heap) User level User stack CSE 422S Operating Systems Organization 17

Threads, Processes, and Kernel Threads // current is a kernel thread // (recall, look for processes named // k in ps aux ) Address space thread_info struct task_struct * task = current; struct mm_struct * mm = current->mm; Kernel level Kernel stack User level NULL CSE 422S Operating Systems Organization 18

Threads, Processes, and Kernel Threads struct task_struct * task1; struct task_struct * task2; // fetch task_structs for 2 threads of the same process task1->mm; task2->mm Task 2 Task 1 .bss (static variables) .text (executable code) .data (heap) User level stack stack Stacks are specific per-thread CSE 422S Operating Systems Organization 19

Creating a New Process: fork() Kernel stack Kernel stack Creates a new process by duplicating the calling process Initialize per-PID data, e.g.: PID, real_parent, statistics Copies file handles, signal handlers, process address space, etc. Resources shared via page-level lazy copy-on-write (COW)* fork() is a library function that passes appropriate flags to clone() system call *E.g. Every process in the system shares the same copy of the C library thread_info thread_info task_struct task_struct ptr* 0x1 ptr* 0x2 ptr* 0x3 ptr* 0x1 ptr* 0x2 ptr* 0x3 User Stack User Stack .bss .bss .data .data .text .text CSE 422S Operating Systems Organization 20

Creating a New Thread: clone() Kernel stack Kernel stack Like fork() but with more control Can be used to create new threads Duplicates the calling process and initializes per-PID data Copying of file handles, signal handlers, process address space, etc. is optional In Linux terminology: Threads are Processes that share parent address space thread_info thread_info task_struct task_struct ptr* 0x1 ptr* 0x2 ptr* 0x3 ptr* 0x1 ptr* 0x2 ptr* 0x3 User Stack .bss .data .text CSE 422S Operating Systems Organization 21

Kernel Source Process creation _do_fork() get_task_pid() (kernel/fork.c) (kernel/pid.c) Kernel thread creation kernel_thread() (kernel/fork.c) Process copying copy_process() (kernel/fork.c) CSE 422S Operating Systems Organization 22

Process Termination: The exit(int status) Library Call Calls exit handlers Flushes stdio buffers Calls __exit(int status) system call Automatically linked into your program int main(int argc, char * argv[]) { //Do something return status; } int main(int argc, char * argv[]) { //Do something exit(status); } Equivalent Do not use exit() from inside a function rather than providing a clean termination path CSE 422S Operating Systems Organization 23

The __exit(int status) System Call Performs process termination and cleanup Does not free process task_struct This allows parent process to obtain status via wait*() functions CSE 422S Operating Systems Organization 24

Exit Handlers Run whenever exit() is called E.g., any return from main() Useful for ensuring resource cleanup along all termination paths Free dynamically allocated memory Close files and sockets Two methods to register: int atexit(void (*func)(void)); Simple, portable int on_exit(void (*func)(int, void*), void *arg); Receives exit status and additional argument CSE 422S Operating Systems Organization 25

Todays Studio Explore user-space utilities for process discovery and creation Explore kernel-level data structures used for process management struct task_struct More experience with kernel modules CSE 422S Operating Systems Organization 26

Error Checking in Kernel Code By convention, user space functions return 0 for success and (usually negative) non-zero values to indicate different errors that may occur Kernel code may return (virtual) pointers, which could be either 0 or non-zero on success Linux exploits unused (upper) range of virtual memory addresses to encode error values include/linux/err.h provides a useful error checking macro and type conversion functions 27 CSE 422S Operating Systems Organization

Error Checking in Kernel Code The macro is illustrative of how all this works /* from Linux 5.10.17 include/linux/err.h */ #define MAX_ERRNO 4095 #define IS_ERR_VALUE(x) unlikely((unsigned long)(void *)(x) >= (unsigned long)-MAX_ERRNO) compiler optimization hint (branch prediction) Examples of how the functions can be used /* from https://www.kernel.org/doc/html/v4.17/crypto/api-samples.html */ /* returns an integer with error value that was in a pointer value */ skcipher = crypto_alloc_skcipher("cbc-aes-aesni", 0, 0); if (IS_ERR(skcipher)) { pr_info("could not allocate skcipher handle\n"); return PTR_ERR(skcipher); } ... /* returns a pointer that contains an error code if alloc fails */ sdesc = kmalloc(size, GFP_KERNEL); if (!sdesc) return ERR_PTR(-ENOMEM); 28 CSE 422S Operating Systems Organization