C++ / a working model

OSTEP / 60 CHAPTERS

Virtualization, concurrency, persistence.

Original notes for every chapter of Operating Systems: Three Easy Pieces. Official PDFs stay on the authors’ site; this site does not host the book.

https://pages.cs.wisc.edu/~remzi/OSTEP/

01

OSTEP / C11

Preface

Inspired by Feynman's lecture notes, the book is organized around virtualization, concurrency and persistence. It describes problem-first chapters, timelines, dialogues and other devices, plus free access, typical course pacing, and practical notes for both instructors and students.

02

OSTEP / C11

A Dialogue on the Book

An opening professor-student exchange explains the title's nod to physics lecture notes, frames operating systems around virtualization, concurrency and persistence, recommends combining lectures with rereading notes and real coding projects, and clarifies that the dialogues exist to step outside linear text and think together.

03

OSTEP / C11

Introduction to Operating Systems

This chapter outlines how an operating system virtualizes limited physical hardware into convenient abstractions while serving as both a resource manager and a standard interface provider.

04

OSTEP / C11

A Dialogue on Virtualization

A light professor-student conversation that reveals how an operating system turns one physical CPU into many virtual CPUs so every program believes it owns the processor exclusively.

05

OSTEP / C11

The Abstraction: The Process

This chapter introduces the process as the OS abstraction of a running program, explains CPU virtualization, the composition of process state, and how processes are created from programs.

06

OSTEP / C11

Interlude: Process API

This interlude presents the core UNIX interfaces for process creation and control. fork duplicates the calling process so parent and child resume from the same return point with different values, wait lets a parent block until a child finishes, and the exec family replaces the entire process image with a new executable. Together they form a compact yet highly expressive API.

07

OSTEP / C11

Mechanism: Limited Direct Execution

The OS virtualizes the CPU efficiently with limited direct execution: user programs run natively on the processor for speed while hardware mode switches and trap instructions keep the kernel firmly in control.

08

OSTEP / C11

Scheduling: Introduction

This chapter builds a basic framework for thinking about scheduling policies by first listing simplifying workload assumptions, then introducing turnaround time as the core performance metric, and finally examining two early algorithms—FIFO and shortest-job-first—along with their limits.

09

OSTEP / C11

Scheduling: The Multi-Level Feedback Queue

The Multi-Level Feedback Queue (MLFQ) dynamically tunes job priorities according to observed runtime behavior, improving both interactive response times and long-job turnaround without a priori knowledge of job lengths.

10

OSTEP / C11

Scheduling: Proportional Share

This chapter presents a scheduling approach that divides processor time according to assigned ratios. Jobs receive tickets and a random draw selects the next runner; several ticket-handling tricks are described along with why the method is simple to code yet only probabilistically fair.

11

OSTEP / C11

Multiprocessor Scheduling (Advanced)

As multicore chips become ubiquitous, the OS must assign threads across several CPUs. This chapter originally explains the coherence problems created by per-core caches, why locks remain necessary even with hardware help, and how a scheduler can exploit cache affinity to cut migration costs.

12

OSTEP / C11

Summary Dialogue on CPU Virtualization

This conversation recaps how an OS virtualizes the CPU through hardware mechanisms and cautious policies, stressing retained control, scheduling trade-offs, and engineering realities in real systems.

13

OSTEP / C11

A Dialogue on Memory Virtualization

The conversation makes clear that CPU virtualization is only the beginning; memory virtualization is the real challenge. Every address a user program produces is virtual. With hardware help the OS translates those addresses into physical ones, giving each process the illusion of a large, private, contiguous memory. The illusion simplifies programming and also isolates processes from one another. Later chapters start with base-and-bounds and then add TLBs and multi-level page tables.

14

OSTEP / C11

The Abstraction: Address Spaces

This chapter shows how an operating system abstracts physical RAM into a private address space per process so that many programs can reside in memory at once, run safely, and still appear to own a large contiguous region starting at address zero.

15

OSTEP / C11

Interlude: Memory API

C programs rely on automatic stack allocation together with explicit heap requests to control data lifetime. Mastering the pairing of malloc and free plus typical misuse patterns is essential for robust software.

16

OSTEP / C11

Mechanism: Address Translation

This chapter presents hardware address translation as the mechanism that lets an OS virtualize memory efficiently and flexibly. Hardware maps every virtual address to a physical one on the fly, giving each process the illusion of a private contiguous space starting at zero while the OS retains isolation and protection.

17

OSTEP / C11

Segmentation

Segmentation equips the MMU with a distinct base-and-limit pair per logical region so that code, heap and stack can reside in separate physical holes and unused virtual gaps occupy no RAM.

18

OSTEP / C11

Free-Space Management

This chapter explains the core difficulties allocators face with variable-sized free regions, focusing on why external fragmentation occurs and how splitting, coalescing, and header recording help keep usable contiguous space available.

19

OSTEP / C11

Paging: Introduction

Paging carves both virtual address spaces and physical memory into identical fixed-size pages and frames, eliminating external fragmentation. A private page table per process records the mappings so hardware can replace a virtual page number with a physical frame number.

20

OSTEP / C11

Paging: Faster Translations (TLBs)

Paging would be too slow without a hardware cache of translations. The TLB exploits locality so that most address translations complete in a few cycles instead of requiring a memory access.

21

OSTEP / C11

Paging: Smaller Tables

Linear page tables devour huge amounts of RAM. This chapter uses fresh wording to explore larger pages and a paging-plus-segmentation hybrid that shrink the tables, while highlighting the internal-fragmentation and extra hardware checks they introduce.

22

OSTEP / C11

Beyond Physical Memory: Mechanisms

The OS employs slower secondary storage as swap area plus a present bit in each PTE so that many large address spaces can coexist even when they do not fit in RAM. Missing pages raise faults that a software handler resolves by transferring data from disk.

23

OSTEP / C11

Beyond Physical Memory: Policies

When physical memory is scarce the operating system must select pages to send to disk. This chapter discusses how to design replacement policies that reduce page faults, using the unrealizable optimal algorithm as a benchmark while examining the simple FIFO method and its limitations.

24

OSTEP / C11

Complete VM Systems

Using VAX/VMS and Linux as concrete examples, this chapter shows how page-table designs, TLB handling, page replacement and extra features for performance, security and functionality are combined into a complete virtual-memory system that works from embedded devices to supercomputers.

25

OSTEP / C11

Summary Dialogue on Memory Virtualization

A student-professor recap that builds a working mental model of virtual memory: programs see only virtual addresses, the TLB makes translation practical, page-table designs must flexibly support sparse spaces, and swapping exposes real hardware limits. The aim is independent diagnosis of unexpected system behavior.

26

OSTEP / C11

A Dialogue on Concurrency

A professor and student introduce concurrency via the everyday scene of many people grabbing peaches from a table, showing that uncoordinated simultaneous grabs cause conflicts while lining up guarantees fairness at the cost of speed. The ideal solution must be both correct and fast. The analogy then maps onto multi-threaded programs: threads act as independent agents and shared memory locations resemble the peaches, so access must be coordinated. OS courses cover this topic because the kernel both supplies synchronization primitives to applications and, as the original concurrent program, must itself manage internal data with extreme care.

27

OSTEP / C11

Concurrency: An Introduction

This chapter presents threads from a fresh angle: a single process may contain several independent flows of execution that all share one address space. Every thread carries its own program counter and registers, so a switch among them leaves the page table untouched. Each thread also owns a private stack. Threads exist mainly so that multiple cores can work in true parallel and so that a program can keep making progress while some of its threads wait for I/O.

28

OSTEP / C11

Interlude: Thread API

This interlude surveys the core POSIX thread-library calls used to launch new flows of execution, wait for them to finish, and protect shared data with mutexes. The interfaces balance ease of use with flexibility; later chapters expand on locks and condition variables through many examples.

29

OSTEP / C11

Locks

Locks let programmers protect critical sections so that updates to shared data occur atomically, avoiding race conditions among concurrent threads.

30

OSTEP / C11

Lock-based Concurrent Data Structures

This chapter examines how locks can be added to ordinary data structures to achieve thread safety, analyzes the performance limitations of naive locking, and presents approximation techniques that improve scalability, with counters serving as the running example.

31

OSTEP / C11

Condition Variables

Condition variables let a thread sleep efficiently until a shared condition becomes true, avoiding useless spinning. They must be used together with a mutex: wait atomically drops the lock and sleeps, signal wakes a waiter, and an explicit state variable prevents lost signals.

32

OSTEP / C11

Semaphores

A semaphore coordinates threads with an integer counter plus blocking and wakeup primitives. Its initial value decides whether it behaves as a mutex or an event notifier. This chapter uses original examples to show wait/post semantics, binary usage, and parent-child ordering, plus a compilable C demo.

33

OSTEP / C11

Common Concurrency Problems

This chapter analyzes recurring defect patterns in concurrent software, highlighting the distinction between deadlocks and non-deadlock issues, the latter mainly involving failed atomicity assumptions and reversed execution orders. Synchronization primitives can effectively mitigate these risks and improve the reliability of multithreaded code.

34

OSTEP / C11

Event-based Concurrency (Advanced)

This chapter presents a way to build concurrent servers without threads. The program centers on an event loop that processes one arriving event at a time, giving the developer full scheduling control and removing any need for locks. The essential restriction is that handlers must never perform operations that can block.

35

OSTEP / C11

Summary Dialogue on Concurrency

This summary explores the mental challenges of concurrent execution and stresses writing reliable concurrent programs through simplified designs and proven patterns.

36

OSTEP / C11

Dialogue on the Topic of Persistence

This original dialogue employs fresh analogies to explain how operating systems keep information alive after shutdowns or failures, revealing the extra effort and design intrigue behind persistent storage.

37

OSTEP / C11

I/O Devices

This chapter explains how an operating system incorporates input/output devices into the overall machine, covering hierarchical bus layouts, the registers a device exposes, the polling-based request protocol, and the use of interrupts so computation can overlap with device work.

38

OSTEP / C11

Hard Disk Drives

This chapter explains how hard disks persist data as a sector array, the platter-track-head geometry, and how seek plus rotational delay dominate access cost. Schedulers reorder requests to raise effective throughput.

39

OSTEP / C11

Redundant Arrays of Inexpensive Disks (RAID)

RAID organizes multiple inexpensive disks into an array that simultaneously improves capacity, throughput and fault tolerance while remaining completely transparent to the host. This chapter covers the external interface, the assumed fault model, the three evaluation axes and the simplest striping organization.

40

OSTEP / C11

Interlude: Files and Directories

This chapter presents an original view of how an operating system virtualizes persistent devices as two complementary abstractions—files and directories—and how the classic UNIX interface of open, read, write and unlink hides inode numbers behind human-readable path names.

41

OSTEP / C11

File System Implementation

This chapter uses a minimal vsfs example to show how core on-disk structures can be designed entirely in software to manage files, focusing on the division of labor among the superblock, bitmaps, inode table and data region, plus how system calls map onto those structures.

42

OSTEP / C11

Locality and The Fast File System

The original UNIX file system treated the disk as random-access memory: inodes sat far from data, free space fragmented, and 512-byte blocks forced extra seeks, yielding only a few percent of possible bandwidth. The Fast File System introduced cylinder (now block) groups plus simple locality heuristics that co-locate related files and metadata, turning long seeks into short ones and restoring sequential transfer rates.

43

OSTEP / C11

Crash Consistency: FSCK and Journaling

File systems keep inodes, bitmaps and data blocks on disk; one operation often needs several writes. A crash can leave a partial update and inconsistency. This chapter originally explains fsck-style later scanning plus journaling (write-ahead logging) that recovers quickly with modest extra work.

44

OSTEP / C11

Log-structured File Systems

Log-structured file systems buffer every update including metadata in memory then flush large sequential segments onto free disk space, matching write-dominated traffic from bigger caches, nearing peak bandwidth and easing RAID small-write costs.

45

OSTEP / C11

Flash-based SSDs

This chapter introduces how NAND flash forms modern solid-state drives, focusing on the physical requirement to erase an entire block before programming any page, cell wear-out, and how these traits shape storage-system design.

46

OSTEP / C11

Data Integrity and Protection

This chapter studies how storage systems keep written data unchanged despite imperfect hardware. It covers partial disk faults (latent sector errors and silent corruptions), redundancy-based recovery, and checksum detection, highlighting the space-time trade-offs involved.

47

OSTEP / C11

Summary Dialogue on Persistence

This dialogue recaps the core difficulties of persistent storage: data must survive crashes, updates require reliable recovery, plus disk scheduling, RAID, checksums and device-aware file-system designs. The same ideas remain useful with flash.

48

OSTEP / C11

A Dialogue on Distribution

A professor-student conversation introduces the core idea of distributed systems, highlights unreliability across machines, and sketches replication plus retry as ways to stay available, setting the stage for later distributed-file-system material.

49

OSTEP / C11

Distributed Systems

Distributed systems assemble many machines over a network into one service. Individual hosts, disks and links fail, yet redundancy can make the whole appear almost never to fail. Communication is inherently lossy, so checksums, acknowledgements and retransmissions are required to build usable protocols.

50

OSTEP / C11

Network File System (NFS)

This chapter introduces the early successful distributed file system NFS, focusing on how the client-server model enables data sharing and transparent access, and how NFSv2 achieves instant recovery after server crashes via a completely stateless protocol.

51

OSTEP / C11

Andrew File System (AFS)

This chapter explores how AFS attains high scalability through whole-file caching on client local disks plus server-driven callbacks that cut server load, contrasting the approach with NFS polling and tracing protocol changes across versions.

52

OSTEP / C11

Summary Dialogue on Distribution

This summary uses a light dialogue to recap core ideas from distributed systems. Component failures are inevitable, yet deploying many disks or machines can conceal most of them. Simple mechanisms such as retries handle transient problems effectively. The exact bits exchanged in protocols govern both failure response and scalability. The conversation closes humorously, underscoring that learning never truly ends.

53

OSTEP / C11

A Dialogue on Security

This dialogue introduces operating system security, highlighting differences from reliability due to intentional adversaries. It covers the need to protect confidential, intact, and available resources, plus the challenges of dealing with intelligent persistent attackers.

54

OSTEP / C11

A Few Words About Security

This chapter introduces the importance of operating system security, explaining why the OS as the foundation of all computing must be protected, and discusses the challenges in achieving security.

55

OSTEP / C11

Authentication

An operating system must reliably identify the principal behind every process before it can enforce security policy. This chapter examines how identities are attached to processes through inheritance and through the initial binding that occurs at login.

56

OSTEP / C11

Access Control

This chapter shows how an operating system converts a security policy into a concrete allow-or-deny verdict for every resource request, emphasizing the duties of the reference monitor, the subject-object model, and the efficiency-versus-flexibility trade-offs of access-control lists versus capabilities.

57

OSTEP / C11

Cryptography

An operating system cannot protect data after it leaves the hardware the kernel actually controls. Cryptography uses a key to turn plaintext into ciphertext so that later possession of the bits still yields neither meaning nor useful alteration. This chapter presents the symmetric-encryption model, the decisive role of key secrecy, and how a hash supplies integrity checking.

58

OSTEP / C11

Distributed System Security

This chapter examines the distinctive security problems of distributed systems, where a single operating system cannot govern remote hosts or the intervening network. Authentication by passwords or public keys, together with certificates issued by trusted authorities, supplies the practical tools for establishing identity and protecting communication.

59

OSTEP / C11

Virtual Machines

A virtual machine monitor inserts a transparent abstraction layer between hardware and operating systems so multiple guest OSes can run concurrently, each believing it owns the machine. This appendix covers historical background, contemporary uses, and key mechanisms for CPU virtualization.

60

OSTEP / C11

Monitors

This appendix presents monitors as a construct that packages shared data with its access operations into one module while automatically supplying mutual exclusion, uses condition variables for wait-and-signal, and contrasts Hoare versus Mesa semantics as they appear in real implementations.