Andrew File System (AFS)

Core Goals for a Scalable Distributed File System

AFS originated at Carnegie Mellon University in the 1980s with the explicit aim of letting one server handle the largest possible client population. Designers treated readily explainable cache consistency as a first-class requirement instead of burying it behind timeout knobs.

How Whole-File Caching Cuts Network Traffic

On open the client pulls the complete file onto its local disk; thereafter every read and write stays inside the local file system with zero network activity. Only a close of a dirty file sends the entire contents back. The contrast with in-memory block caching is immediate.

Scalability Limits Exposed by the First Protocol

The early protocol handed the server full pathnames, so CPU time vanished into directory walks. Clients also issued frequent validity probes, leaving the server mostly answering “unchanged.” Consequently a single server could support only about twenty clients.

Protocol Advances via Callbacks and File Identifiers

Version two lets the server promise “I will tell you if this file changes,” eliminating periodic probes. Pathnames give way to FIDs composed of volume, file and uniquifier IDs; clients themselves cache directory pieces step by step, slashing server-side path resolution.

Pitfalls

• Confusing AFS local-disk whole-file caching with NFS-style in-memory block caching
• Ignoring the extra server memory needed to track a callback for every cached copy
• Underestimating how letting the client walk the path itself relieves server CPU

Minimum C11 · complete program · Download .c

#include <stdio.h>
#include <string.h>
#include <stdbool.h>

typedef struct {
    char name[64];
    char data[128];
    int ver;
} AFSFile;

AFSFile server_files[2];
int num_files = 2;

typedef struct {
    char cached_name[64];
    char cached_data[128];
    int cached_ver;
    bool has_callback;
    bool is_dirty;
} VenusCache;

void init_server(void) {
    strcpy(server_files[0].name, "notes.txt");
    strcpy(server_files[0].data, "Hello AFS world");
    server_files[0].ver = 1;
    strcpy(server_files[1].name, "data.bin");
    strcpy(server_files[1].data, "binary stuff");
    server_files[1].ver = 1;
}

void fetch_file(VenusCache *cache, const char *fname) {
    for (int i = 0; i < num_files; i++) {
        if (strcmp(server_files[i].name, fname) == 0) {
            strcpy(cache->cached_name, fname);
            strcpy(cache->cached_data, server_files[i].data);
            cache->cached_ver = server_files[i].ver;
            cache->has_callback = true;
            cache->is_dirty = false;
            printf("FETCH: whole file %s v%d cached locally\n", fname, cache->cached_ver);
            return;
        }
    }
}

void store_file(VenusCache *cache) {
    if (!cache->is_dirty) return;
    for (int i = 0; i < num_files; i++) {
        if (strcmp(server_files[i].name, cache->cached_name) == 0) {
            strcpy(server_files[i].data, cache->cached_data);
            server_files[i].ver++;
            cache->cached_ver = server_files[i].ver;
            cache->is_dirty = false;
            printf("STORE: file %s updated to v%d on server\n", cache->cached_name, server_files[i].ver);
            printf("CALLBACK: notified other clients of change\n");
            return;
        }
    }
}

void local_write(VenusCache *cache, const char *newdata) {
    strcpy(cache->cached_data, newdata);
    cache->is_dirty = true;
    printf("LOCAL WRITE: modified cache of %s\n", cache->cached_name);
}

int main(void) {
    init_server();
    VenusCache client1 = {0};
    VenusCache client2 = {0};
    printf("AFS Simulation Start\n");
    fetch_file(&client1, "notes.txt");
    local_write(&client1, "Updated notes by client1");
    store_file(&client1);
    fetch_file(&client2, "notes.txt");
    printf("Client2 reads: %s\n", client2.cached_data);
    local_write(&client1, "Second update");
    store_file(&client1);
    printf("After callback, client2 cache invalid, would refetch\n");
    fetch_file(&client2, "notes.txt");
    printf("Client2 now has: %s\n", client2.cached_data);
    printf("AFS Simulation End\n");
    return 0;
}

Compile locally

gcc -std=c11 -Wall -Wextra -Wpedantic -Werror ostep-50-afs.c -o example && ./example

Expected result

AFS Simulation Start
FETCH: whole file notes.txt v1 cached locally
LOCAL WRITE: modified cache of notes.txt
STORE: file notes.txt updated to v2 on server
CALLBACK: notified other clients of change
FETCH: whole file notes.txt v2 cached locally
Client2 reads: Updated notes by client1
LOCAL WRITE: modified cache of notes.txt
STORE: file notes.txt updated to v3 on server
CALLBACK: notified other clients of change
After callback, client2 cache invalid, would refetch
FETCH: whole file notes.txt v3 cached locally
Client2 now has: Second update
AFS Simulation End

CHECK YOUR UNDERSTANDING

Close the answer. Explain it.

Why do callbacks scale AFS better than client-initiated periodic probes?

Check the sources

• OSTEP chapter PDF

Drafts and official chapters change. The version mark is only the example’s minimum.