Data Integrity and Protection

Partial Failures in Contemporary Disks

Early designs assumed a disk is either fully healthy or completely dead. In reality a drive can appear operational yet lose individual sectors or silently alter contents. Latent sector errors stem from head scratches or cosmic rays and are reported by on-drive ECC; silent corruptions arise from firmware writing to the wrong place or bus errors, leaving the disk unaware. Consumer drives show these events far more often than enterprise models, yet neither class can be ignored.

Dealing with Detectable Sector Errors

Once a latent sector error occurs the drive reports failure at once. The storage layer simply uses existing redundancy: a mirror reads the other copy, a parity group reconstructs from remaining blocks. The real danger is a second error appearing while a whole disk is being rebuilt. Single-parity RAID then cannot finish, so some designs add a second parity disk, trading capacity for higher rebuild success.

Catching Silent Corruption with Checksums

Silent corruption yields no error code, so a short digest must be stored beside the data. The checksum is computed on write and recomputed on read; a mismatch declares the block bad and recovery switches to another copy. Fast XOR or additive sums miss some patterns, while CRC or cryptographic hashes catch more at higher CPU cost. There is no free lunch: stronger protection always costs more.

Pitfalls

• Assuming on-disk ECC is enough and skipping higher-level checks
• Rebuilding RAID without extra redundancy so a second sector error loses data
• Picking a checksum too weak to notice particular bit-flip patterns

Minimum C11 · complete program · Download .c

#include <stdio.h>
#include <stdint.h>
#include <string.h>

#define BLOCK_SIZE 16

static uint8_t compute_xor_checksum(const uint8_t *block, size_t n) {
    uint8_t cs = 0;
    for (size_t i = 0; i < n; ++i) {
        cs ^= block[i];
    }
    return cs;
}

int main(void) {
    uint8_t block[BLOCK_SIZE];
    const char *msg = "Hello OSTEP!";
    memset(block, 0, BLOCK_SIZE);
    strncpy((char *)block, msg, BLOCK_SIZE - 1);
    uint8_t stored_cs = compute_xor_checksum(block, BLOCK_SIZE);
    printf("Stored data: %s\n", (char *)block);
    printf("Stored checksum: 0x%02X\n", stored_cs);
    block[3] ^= 0x01;  /* silent corruption */
    uint8_t read_cs = compute_xor_checksum(block, BLOCK_SIZE);
    printf("Read checksum: 0x%02X\n", read_cs);
    if (read_cs != stored_cs) {
        printf("Corruption detected! Data integrity violated.\n");
    } else {
        printf("Checksums match.\n");
    }
    return 0;
}

Compile locally

gcc -std=c11 -Wall -Wextra -Wpedantic -Werror ostep-45-data-integrity.c -o example && ./example

Expected result

Stored data: Hello OSTEP!
Stored checksum: 0x1E
Read checksum: 0x1F
Corruption detected! Data integrity violated.

CHECK YOUR UNDERSTANDING

Close the answer. Explain it.

Why must checksums still be computed and verified even when RAID is already in use?

Check the sources

• OSTEP chapter PDF

Drafts and official chapters change. The version mark is only the example’s minimum.