Extent Descriptors#

Extent descriptors (type 0x40) map a file’s logical cluster offsets (VCNs) to virtual LCNs (VLCNs). A VLCN is not a physical address: it must be translated through the Container Table to obtain the physical cluster where the data actually lives. Every extent — even a single contiguous run — is a fixed 24-byte entry.

Do not sanity-check a VLCN against the volume’s cluster count. The virtual address space is wider than the physical one — it starts above the first container and runs roughly twice as far — so a perfectly valid extent can name a cluster number larger than the volume has. The only sound test is whether the Container Table can translate it. Getting this wrong is quietly destructive: a decoder that walks extents in order and stops at the first “implausible” one throws the whole map away, and the file then looks as though it had no extents at all. On one 4 GB volume that mistake hid the contents of 583 of its 629 files, every one of which read back byte-perfect once the check was corrected.

Extent Entry — 24 bytes#

Extent Flags#

The flags occupy two bytes at extent+0x08; the two bytes above them are the record size. Read as one 32-bit value the pair reads (record_size << 16) | flags, which is why the constants seen in the wild all begin 0x18 — that leading 0x18 is 24, the size of a plain record, not a flag bit.

Measured across 48,474 extent records on 40 volumes: the record size is 24 + (bit 0x80 ? run*4 : 0) in every case, with no exception. Observed flag values are 0x0040, 0x0050, 0x0060 (plain) and 0x00d0 (checksummed).

Run cardinality is carried explicitly by the Run length field at extent+0x14, not by the flag bits: both 0x0040 and 0x0050 appear with single-cluster and multi-cluster runs.

Bit 0x10 means “written”, as of the row version that carries it. A run whose flags word is known and has bit 0x10 clear was allocated but not yet written when that row version was committed, and its clusters are not the file’s content: they are zero-filled, never read. Three measurements fix the meaning — a purpose-built lab volume whose files extract byte-exact only under this rule (7 of 7), a real installation where the clear-bit runs of pre-allocated files hold another file’s bytes (693 runs, among them readable text), and the inverse control, a file whose genuinely written zeros must still be read.

Two qualifications follow from that measurement:

• The bit belongs to the row version, not to the cluster. A deleted remnant is often an earlier version of a file that is still live, committed between allocation and the write. For the same VLCN the live row can carry 0x00D0 (written) while the remnant carries 0x0040 (not written). Recovering that remnant zero-fills the run — which is what the file held at that moment — even though the clusters hold data now, and a carve reports how many non-zero bytes lie under the zeros it wrote.
• A flags word of 0 is unknown, not “unwritten”. The heuristic marker-scan decoder recovers no flags, so its runs carry 0; treating 0 as “not written” would zero real content. Only a positively known flags word with bit 0x10 clear marks a run unwritten.

Records are 8-byte aligned, so the step from one record to the next is record_size rounded up to a multiple of 8 — the size and the step are different quantities, and a reader must step by the second. A checksummed single-cluster record is 28 bytes (24 + one 4-byte CRC32-C) and is followed 32 bytes later; a three-cluster one is 36 bytes, followed 40 bytes later. The bytes in between are padding, not a field. A plain record is already a multiple of 8 and needs none. Measured on the single-node extent maps of win11refstestmftecmd: stepping by the aligned step reaches all 142 checksummed clusters in them, stepping by the record size only 127, and wherever a record follows a 28-byte one it starts at +32, never at +28. A node’s records are the ones its key index names — a node can keep a superseded record in a slot the index no longer points at — so a complete reader follows the index, and descends into child pages where the map spans more than one node: that is how integrity reads them, 55,586 checksummed clusters in 189 files on that volume, every one matching its stored CRC-32C.

Checksummed and plain records share the same header format and both point at real file data; they differ only in the trailing checksums and the size that implies.

No single-extent “shortcut” form#

There is no compact 16-byte single-extent form. A contiguous file uses the standard 24-byte extent entry (one entry, run_length = the whole contiguous run). Non-resident files resolve via the 24-byte stride only; sampled single-extent files content-match (24-byte VLCN@0x00 → PLCN holds the file’s bytes).

This 16-byte region is the embedded $DATA sub-record header: the bytes 02000080 80000e00 … decode as the multi-instance marker 0x80000002 followed by the $DATA descriptor 0x000E0080 — i.e. the “VLCN” is the descriptor, not a cluster. See the Embedded $DATA section below.

VCN ordering#

Extents may be stored out of order on disk. A parser must sort by file_VCN before reassembling file content — roughly half of multi-extent entries observed are unsorted.

VLCN to PLCN translation#

A VLCN must be resolved through the Container Table to reach the physical disk address:

physical_LCN = container_phys_start + (vlcn & (CPC - 1))

Where CPC (clusters per container) is read from the Container Table at value + 0x18, and the container index is computed as vlcn >> CPC.bit_length() (shift = 15 for 4 KiB clusters, 11 for 64 KiB).

See Container Table for the full address-translation formula and the failover/checksum details.

Embedded extent sub-record header — 40 bytes#

When $DATA (type 0x80) appears as an embedded multi-instance sub-record in a file value, the extent list is preceded by two headers. This is the extent-bearing $DATA stream summary (marker 0x80000002, descriptor 0x000E0080) — distinct from the smaller resident-SI $DATA record.

$DATA sub-record header (offsets are the v3.7+ layout)#

On ReFS v3.4 the four size fields shift +4 (total allocated 0x34, stream size 0x3C, valid data length 0x44); the disk-allocated slot (0x4C) is 0 and the allocation is carried by the total-allocated field. The inner header size (0x88) and the extent sub-record at +0x88 are unchanged across versions.

Extent sub-record header (at inner_header_size offset)#

Extent entries following this header use the same 24-byte format as the Extent Entry table above. Snapshot and copy-on-write $DATA reuse this identical extent format (the driver routes them through the same allocation-lookup routine as ordinary file reads).

Where the extent records live — a B+-tree node#

A file’s extent records are not a bare list. They are the rows of a small B+-tree node held inside the file’s own $DATA record, at offset 0x88 of that record. A file with a single run looks like a plain array because the node holds one row; a fragmented or large file needs more, and then the node framing has to be read.

The node begins with a 40-byte header:

This is the same node header the B+-tree page uses, and the same invariant applies: (u32@+0x20 − u32@+0x10) / 4 == u32@+0x14. Read the array’s start from +0x10; deriving it by subtracting 4 × row_count from a value at +0x20 gives the same answer on a well-formed node — which is why that reading went unnoticed — but it cannot tell a damaged header from a good one, and it treats +0x20 as an 8-byte field when +0x24 is a separate word. Verified on 480 extent nodes with 0 violations.

Rows are addressed by that trailing index of 4-byte slots — a 2-byte row offset plus a 2-byte word that is a B+-tree upper-bound sentinel, not the row’s file VCN. Read the file VCN from inside the row. The advice is unchanged; the reason for it is: the word reads 0xFFFF on 1,364 leaf rows whose own file VCN is below 0xFFFF (against 433 where it happens to equal fvcn & 0xFFFF), so it is not a saturated VCN. Worked case: win1121h2test OID 0x701, sub_id 0x1000, count 1, key-index bytes 28 00 ff ff — row offset 0x28, sentinel 0xFFFF, and that row’s file VCN is 0 with run 1.

Two further consequences for anyone walking these rows:

• The extent area is not 0x28 + 24 × count. u32@ihdr+0x04 disagrees with that formula on 486 of 2,172 extent-node headers. On win11refs2g_sha256checksums OID 0x707 the header says count 2 and end 0x70, and the area holds three 24-byte slots (0x28, 0x40, 0x58) of which the key index lists only two — the slot at 0x28 is unindexed.
• Rows are not a contiguous array in key-index order on 961 of 1,865 multi-row nodes.

An unindexed slot still holds a well-formed row. That matters for recovery: a deletion can drop a row from the index while the bytes remain, so a reader bounded strictly to the index will miss content that is still on disk, and a reader that ignores the index will list rows twice.

When the map outgrows one node, the level byte becomes non-zero and each row’s value is a 48-byte node reference holding four cluster numbers. Those are virtual cluster numbers — translate them through the Container Table — and together they address one 16 KiB page. That page carries the same node structure again, at offset 0x50 + <the 32-bit value at page+0x50>, so the walk repeats unchanged.

Picking the right record. A file that has stream snapshots carries one $DATA record per version, all under the same key and all the same size. They are told apart by a sub-stream id in the key at key+0x10: 0x1000 is the live stream, and 0x1001, 0x1002 and so on are the snapshots, oldest first. A reader that picks by size or takes the first match will read a snapshot’s extents and hand back an older version of the file as though it were the current one.

How often each map form occurs#

A volume does not use one extent-map form. Across 102 images and 109,588 extent-backed streams, three forms occur, and which one a stream uses is a property of the record, not of the ReFS version:

Where the $DATA holder sits is a separate question, and it does not determine the map form:

Both separate-record locations carry a mix of the contiguous array and the index array, so a reader cannot choose how to parse the map from where the record lives — it has to look at what the record contains.

Which form a reader should trust. The node form above is authoritative: its rows are addressed by the node’s index array, and a row the index array does not reference is dead space. A reader that walks the record’s bytes positionally instead can pick such a row up, or match a value that merely looks like an entry — and the usual guard, that the runs exactly cover the file’s allocation, does not catch it, because a wrong single run of the right length covers just as well. Reading the node structurally is what distinguishes them.

By version. ReFS 3.4 uses the embedded index array for every stream. The contiguous array appears only on 3.14. The B+-tree node form is not a 3.14 feature: it already carries the majority of streams on 3.7 (295 of 469) and 3.9 (1,052 of 1,502).

Maps that outgrow one node. 139 streams have so many runs that the node becomes an index node whose rows point at child pages, as described above. Rare, but the reason the form exists: no flat array walk reaches those extents.

Forensic notes#

• Two-level translation (VCN → VLCN → PLCN) is fundamental to ReFS. A parser that treats VLCN values as direct physical addresses — a common mistake for NTFS-trained tools — will read wrong data.
• Sorting by file_VCN is mandatory before content reassembly.
• Every extent (including a single contiguous run) is a 24-byte entry; there is no 16-byte single-extent form. A 0x80000002 marker in this region is the embedded $DATA sub-record header, not an extent.

Cross-references#

• Container Table — VLCN-to-PLCN translation
• Directory Entries — non-resident file values link to type 0x40 extent rows
• Resident Storage — small files are stored inline, not in extents

Sparse files: the ratio is the signal#

A sparse region is unallocated, so the gap between a file’s logical size and its allocated size is what identifies one. A 64 KiB file with a punched middle was measured at 65,536 logical against 8,192 allocated — two clusters backing a sixteen-cluster file — carrying Archive|SparseFile. A fully sparse file goes further: its allocated size is 0. Both measured on format 3.14.

Extraction fills the hole with zeros, which is what the volume holds there. On a sparsely-stored raw image those zeros are indistinguishable from a region the acquisition never captured — see the note on image holes in what survives.

Evidence#

The map-form and holder-location counts are a raw-disk census of 102 images and 109,588 extent-backed streams, every one attributed. The census was taken while the marker scan was consulted first, which is how forefst read these records up to and including v1.11.3; from v1.12.0 the structural readings are consulted first and the scan is a logged fallback, which corrected 23 streams across 11 files and gave 593 files a decoded extent map that had none before.

The 24-byte extent entry layout (VLCN@0x00, flags@0x08, file_VCN@0x0C, padding@0x10, run_length@0x14) is raw-disk decoded and corroborated in the driver: CmsStream::LookupAllocation (with AddAllocation / DeleteAllocation) reads run+0x00 VLCN, +0x0C file_vcn, +0x14 run_length. The VLCN→PLCN translation formula is confirmed in the CmsVolumeContainer container subsystem and verified on disk across the corpus.

The extent-flag meanings are raw-disk verified: flags 0x0040 is the standard data run, 0x00d0 carries per-cluster checksums, and the 0x0050 vs 0x0040 distinction (bit 0x10) is not run cardinality — both occur with single- and multi-cluster runs. A plain record is 24 bytes; the 16-byte region at the descriptor is the embedded $DATA sub-record header. Snapshot/CoW DATA uses the identical 24-byte extent format (same CmsStream::LookupAllocation routine).

The checksum appended to an integrity record is CRC32-C (Castagnoli, poly 0x82f63b78) of the 4 KiB cluster: confirmed in the driver — the crc32c_4096 kernels via ComputeOneChecksum (which uses the 4096-byte path only when the span is one cluster) — and on disk, 886 checksummed clusters recomputed across three integrity volumes with 0 mismatches (a cross-algorithm control ruled out plain CRC-32 and CRC64). The sparse-hole entry is identified by flags bit 0x20, and that bit alone: a hole’s VLCN mirrors its file VCN, so it is usually non-zero (for example file_vcn=1, VLCN=0x1, run=25343, flags=0x180060). Measured across all six extent-decoding paths on four volumes — 8,864 accepted records — every record carrying VLCN == 0 also carries bit 0x20, and records carry bit 0x20 without a zero VLCN, so the flag is the discriminator. A hole’s run is zero-filled and never read from disk.

VLCN == 0 remains a read-time floor rather than a hole test: VLCN 0 → PLCN 0 is the boot region on every volume, so cluster 0 is never a file’s content. An entry with a zero VLCN and no 0x20 flag is a malformed record — it is rejected, so the surrounding map fails its cover check cleanly, rather than being served as a run of zeros.

A sparse file’s entries tile only its allocated clusters, at their logical positions, with no entry for the range between them. A file of 16 clusters with 2 allocated appears as one entry at file VCN 0 and one at file VCN 15; the uncovered range is the hole.

The remaining field-level statements on this page are registered as — each with its own evidence tier and witness in the claim register. The logical-versus-allocated signal is confirmed, and the alloc_size = 0 case for a fully sparse file is confirmed. The extent-record key shape that identifies a non-inline stream’s own extent sub-record — key offset 0x10, key size 0x50 — is confirmed.