Metadata Log (MLog)#

The MLog implements write-ahead logging for atomic metadata updates. It is redo-only: on crash recovery, committed transactions are replayed and there is no undo mechanism — copy-on-write keeps prior pages intact, so undo is unnecessary. Each MLog data page is one LogCore data record nesting four layers: a record header, an entry header, a redo block, and the inner redo records that carry the operations of a single transaction.

Record structure — four layers#

Each MLog data page is a single LogCore data record. The on-disk format is four nested layers, not two. All offsets are version-invariant — the only version difference is the entry-header payload-offset value (Layer 2 +0x28).

page/record
 ├─ 0x00 Layer 1: LogCore record header (0x78 = 120 bytes)
 ├─ 0x78 Layer 2: entry header (56 bytes; 64 on Insider)
 ├─ 0xB0 Layer 3: redo block _SmsRedoHeader (0xB8 on Insider)
 └─ Layer 4: _SmsRedoRecord entries

Layer 1 — LogCore record header (record+0x00, 120 bytes)#

Layer 2 — entry header (record+0x78)#

56 bytes on v3.4–v3.14, 64 bytes on Insider 29574. Offsets are entry-relative (absolute page offset in parentheses):

The redo block starts at record + 0x78 + payload_offset = record+0xB0 (v3.4–v3.14) / record+0xB8 (Insider). Robust, version-safe pointer: redo = record + le32(record,0x54) + le32(record + le32(record,0x54), 0x28), taken only when the entry type (== 2) is a data record.

Layer 3 — redo block _SmsRedoHeader#

Each data page holds exactly one redo block — typically all records of a single transaction (start → operations → commit).

Layer 4 — _SmsRedoRecord (minimum 0x38 = 56 bytes)#

Note (the equally-sized 56-byte structures): the Layer 2 entry header and the Layer 4 _SmsRedoRecord are both 56 bytes but are distinct. The entry header carries the per-record checksum (at +0x08); the _SmsRedoRecord does not (it starts with record_size/opcode). The literature’s “56-byte log record header with checksum” is the entry header; the “120-byte entry header” is the Layer 1 record header.

Iteration logic (from ForEachRedoInBlock)#

remaining = total_size - first_record_offset
ptr = header + first_record_offset
if (total_size < first_record_offset + 8) or (total_size < 0x38): error
while (record.size <= remaining) and (record.size != 0):
 process(record)
 remaining -= record.size
 if remaining < 0x38: break
 ptr += record.size

Overall layout#

The MLog is split into a control area and a data area.

MLog location#

The MLog is located by its fixed physical LCN, not by file path. The MLog Logfile Information Table (OID 0x9, accessed via the Object Table, plus duplicate OID 0xA — see System OIDs) stores the control and data area LCN range.

Data area addressing is physical. Unlike most ReFS structures, MLog data area LCNs (from the Logfile Information Table key=1 row) are physical cluster addresses — no container translation is needed.

Data-area block packing (4 KiB log blocks). The data area is addressed in volume clusters, but the MLog’s own I/O unit — the log block, which is exactly one LogCore record — is 4 KiB always, independent of the cluster size (it is record+0x0C, hard-set to 0x1000 by MlLogOpenLog). Consequences:

On a 64K-cluster volume each cluster holds 16 consecutive log records with consecutive LSNs, so a scanner must iterate 4 KiB blocks, not clusters — otherwise a 64K-cluster log exposes only 1/16 of its records. On 4 KiB-cluster volumes the two are identical.

Verifying a record#

Every log record — control page and data record alike — carries an 8-byte integrity value at entry header +0x08 (= page+0x80, the entry-header offset at page+0x54 being 0x78). The driver recomputes it before replaying a record, and it can be recomputed from the page alone:

field_off = le32(page, 0x54) + 8          # the integrity value's own position
block     = le32(page, 0x0C)              # 0x1000 on every volume, whatever the cluster size
zero the 8 bytes at field_off
result = 0
for each block of `block` bytes:
    acc = [rotate_left_1(result), 0, 0, 0]     # chain seed; 0 for the first block
    for i, word in enumerate(little-endian u64 words of the block):
        acc[i mod 4] ^= word
    v      = acc[0] ^ acc[1] ^ acc[2] ^ acc[3]
    result = (v >> 32) ^ (v & 0xFFFFFFFF)

A record is exactly one log block, so the chain seed is 0 in practice. The record type at entry header +0x30 says which kind of record it is — 1 for a control page, 2 for a data record; no other value occurs on disk.

Across the image corpus (ReFS 3.4 → 3.15, 4 K and 64 K clusters) this reproduces the stored value on all 599,039 data records and all 95 control pages.

An integrity check, not a signature. It detects any altered byte — a single-bit flip at any of a record’s 4,096 offsets is caught — but being an XOR fold it is blind to a reordering of 8-byte words that leaves each word at the same position modulo 4.

forefst <image> mlog --stats runs this over every record in the data area and over both control copies, reporting each failure with its stored and recomputed value.

Control page layout — ~252 bytes (18 fields decoded)#

The remaining fields are NOT at fixed page offsets. They live in a payload block accessed by reading entry_header_offset at page+0x54, then payload_offset at entry_header+0x28. The field offsets below are relative to the payload base (data_off = entry_header_offset + payload_offset):

18 field positions decoded (16 with assigned meaning). The payload base typically falls around page offset 0xB0 (the entry-header offset at 0x54 is 0x78/120) but is not fixed — always dereference via the entry header.

The control area is a compact header. Although the control area reserves 0xE48 (3656) bytes (self-described by the 0x98 length word), it is populated only through ≈byte 0xF8; everything from ~0xF8 to 0xE48 (≈93% of the area) is zero. Those bytes are not hidden fields — they are the zero gaps between the small fixed descriptors above plus the zero tail. The header is stable per volume (magic@0x04, UUID@0x10 echoed at payload+0x28, area-length@0x98, the data-area bounds, and the LSN/generation pair do not change between captures of the same volume); the dynamic log progression lives in the data-area records, not the control header.

Redo opcodes#

The opcode is read at _SmsRedoRecord + 0x04 and dispatched by CmsLogRedoQueue::PerformRedo. Opcode counts vary by version.

The dispatched opcode ranges are contiguous; the only in-range value that is not a handler is v3.14 0x17 (returns NTSTATUS 0xC0000427, the same generic unhandled-opcode error as out-of-range opcodes). Every in-range value branches to a named handler; the only in-range value that is not a handler is v3.14 0x17, which returns the generic unhandled-opcode error.

Cross-version: the v3.4 core (0x00–0x1C) carries forward almost unchanged (4 renamed, 0x17 turned into an error, 0 removed); v3.14 adds 0x1D–0x2B (15 new values, mainly stream / table-set / refcount).

v3.14 dispatch table (44 opcode values, ~39 handlers)#

Only v3.14 0x17 falls through to NTSTATUS 0xC0000427 (the same generic unhandled-opcode error as out-of-range opcodes). Opcodes 0x25/0x2B dispatch through the CmsTableSetBase v-table (indirect; concrete handler not statically resolvable).

Full dispatch tables with cross-version mapping are maintained in the project’s redo-opcode reference.

Transaction structure#

Each MLog data page holds one redo block containing the records for a single atomic transaction, marked by the transaction-start flag (record flags bit 0, at record+0x2C — the same flag the driver’s replay tests). The commit flag (bit 1) is always 0 on disk — a commit is recorded at the checkpoint/log-core layer, not per record — so transactions are delimited by the start flag, and each fits within one 4 KiB log block.

Event timestamps — when the operation happened#

Some operations carry their own time and some do not. A create or write embeds the object’s updated $STANDARD_INFORMATION in its own records, so the operation FILETIME reads straight out of the transaction. A rename or a permanent delete carries no inline time — ReFS writes the object’s new $SI as a separate $SI-update in the following log block, whose MACB timestamps sit at that block’s offset +0x140. mlog --parse pairs the two, so a rename/delete now shows its real event time instead of a blank — the same time then flows into timeline. (A transaction that already carries its own time is never overridden.)

Concrete actions — grouping redo opcodes into what a user did#

One file operation is not one redo record. ReFS writes a file operation as a cluster of low-level B+-tree redo opcodes (open a table, insert a row, set a parent id, update a stream summary, allocate clusters…), often split across several transactions. forefst mlog --parse groups the opcodes of each transaction into a single concrete action, and decides the ambiguous ones from facts in the record (parent OIDs, which table is destroyed) — not from opcode presence alone. Every action is still backed by its raw redo records; mlog --parse -v prints each one as opcode name target_oid @PLCN+offset key, so any field can be verified against the raw disk bytes. Actions are reported in two groups.

File operations — what a user did:

Low-level / metadata records — the B+-tree redo that accompanies the file operations (kept as facts):

MOVE vs RENAME — decided by the parent OID, not by an opcode#

A rename and a move look almost identical in the log: both remove the old name entry (RedoDeleteRow) and insert the new one (RedoInsertRow), and ReFS emits RedoReparentTable for both — so a reparent opcode does not mean “move”. forefst instead compares the parent-directory table OID carried by the two name-entry records (parse_mlog_deep_record reads it at key-component[0] + 0x14):

• old-name DeleteRow.target_oid == new-name InsertRow.target_oid → RENAME (renamed in place)
• old-name DeleteRow.target_oid ≠ new-name InsertRow.target_oid → MOVE (reparented to another directory)

This is a fact from the bytes (verifiable with -v), so the label is trustworthy. When only one of the two name entries is present in a transaction, the parent cannot be compared and the record is reported as REPARENT rather than guessed. Validated against an independent operation log (Generate-FSActivity replay + the USN journal): on that single validation run, MOVE and RENAME agreed with ground truth on every resolvable object.

DELETE vs a rename/move’s row removal#

RedoDeleteRow removes a single B+-tree row — and it fires on the OLD name entry of a rename and the source of a move, not only on a real deletion. A bare DeleteRow is therefore not evidence a file was deleted. A real deletion destroys the object’s own table with RedoDeleteTable (0x0F). forefst reports DELETE only when RedoDeleteTable is present; a DeleteRow with no table destroyed is ENTRY_REMOVE (the old-name cleanup of a rename/move, or a hard-link unlink). Reading a [DELETE] as “the file was deleted” without this rule is a classic false conclusion — the tool now makes the distinction explicit.

Timestamp extraction#

MLog records do not have a dedicated timestamp field in their headers. Timestamps are embedded in the value data of records that modify file metadata:

Timestamps are Windows FILETIME format (100-ns intervals since 1601-01-01). Not all transactions contain timestamps — system-level operations (allocator, container table) typically have no embedded time. Timestamps reflect the file operation time, not the log write time.

Key component layout#

InsertRow records carry structured key data. Component[0] of the key path has this 28-byte layout:

The schema pair identifies which table and attribute type the record targets. Common pairs: 0xe030/0x0130 (ObjectTable/$FILE_NAME), 0xe030/0x0160 (ObjectTable/ReparseIndex).

Data area sizing#

Recovery process#

The CmsRestarter class performs:

1. Analysis pass: Scans from oldest needed LSN (stored at CHKP+0x70) to identify dirty pages and active transactions
2. Redo pass: Replays all committed redo records in LSN order

There is no undo pass. Copy-on-write ensures uncommitted transactions never overwrote prior state.

Forensic value#

The MLog is the durable transaction log — the on-disk record of committed activity. It matters forensically because the committed metadata tree (what files and summary read) can omit the most recent pending activity: an operation that has been written to the log but not yet folded into a checkpoint is present in the MLog while it is still absent from the committed tree. Reading the MLog can therefore surface recent file operations that the tree listing does not yet show.

• The MLog does not carry pre-images (unlike NTFS $LogFile), so it cannot directly reconstruct previous file states
• Opcodes reveal what operations occurred (inserts, deletes, modifications)
• The LSN range and record count indicate volume activity level
• The oldest log record ref in the CHKP ties the current checkpoint to the log
• Transaction classification maps opcode sequences to human-readable actions (CREATE, DELETE, RENAME, MOVE, WRITE, etc.)
• OID-to-path resolution links redo records to specific files and directories
• Embedded timestamps in InsertRow and UpdateRow value data provide approximate operation times (most, but not all, transactions carry timestamps)
• CSV export enables bulk analysis and timeline correlation with other forensic artifacts

Tooling#

MLog parsing is integrated into forefst.py (parsing + display). Available modes via forefst.py <image> mlog:

Validated across v3.4 / v3.7 / v3.9 / v3.10 / v3.14 / Insider volumes — all PASS the LogCore framing check (entry header @0x78, type 2, single per-volume magic), and every record’s integrity value recomputes: 599,039 data records and 95 control pages, no mismatches. The 4 KiB-block scanner gives full coverage on 64K-cluster volumes.

Cross-references#

• Checkpoint (CHKP) — oldest log record ref at CHKP+0x70
• Page Header — MLog pages share the "MLog" signature at offset 0x00 but use a distinct header layout (per-volume format magic at 0x04 — not a CRC — not the common 80-byte format)
• Copy-on-Write — why redo-only logging is sufficient
• System OIDs — OID 0x9/0xA (Logfile Information Table) stores the MLog LCN range

Operation patterns from prior work, re-measured#

Lee’s journal analysis published seven opcode sequences, one per filesystem operation. Re-running them against three v3.4 images is a good illustration of how far a published pattern travels: three reproduce exactly, one is mislabelled, one does not occur as a single transaction at all, and two carry the right opcodes in a different order.

The practical lesson for an examiner is that a sequence identifies an operation only up to its first few opcodes, and that a published label may name the wrong operation. forefst mlog --parse classifies from the opcodes it actually reads rather than from a fixed table of sequences.

What the log physically contains#

The data area holds page images, not field-level edits: a scan finds 58–94 whole MSB+ pages in it. Combined with copy-on-write, that means a logged page is recoverable in full rather than as a diff — which is why the redo-only design needs no undo records.

Evidence#

The four-layer record format was decoded byte-for-byte from LogCoreWriteDataRecord (write), LogCoreScanDataRecord / LogValidateEntryHeader (read/validate) and LogInitializeEntryHeader, cross-confirmed in v3.4 (Win10), v3.14 (Win11) and Insider 29574, and raw-disk verified across the corpus. The fixed log-block size (0x1000) is set by MlLogOpenLog. The redo opcode dispatch (each value branching to a PDB-named handler) is decompiled from CmsLogRedoQueue::PerformRedo; the 64K-cluster 4 KiB-block packing and the compact-control-header (≈93% zero) results are raw-disk decoded. The recovery passes come from the CmsRestarter class. The format magic is a per-volume constant, not a CRC. The remaining field-level statements on this page are registered as — each with its own evidence tier and witness in the claim register. The published operation patterns and their re-measurement are confirmed; the physical page-image logging model is confirmed, and the second control entry’s LCN is confirmed.