Metadata Log (MLog)#

The MLog implements write-ahead logging for atomic metadata updates. It is redo-only: on crash recovery, committed transactions are replayed and there is no undo mechanism — copy-on-write keeps prior pages intact, so undo is unnecessary. Each MLog data page is one LogCore data record nesting four layers: a record header, an entry header, a redo block, and the inner redo records that carry the operations of a single transaction.

Record structure — four layers#

Each MLog data page is a single LogCore data record. The on-disk format is four nested layers, not two. All offsets are version-invariant — the only version difference is the entry-header payload-offset value (Layer 2 +0x28).

page/record
 ├─ 0x00 Layer 1: LogCore record header (0x78 = 120 bytes)
 ├─ 0x78 Layer 2: entry header (56 bytes; 64 on Insider)
 ├─ 0xB0 Layer 3: redo block _SmsRedoHeader (0xB8 on Insider)
 └─ Layer 4: _SmsRedoRecord entries

Layer 1 — LogCore record header (record+0x00, 120 bytes)#

Layer 2 — entry header (record+0x78)#

56 bytes on v3.4–v3.14, 64 bytes on Insider 29574. Offsets are entry-relative (absolute page offset in parentheses):

The redo block starts at record + 0x78 + payload_offset = record+0xB0 (v3.4–v3.14) / record+0xB8 (Insider). Robust, version-safe pointer: redo = record + le32(record,0x54) + le32(record + le32(record,0x54), 0x28), taken only when the entry type (== 2) is a data record.

Layer 3 — redo block _SmsRedoHeader#

Each data page holds exactly one redo block — typically all records of a single transaction (start → operations → commit).

Layer 4 — _SmsRedoRecord (minimum 0x38 = 56 bytes)#

Note (the equally-sized 56-byte structures): the Layer 2 entry header and the Layer 4 _SmsRedoRecord are both 56 bytes but are distinct. The entry header carries the per-record checksum (at +0x08); the _SmsRedoRecord does not (it starts with record_size/opcode). The literature’s “56-byte log record header with checksum” is the entry header; the “120-byte entry header” is the Layer 1 record header.

Iteration logic (from ForEachRedoInBlock)#

remaining = total_size - first_record_offset
ptr = header + first_record_offset
if (total_size < first_record_offset + 8) or (total_size < 0x38): error
while (record.size <= remaining) and (record.size != 0):
 process(record)
 remaining -= record.size
 if remaining < 0x38: break
 ptr += record.size

Overall layout#

The MLog is split into a control area and a data area.

MLog location#

The MLog is located by its fixed physical LCN, not by file path. The MLog Logfile Information Table (OID 0x9, accessed via the Object Table, plus duplicate OID 0xA — see System OIDs) stores the control and data area LCN range.

Data area addressing is physical. Unlike most ReFS structures, MLog data area LCNs (from the Logfile Information Table key=1 row) are physical cluster addresses — no container translation is needed.

Data-area block packing (4 KiB log blocks). The data area is addressed in volume clusters, but the MLog’s own I/O unit — the log block, which is exactly one LogCore record — is 4 KiB always, independent of the cluster size (it is record+0x0C, hard-set to 0x1000 by MlLogOpenLog). Consequences:

On a 64K-cluster volume each cluster holds 16 consecutive log records with consecutive LSNs, so a scanner must iterate 4 KiB blocks, not clusters — otherwise a 64K-cluster log exposes only 1/16 of its records. On 4 KiB-cluster volumes the two are identical.

Control page layout — ~252 bytes (18 fields decoded)#

The remaining fields are NOT at fixed page offsets. They live in a payload block accessed by reading entry_header_offset at page+0x54, then payload_offset at entry_header+0x28. The field offsets below are relative to the payload base (data_off = entry_header_offset + payload_offset):

18 field positions decoded (16 with assigned meaning). The payload base typically falls around page offset 0xB0 (the entry-header offset at 0x54 is 0x78/120) but is not fixed — always dereference via the entry header.

The control area is a compact header. Although the control area reserves 0xE48 (3656) bytes (self-described by the 0x98 length word), it is populated only through ≈byte 0xF8; everything from ~0xF8 to 0xE48 (≈93% of the area) is zero. Those bytes are not hidden fields — they are the zero gaps between the small fixed descriptors above plus the zero tail. The header is stable per volume (magic@0x04, UUID@0x10 echoed at payload+0x28, area-length@0x98, the data-area bounds, and the LSN/generation pair do not change between captures of the same volume); the dynamic log progression lives in the data-area records, not the control header.

Redo opcodes#

The opcode is read at _SmsRedoRecord + 0x04 and dispatched by CmsLogRedoQueue::PerformRedo. Opcode counts vary by version.

The dispatched opcode ranges are contiguous; the only in-range value that is not a handler is v3.14 0x17 (returns NTSTATUS 0xC0000427, the same generic unhandled-opcode error as out-of-range opcodes). Every in-range value branches to a named handler; the only in-range value that is not a handler is v3.14 0x17, which returns the generic unhandled-opcode error.

Cross-version: the v3.4 core (0x00–0x1C) carries forward almost unchanged (4 renamed, 0x17 turned into an error, 0 removed); v3.14 adds 0x1D–0x2B (15 new values, mainly stream / table-set / refcount).

v3.14 dispatch table (44 opcode values, ~39 handlers)#

Only v3.14 0x17 falls through to NTSTATUS 0xC0000427 (the same generic unhandled-opcode error as out-of-range opcodes). Opcodes 0x25/0x2B dispatch through the CmsTableSetBase v-table (indirect; concrete handler not statically resolvable).

Full dispatch tables with cross-version mapping are maintained in the project’s redo-opcode reference.

Transaction structure#

Each MLog data page holds one redo block containing the records for a single atomic transaction. A transaction typically follows the pattern: start (flags bit 0) → operations → commit (flags bit 1).

Concrete actions — grouping redo opcodes into what a user did#

One file operation is not one redo record. ReFS writes a file operation as a cluster of low-level B+-tree redo opcodes (open a table, insert a row, set a parent id, update a stream summary, allocate clusters…), often split across several transactions. forefst mlog --parse groups the opcodes of each transaction into a single concrete action, and decides the ambiguous ones from facts in the record (parent OIDs, which table is destroyed) — not from opcode presence alone. Every action is still backed by its raw redo records; mlog --parse -v prints each one as opcode name target_oid @PLCN+offset key, so any field can be verified against the raw disk bytes. Actions are reported in two groups.

File operations — what a user did:

Low-level / metadata records — the B+-tree redo that accompanies the file operations (kept as facts):

MOVE vs RENAME — decided by the parent OID, not by an opcode#

A rename and a move look almost identical in the log: both remove the old name entry (RedoDeleteRow) and insert the new one (RedoInsertRow), and ReFS emits RedoReparentTable for both — so a reparent opcode does not mean “move”. forefst instead compares the parent-directory table OID carried by the two name-entry records (parse_mlog_deep_record reads it at key-component[0] + 0x14):

• old-name DeleteRow.target_oid == new-name InsertRow.target_oid → RENAME (renamed in place)
• old-name DeleteRow.target_oid ≠ new-name InsertRow.target_oid → MOVE (reparented to another directory)

This is a fact from the bytes (verifiable with -v), so the label is trustworthy. When only one of the two name entries is present in a transaction, the parent cannot be compared and the record is reported as REPARENT rather than guessed. Validated against an independent operation log (Generate-FSActivity replay + the USN journal): on that single validation run, MOVE and RENAME agreed with ground truth on every resolvable object.

DELETE vs a rename/move’s row removal#

RedoDeleteRow removes a single B+-tree row — and it fires on the OLD name entry of a rename and the source of a move, not only on a real deletion. A bare DeleteRow is therefore not evidence a file was deleted. A real deletion destroys the object’s own table with RedoDeleteTable (0x0F). forefst reports DELETE only when RedoDeleteTable is present; a DeleteRow with no table destroyed is ENTRY_REMOVE (the old-name cleanup of a rename/move, or a hard-link unlink). Reading a [DELETE] as “the file was deleted” without this rule is a classic false conclusion — the tool now makes the distinction explicit.

Timestamp extraction#

MLog records do not have a dedicated timestamp field in their headers. Timestamps are embedded in the value data of records that modify file metadata:

Timestamps are Windows FILETIME format (100-ns intervals since 1601-01-01). Not all transactions contain timestamps — system-level operations (allocator, container table) typically have no embedded time. Timestamps reflect the file operation time, not the log write time.

Key component layout#

InsertRow records carry structured key data. Component[0] of the key path has this 28-byte layout:

The schema pair identifies which table and attribute type the record targets. Common pairs: 0xe030/0x0130 (ObjectTable/$FILE_NAME), 0xe030/0x0160 (ObjectTable/ReparseIndex).

Data area sizing#

Recovery process#

The CmsRestarter class performs:

1. Analysis pass: Scans from oldest needed LSN (stored at CHKP+0x70) to identify dirty pages and active transactions
2. Redo pass: Replays all committed redo records in LSN order

There is no undo pass. Copy-on-write ensures uncommitted transactions never overwrote prior state.

Forensic value#

• The MLog does not carry pre-images (unlike NTFS $LogFile), so it cannot directly reconstruct previous file states
• Opcodes reveal what operations occurred (inserts, deletes, modifications)
• The LSN range and record count indicate volume activity level
• The oldest log record ref in the CHKP ties the current checkpoint to the log
• Transaction classification maps opcode sequences to human-readable actions (CREATE, DELETE, RENAME, MOVE, WRITE, etc.)
• OID-to-path resolution links redo records to specific files and directories
• Embedded timestamps in InsertRow and UpdateRow value data provide approximate operation times (most, but not all, transactions carry timestamps)
• CSV export enables bulk analysis and timeline correlation with other forensic artifacts

Tooling#

MLog parsing is integrated into forefst.py (parsing + display). Available modes via forefst.py <image> mlog:

Validated across v3.4 / v3.7 / v3.9 / v3.10 / v3.14 / Insider volumes — all PASS the LogCore framing check (entry header @0x78, type 2, single per-volume magic). The 4 KiB-block scanner gives full coverage on 64K-cluster volumes.

Cross-references#

• Checkpoint (CHKP) — oldest log record ref at CHKP+0x70
• Page Header — MLog pages share the "MLog" signature at offset 0x00 but use a distinct header layout (per-volume format magic at 0x04 — not a CRC — not the common 80-byte format)
• Copy-on-Write — why redo-only logging is sufficient
• System OIDs — OID 0x9/0xA (Logfile Information Table) stores the MLog LCN range

Evidence#

The four-layer record format was decoded byte-for-byte from LogCoreWriteDataRecord (write), LogCoreScanDataRecord / LogValidateEntryHeader (read/validate) and LogInitializeEntryHeader, cross-confirmed in v3.4 (Win10), v3.14 (Win11) and Insider 29574, and raw-disk verified across the corpus. The fixed log-block size (0x1000) is set by MlLogOpenLog. The redo opcode dispatch (each value branching to a PDB-named handler) is decompiled from CmsLogRedoQueue::PerformRedo; the 64K-cluster 4 KiB-block packing and the compact-control-header (≈93% zero) results are raw-disk decoded. The recovery passes come from the CmsRestarter class. The format magic is a per-volume constant, not a CRC.