Hawkynt.FileFormats.FileSystems 1.0.0.337

Hawkynt.FileFormats.FileSystems

Pure-managed filesystem readers / writers + disk-image container readers extracted from CompressionWorkbench. Sister package to Hawkynt.FileFormats.Audio / Hawkynt.FileFormats.Archives / Hawkynt.FileFormats.Images, all built on top of Hawkynt.Compression.Core.

The package bundles every filesystem and disk-image assembly into lib/, so consumers add a single dependency and can mount-and-walk a .vhd / .qcow2 / .iso / .dmg / .d64 etc. and read out its files entirely in managed code — no libguestfs, no platform mounts, no elevated privileges.

When to use this package

• Forensic / archival inspection: open a disk image without mounting it; walk inodes / MFT / catalog records / B-trees in process memory
• Cross-platform tooling: read NTFS from Linux, ext4 from Windows, HFS+ from anywhere — same managed code, no native runtime
• Retro-computing pipelines: dump Commodore D64 / Apple DOS / Atari 8-bit / Spectrum / BBC Micro / CP/M disks straight from emulator captures
• Cloud / VM image inspection: peek inside QCOW2 / VMDK / VDI / VHDX without spinning up a hypervisor
• WORM-creatable images: build filesystem images for tests / fuzzing / firmware packaging — see the source repo's WSL-validated mkfs-parity tests

Every creatable filesystem builds real nested directory trees (not flattened paths) of arbitrary size — multi-block / multi-leaf directory growth (B-tree node splitting on btrfs / xfs / jfs / reiserfs / apfs / hfs+ / ntfs $INDEX_ALLOCATION, fat-ZAP on zfs, multi-segment dentry blocks on f2fs, indirect/extent-backed directories on ext / ufs / minix / udf / ocfs2 / prodos / bfs / hpfs). Creatable images can also wipe unused space and cluster-tip slack (IWipeEmpty), defragment (including fusing fragmented directory extents), and pick optimal cluster/block sizes via the layout optimizer. Where an external tool exists, image conformance is validated against the real OS utility — fsck.fat, fsck.exfat, e2fsck, xfs_repair, btrfs check, fsck.f2fs, fsck.hfsplus, fsck.jfs, reiserfsck, ntfs-3g, xorriso, mtools — in the test suite (setup-test-environment.sh installs them).

Skip it when:

• You only need to read files from a real mounted volume — System.IO does that with the OS's filesystem driver, no need for in-process parsers
• You need block-level write semantics with journaling guarantees — write support here is geared to one-shot image creation, not concurrent runtime mounts

Quick start — read a disk image

using FileSystem.Fat;

var img    = File.ReadAllBytes("disk.img");
using var fs = new FatReader(img);
foreach (var entry in fs.ListRecursive())
  Console.WriteLine($"{entry.Path}  {entry.Size,10}  {entry.Modified:O}");

Quick start — peer inside a virtual disk image

using FileFormat.Vhd;
using FileSystem.Ntfs;

using var stream = File.OpenRead("system.vhd");
var inner = new VhdReader().OpenContents(stream);   // exposes the partition table
foreach (var partition in inner.Partitions) {
  if (partition.Type == "NTFS") {
    var ntfs = new NtfsReader(partition.Open());
    foreach (var path in ntfs.WalkPaths())
      Console.WriteLine(path);
  }
}

Contents

State legend:

• R — read-only: open the image and walk it.
• WORM — Write-Once-Read-Many: read AND can synthesise a fresh image from scratch (IArchiveCreatable), but cannot modify an existing image in place. Enough to build minimal images for tests / fuzzing / firmware packaging; not enough to act as a runtime driver.
• R/W — read + true in-place modification (IArchiveModifiable): add / replace / remove files inside an existing image with consistent free-space bookkeeping. The runtime path matches what mkfs + a real driver would do.

Disk-image containers (FileFormat.*)

Filesystem readers / writers (FileSystem.*)

Microsoft / Windows

Unix / Linux

Apple / classic Mac

Compressed / embedded / flash

Optical

Retro / vintage

Mainframe / minicomputer

Stub-tier (detection-only / opaque-blob surface)

These descriptors recognise the container and surface either the whole image or the inner payload as opaque entries plus a metadata.ini. No directory walk is attempted — either the on-disk spec is not public, the payload is encrypted, or the format is a proprietary compressed wrapper whose codec is out of scope. CanCreate / CanModify are intentionally not advertised; upgrading any of these to WORM requires the bullet under "what would be needed".

WSL-validated filesystems

Several filesystems' writers have CI-validated round-trips against the actual fsck / repair / check binaries shipped with Linux — the same programs the kernel uses to trust a filesystem before mount.

The matrix is self-documenting via tests in Compression.Tests/ExternalFsInteropTests.cs — each gate skips cleanly with the exact sudo apt install -y package name when its tool is missing, so CI can run on a host with only e2fsprogs and the rest skip without failing.

Disk-image container validation matrix

qemu-img is the canonical disk-container validator (it parses VHD / VMDK / QCOW2 / VDI / VHDX with the same loader path that qemu-system-* uses to boot a VM). Tests gate on either the Windows-native binary or the WSL package — install sudo apt install -y qemu-utils (smaller than qemu-system-* — only ships the image tools).

The full forensic-style chain (Vmdk_ContainingExt_RoundTripExtractsFiles) builds an ext FS in-memory with two known-content files, wraps it in a VMDK, optionally validates with qemu-img check, then walks the chain back via our own readers (VmdkReader.Extract → ExtReader) asserting both files come out byte-equal — runs unconditionally; qemu-img is a bonus, not a gate.

Why no R/W for VHDX yet, why no mkfs.ext1?

The state column reflects what the writer can do; readers cover everything the table mentions. WORM means we ship a spec-compliant image creator that round-trips through the real external tool; R means the writer either is not implemented or does not yet pass external validation. Where the upstream format has no validator (ext1, HAMMER, MFS, retro 8-bit FSes, proprietary game FSes), we round-trip through our own reader/writer pair plus byte-level spec-offset assertion tests.

Filesystem-aware recovery

FilesystemCarver (in Compression.Analysis) scans an image for known superblock signatures at canonical offsets (ext at +1080, FAT at +54 / +82, SquashFS at 0, APFS at +32, Btrfs at +0x10020 …), validates each hit by asking the matching reader to walk it, and extracts every readable entry — useful for SD-card / disk-dump recovery when the partition table is gone but the inner filesystem superblock survives.

using var fs = File.OpenRead("sdcard.img");
var hits = new FilesystemCarver().CarveStream(fs);
foreach (var c in hits) {
  var result = FilesystemExtractor.ExtractCarved(fs, c, $"out/{c.FormatId}_0x{c.ByteOffset:X}");
  Console.WriteLine($"{c.FormatId}: {result.FilesExtracted} files, {result.FilesFailed} failed");
}

CLI: cwb recover sdcard.img (auto), cwb recover raw.img --mode filesystems --out out/, cwb recover raw.img --mode files --format Jpeg,Png (photorec-style file carving).

Versioning

Version-locked 1:1 with Hawkynt.Compression.Core. Pin both at the same version.

License

LGPL-3.0-or-later. See the source repository for the full license text.