Skip to content

Consolidate filesystem walkers across platforms #101

@m3nu

Description

@m3nu

Context

We currently maintain two filesystem walker implementations for backup:

  1. Inode-sorted walker (inode_walk.rs) — Linux only (#[cfg(target_os = "linux")]). Sorts stat() calls by inode number for 3-8x HDD speedup on ext4/xfs/reiserfs. Manages its own gitignore stack, exclude patterns, and marker file filtering.

  2. ignore-crate walker (walk/mod.rs, build_configured_walker) — non-Linux fallback. Delegates gitignore filtering to the ignore crate's WalkBuilder, with our own filter_entry closure for exclude patterns, marker files, and cross-device filtering.

This dual implementation causes a practical gap: on non-Linux, gitignore-excluded files can't be logged at debug level because the ignore crate filters them internally before our code sees them (discovered in #73).

Proposal

Consolidate to a single walker implementation across all platforms:

  • Extend the inode walker to all Unix — change cfg(target_os = "linux") to cfg(unix). Make inode_sort_beneficial() return false on non-Linux (no ext4/xfs detection needed). macOS gets filename-order DFS with full gitignore logging.

  • Extend to Windows — the core walk logic (read_dir, symlink_metadata, DFS stack) is cross-platform. Windows-specific: use dummy inode (0, no sorting), skip statfs, handle one_file_system/device via platform abstraction.

  • Keep ignore::gitignore — we still need the ignore crate for gitignore pattern matching (Gitignore, GitignoreBuilder). Only the WalkBuilder walker would be dropped.

  • Remove build_configured_walker and walk_source_ignore — replaced by the consolidated walker. The existing filter_equivalence_with_ignore_walker test can be kept as a regression test or adapted.

Benefits

  • Single code path for filtering logic (excludes, gitignore, markers, cross-device)
  • Consistent debug logging of excluded files on all platforms
  • Less code to maintain

Considerations

  • Needs Windows CI testing
  • one_file_system semantics may differ on Windows (drive letters vs device IDs)
  • The ignore crate dependency remains for pattern matching

Related: #73

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions