Skip to content

Commit 380033d

Browse files
tyrielvtyrielv
committed
Add migration status docs and miniksa reference for session handoff
Documentation for picking up the .NET 10 NativeAOT migration work from a clean slate. Includes: - docs/net10-migration-status.md: Current CI results, remaining failures with analysis, .NET 10 behavioral changes discovered, build commands, deferred items - docs/miniksa-reference.md: What was applied from miniksa's prototype branch, what wasn't applied yet, and what original work we did beyond his branch Assisted-by: Claude Opus 4.6 Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
1 parent 378a612 commit 380033d

2 files changed

Lines changed: 245 additions & 0 deletions

File tree

‎docs/miniksa-reference.md‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
# Miniksa's NativeAOT Prototype — Applied vs. Remaining
2+
3+
**Reference branch:** [`miniksa:user/miniksa/net10-nativeaot`](https://github.com/miniksa/VFSForGit/tree/user/miniksa/net10-nativeaot)
4+
5+
Miniksa (Michael Niksa) did the original prototype work for migrating VFSForGit
6+
to .NET 10 with NativeAOT. He also maintains the
7+
[ProjFS-Managed-API](https://github.com/microsoft/ProjFS-Managed-API) package
8+
which was rewritten from C++/CLI to pure C# P/Invoke for AOT compatibility
9+
(v2.1.0, commit `0fff2cd`).
10+
11+
## What We Applied from Miniksa's Branch
12+
13+
### Fully Applied
14+
- **TFM retarget approach:** `net10.0-windows10.0.17763.0` target framework
15+
- **Directory.Build.props structure:** Centralized TFM, RID, AOT settings
16+
- **PublishAot + SelfContained:** Same AOT compilation strategy
17+
- **Test project opt-out:** `PublishAot=false` for test/tooling projects
18+
- **Build.bat 3-step approach:** dotnet restore → VS MSBuild for C++ → dotnet publish for managed
19+
- **Output path changes:** `publish\` subdirectory for managed AOT binaries
20+
- **SDK pinning:** `global.json` with `rollForward: disable`
21+
- **Microsoft.Windows.ProjFS 2.1.0:** Pure C# P/Invoke package replacing C++/CLI
22+
23+
### Applied with Modifications
24+
- **GVFSJsonContext.cs:** We created our own source-generated JSON context with 25+ types
25+
(miniksa's branch had a similar but not identical set of serializable types — ours was built
26+
from scratch by analyzing all JSON serialization call sites)
27+
- **GVFSJsonOptions.cs TypeInfoResolverChain:** Same concept (source-gen + reflection fallback)
28+
but our implementation includes IL2026/IL3050 suppressions and different ordering
29+
- **layout.bat cleanup:** Same idea of removing managed artifacts, but our implementation
30+
also handles orphaned PDBs and runtimeconfig files
31+
- **CreateBuildArtifacts.bat:** Updated for publish\ paths, but we also added trailing `\*`
32+
fixes for xcopy
33+
34+
### Not Yet Applied (Still in Miniksa's Branch)
35+
- **Full WinHttpHandler replacement:** Miniksa replaced `HttpClientHandler` usage with
36+
`WinHttpHandler` for better AOT compatibility. We deferred this — current code uses
37+
`HttpClientHandler` with `UseDefaultCredentials` removed. May need this if HTTP-related
38+
AOT issues surface.
39+
- **Additional trimming configuration:** Miniksa may have `TrimmerRootDescriptor.xml` or
40+
`ILLink` settings that we haven't ported.
41+
- **ARM64-specific adjustments:** Miniksa's branch may have ARM64-specific changes beyond
42+
what we've done (we set `RuntimeIdentifier=win-x64` globally).
43+
44+
## Work We Did That Wasn't in Miniksa's Branch
45+
46+
### Phase 1 & 2 (Merged Separately)
47+
- **Dead code removal** (PRs #1937–#1939, #1941, #1946): Removed obsolete code paths,
48+
deprecated APIs, and unused dependencies to reduce the surface area before the TFM retarget
49+
- **System.Text.Json migration:** Full migration from Newtonsoft.Json to STJ, including
50+
custom converters (EventMetadataConverter for `Dictionary<string, object>`)
51+
52+
### CI Infrastructure
53+
- **functional-tests.yaml:** Added Verify GVFS installation diagnostic step, timeout handling,
54+
`--workers=1` sequential execution
55+
- **RunFunctionalTests-Dev.ps1:** Created dev-mode functional test runner that works without
56+
admin/system install
57+
- **CI debug logging:** `[CI-DEBUG]` output in test framework for diagnosing CI-specific issues
58+
59+
### Bug Fixes Discovered During Migration
60+
- **Null triggeringProcessImageFileName guard:** ProjFS v2.1.0 regression where
61+
`Marshal.PtrToStringUni(IntPtr.Zero)` returns null instead of the old C++/CLI behavior
62+
of returning `String.Empty`. This affects any ProjFS consumer, not just VFSForGit.
63+
- **CountingStream for DeflateStream truncation:** .NET 10 behavioral change where
64+
`DeflateStream` silently returns partial data on truncated zlib instead of throwing
65+
`InvalidDataException`. This is a general .NET migration concern, not VFSForGit-specific.
66+
- **ConsoleOutputPayload visibility:** Made internal for source generator access
67+
- **CentralPackageTransitivePinningEnabled:** Required for consistent dependency resolution
68+
with the CI NuGet feed
69+
70+
### Payload/Installer Cleanup
71+
- **Removed all ProjectReferences from GVFS.Payload.csproj:** Build.bat handles ordering;
72+
ProjectReferences caused issues with mixed C++/managed builds
73+
- **Removed CopyToOutputDirectory for hooks:** GVFS.Hooks binaries are placed by Build.bat,
74+
not by project references
75+
- **FilesToSign updates:** Only native executables in the AOT payload need signing
76+
77+
## Key Differences from Miniksa's Approach
78+
79+
1. **We split into phases (1–5)** while miniksa's branch was a single large change.
80+
This makes the migration reviewable and bisectable.
81+
82+
2. **We kept `DefaultJsonTypeInfoResolver` as fallback** in the TypeInfoResolverChain.
83+
Miniksa may have been more aggressive about removing reflection-based JSON. Our approach
84+
is more conservative — source-gen for known types, reflection for unknown — which avoids
85+
runtime failures from missing type metadata at the cost of larger binaries.
86+
87+
3. **We use `dotnet publish` instead of `dotnet build`** for managed projects. `dotnet build`
88+
with PublishAot produces apphost stubs without Win32 version resources; `dotnet publish`
89+
produces proper native binaries that InnoSetup's `GetFileVersion` can read.
90+
91+
4. **SDK version pinning:** We pin to `10.0.202` with `rollForward: disable` to prevent
92+
CI from using a different SDK version that would pull different runtime pack versions
93+
through the NuGet feed.

‎docs/net10-migration-status.md‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
# .NET 10 NativeAOT Migration — Status & Remaining Work
2+
3+
**Branch:** `tyrielv/net10-tfm-retarget`
4+
**PR:** [#1947](https://github.com/microsoft/VFSForGit/pull/1947)
5+
**Base:** `upstream/master` (includes merged Phases 1–2)
6+
7+
## Migration Phases Overview
8+
9+
| Phase | Description | Status |
10+
|-------|-------------|--------|
11+
| 1 | Remove dead/deprecated code | ✅ Merged (PRs #1937, #1946, #1941, #1939, #1938) |
12+
| 2 | System.Text.Json migration | ✅ Merged (PR above) |
13+
| 3 | TFM retarget + API migrations | ✅ In this PR |
14+
| 4 | NativeAOT self-contained deployment | ✅ In this PR |
15+
| 5 | Cleanup & optimization | 🔲 Future PR |
16+
17+
## What This PR Contains
18+
19+
### Commit 1: TFM Retarget + API Migrations + CI
20+
- Retargeted all 16 managed csproj files from `net471` to `net10.0-windows10.0.17763.0`
21+
- Updated `global.json`, `Directory.Build.props`, `Version.props`, `Directory.Packages.props`
22+
- Fixed API migrations: `ProcessHelper` (Environment.ProcessPath), `OrgInfoApiClient` (WebUtility.UrlEncode), `WindowsPlatform` (NamedPipeServerStreamAcl.Create), `HttpRequestor` (removed UseDefaultCredentials), `WindowsFileSystem` (DirectoryInfo/FileSecurity APIs)
23+
- Updated CI `build.yaml`, scripts (`Build.bat`, `RunUnitTests.bat`, `RunFunctionalTests.bat`, `CreateBuildArtifacts.bat`, `RunFunctionalTests-Dev.ps1`)
24+
- Fixed obsolete API warnings (Uri.EscapeUriString, X509Certificate2 ctor)
25+
- 803 unit tests passing
26+
27+
### Commit 2: NativeAOT Self-Contained Deployment
28+
- Added `SelfContained=true`, `PublishAot=true`, `OptimizationPreference=Speed` to Directory.Build.props
29+
- Opted out test projects and GVFS.MSBuild from AOT
30+
- Rewrote `Build.bat`: 3-step approach (dotnet restore → VS MSBuild for C++ → dotnet publish for managed)
31+
- Updated all script paths for `publish\` subdirectory
32+
- Created `GVFSJsonContext.cs` — source-generated JSON serializer context (25+ types)
33+
- Updated `GVFSJsonOptions.cs` with TypeInfoResolverChain (source-gen + reflection fallback)
34+
- Cleaned installer payload: removed stray `.runtimeconfig.json`, orphaned PDBs
35+
- AOT binaries: GVFS.exe ~20MB, GVFS.Mount.exe ~20MB, GVFS.Service.exe ~6.7MB
36+
37+
### Commit 3: CI Diagnostic Logging + Timeout Handling
38+
- Added `[CI-DEBUG]` logging to clone/mount/gvfs process invocations
39+
- Added 5-minute timeout per gvfs process call
40+
- Added `--workers=1` for sequential CI test execution
41+
- Added `timeout-minutes: 60` to functional-tests.yaml
42+
43+
### Commit 4: Null Guard for ProjFS triggeringProcessImageFileName
44+
- Fixed `ArgumentNullException` crash in mount process callbacks
45+
- Root cause: ProjFS managed API v2.1.0 returns null for kernel-level operations
46+
- Fix: null-coalesce to `string.Empty` in `FileSystemCallbacks`
47+
48+
### Commit 5: CountingStream for Truncated Object Detection
49+
- Fixed `TruncatedObjectRedownloaded` test failure (4/4 in CI)
50+
- Root cause: .NET 10 `DeflateStream` silently returns partial data on truncated zlib
51+
- Fix: `CountingStream` wrapper verifies bytes read matches header-declared size
52+
53+
## Current CI Results (as of last push)
54+
55+
| Slice | Status | Notes |
56+
|-------|--------|-------|
57+
| 0 | ✅ 4/4 | |
58+
| 1 | ✅ 4/4 | Fixed by CountingStream |
59+
| 2 | ⚠️ 2/4 | Intermittent: `gvfs prefetch --hydrate` AV crash (0xC0000005) |
60+
| 3 | ⚠️ 1/4 | Intermittent: same prefetch AV crash |
61+
| 4 | ✅ 4/4 | |
62+
| 5 | ✅ 4/4 | |
63+
| 6 | ✅ 4/4 | |
64+
| 7 | ❌ 0/4 | `ChangeTimestampAndDiff`: pathspec not found — see below |
65+
| 8 | ⚠️ 3/4 | 1 intermittent failure |
66+
| 9 | ❌ 0/4 | All flavors hang/timeout — see below |
67+
68+
**Overall: 7 of 10 slices fully passing, up from 0 at start of CI debugging.**
69+
70+
## Remaining Failures to Investigate
71+
72+
### 1. `ChangeTimestampAndDiff` (Slice 7, 4/4 fail)
73+
74+
**Error:** `git checkout GVFS\GVFS.Common\GVFSContext.cs` fails with "pathspec did not match any file(s) known to git"
75+
76+
**Analysis:**
77+
- The file path is hardcoded in `GitCommandsTests.EditFilePath` (line 23)
78+
- The file DOES exist in the test repo (`ForTests`) — confirmed by manual clone
79+
- `ValidateGitCommand` runs git in both the GVFS enlistment and a control repo
80+
- The error comes from the GVFS enlistment side, suggesting a projection issue
81+
- Could be related to the mount process or index projection failing for this specific file
82+
- **Not yet locally reproduced** — local test framework was setting up 20 parallel fixtures due to wrong NUnit filter. Use `--test=` (not `--where=`) to run individual tests
83+
84+
### 2. Slice 9 Hang (4/4 timeout)
85+
86+
**Analysis:**
87+
- All 4 flavors time out at 60 minutes
88+
- Need to identify which test(s) fall into slice 9 — run test discovery with `--slice=9,10`
89+
- Could be the same mount crash or a deadlock in a specific test
90+
91+
### 3. Prefetch --hydrate AV (Slices 2/3, intermittent)
92+
93+
**Error:** `gvfs prefetch` exits with -1073741819 (0xC0000005 ACCESS_VIOLATION)
94+
95+
**Analysis:**
96+
- Happens during `PrefetchWithStats` → multithreaded pipeline (diff → blobFinder → downloader → packIndexer → fileHydrator)
97+
- `HydrateFilesStage` calls `NativeFileReader.TryReadFirstByteOfFile` via P/Invoke (CreateFile/ReadFile)
98+
- Intermittent (2/4 and 3/4) — likely a race condition or timing-dependent AOT issue
99+
- P/Invoke declarations use `DllImport` — could benefit from `LibraryImport` for AOT
100+
- The `SafeFileHandle` marshaling under AOT may have edge cases
101+
102+
## .NET 10 Behavioral Changes Discovered
103+
104+
### DeflateStream Truncation Behavior
105+
- **Old (.NET 4.7.1):** Throws `InvalidDataException` on truncated zlib data
106+
- **New (.NET 10):** Silently returns partial data, no exception
107+
- **Impact:** Any code relying on DeflateStream exceptions for corruption detection needs explicit length checks
108+
- **Fix applied:** `CountingStream` in `GitRepo.GetLooseBlobStateAtPath`
109+
110+
### ProjFS Managed API Null Behavior
111+
- **Old (C++/CLI):** `TriggeringProcessImageFileName` returned `String.Empty` for NULL native pointers
112+
- **New (P/Invoke v2.1.0):** Returns `null` from `Marshal.PtrToStringUni(IntPtr.Zero)`
113+
- **Impact:** Any consumer using the value as a dictionary key or in null-intolerant APIs will crash
114+
- **Fix applied:** Null-coalesce in `FileSystemCallbacks`
115+
- **Upstream fix needed:** `Microsoft.Windows.ProjFS` package should match old behavior
116+
117+
## Deferred Items (Phase 5 / Future PRs)
118+
119+
- ProjFS driver installation removal (branch `tyrielv/remove-projfs-install` created but not implemented)
120+
- `upgrade.ring` config removal
121+
- Full `WinHttpHandler` replacement (see miniksa's approach)
122+
- `info.bat` update for ProjFS paths
123+
- Remove CI debug logging (`[CI-DEBUG]` lines) after tests stabilize
124+
- Re-enable parallel workers (remove `--workers=1`) after fixing remaining issues
125+
- File ProjFS upstream issue for null `TriggeringProcessImageFileName`
126+
127+
## Build & Test Commands
128+
129+
```powershell
130+
# Build (from src\)
131+
scripts\Build.bat Debug
132+
133+
# Unit tests
134+
..\out\GVFS.UnitTests\bin\Debug\net10.0-windows10.0.17763.0\win-x64\publish\GVFS.UnitTests.exe
135+
136+
# Functional tests (dev mode, no admin)
137+
powershell -File scripts\RunFunctionalTests-Dev.ps1 Debug '--test=<fully.qualified.test.name>'
138+
139+
# Install local build
140+
$v = & { $n = [DateTime]::Now; "0.2.$($n.ToString('yy'))$($n.DayOfYear.ToString('D3')).$([int]($n.TimeOfDay.TotalSeconds / 2))" }
141+
scripts\Build.bat Debug $v
142+
# Then run installer from out\GVFS.Installers\bin\Debug\win-x64\
143+
```
144+
145+
## CI NuGet Feed
146+
147+
CI uses `https://pkgs.dev.azure.com/gvfs/ci/_packaging/Dependencies/nuget/v3/index.json` as the sole NuGet source. To pull through new packages:
148+
1. Clear ALL local caches: `dotnet nuget locals all --clear`
149+
2. Run `dotnet restore` — packages automatically pull through from nuget.org
150+
3. Never push packages manually to the feed
151+
152+
SDK version is pinned in `global.json` to `10.0.202` with `rollForward: disable`.

0 commit comments

Comments
 (0)