dotnet-watch tests failing

[dsplaisted]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=629827
Build error leg or test failing: dotnet-watch.Tests.dll.1.WorkItemExecution
Pull request: #39888

Error Message

##[error].packages/microsoft.dotnet.helix.sdk/8.0.0-beta.24177.1/tools/Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(89,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Work item dotnet-watch.Tests.dll.1 in job 82728a56-9430-4356-ab82-1c21670792a7 has failed

{
  "ErrorMessage": "",
  "ErrorPattern": "Work item dotnet-watch\\.Tests\\.dll\\.1 in job [a-z0-9\\-]+ has failed",
  "BuildRetry": true,
  "ExcludeConsoleLog": true
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=629827
Error message validated: [Work item dotnet-watch\.Tests\.dll\.1 in job [a-z0-9\-]+ has failed]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 4/4/2024 9:51:10 AM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
1432638	dotnet/sdk	🟣 Run TestBuild Tests	Log	#54418
1431312	dotnet/sdk	🟣 Run TestBuild Tests	Log	#54319
1431105	dotnet/sdk	🟣 Run TestBuild Tests	Log	#54319
2981352	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#61441
2981096	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#61243
1428739	dotnet/sdk	🟣 Run FullFramework Tests	Log	#54320
1429644	dotnet/sdk	🟣 Run TestBuild Tests	Log	#54395
1429647	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53950
2980690	dotnet-sdk	Run TestBuild Tests	Log	#61388
2980475	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#61229
2980383	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#61229
1428998	dotnet/sdk	🟣 Run TestBuild Tests	Log	#54394
2980193	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#61229
2979183	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#61229
2979274	dotnet-sdk	Run FullFramework Tests	Log	#61302
2979273	dotnet-sdk	Run FullFramework Tests	Log	#61301
2979015	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#61243
1427093	dotnet/sdk	🟣 Run FullFramework Tests	Log	#54329
1425994	dotnet/sdk	Run TestBuild Tests	Log	#54370
1421610	dotnet/sdk	🟣 Run FullFramework Tests	Log	#54313
2978158	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#61243
2978072	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#61229
1424056	dotnet/sdk	Run TestBuild Tests	Log	#54326
2976122	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#61144
1422779	dotnet/sdk	Run TestBuild Tests	Log	#54273
2975065	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#60782
2974382	dotnet-sdk	Run Tests in Helix	Log	#60891
2974045	dotnet-sdk	Run TestBuild Tests	Log	#60811
1419065	dotnet/sdk	Run TestBuild Tests	Log	#53437
1418648	dotnet/sdk	Run TestBuild Tests	Log	#54290
1418581	dotnet/sdk	Run TestBuild Tests	Log	#54288
1418246	dotnet/sdk	Run FullFramework Tests	Log	#54280
1418071	dotnet/sdk	🟣 Run FullFramework Tests	Log	#54276
2973116	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#60995
1416849	dotnet/sdk	Run TestBuild Tests	Log	#54261
1416727	dotnet/sdk	Run TestBuild Tests	Log	#54259
2972404	dotnet-sdk	🟣 Run FullFramework Tests	Log
2972435	dotnet-sdk	Run Tests in Helix and non Helix in parallel	Log	#60782
2970561	dotnet-sdk	🟣 Run TestBuild Tests	Log
2970601	dotnet-sdk	Run TestBuild Tests	Log	#60902
1413936	dotnet/sdk	Run FullFramework Tests	Log	#54054
1413347	dotnet/sdk	🟣 Run TestBuild Tests	Log	#54235
2969894	dotnet-sdk	Run FullFramework Tests	Log	#60836
2969673	dotnet-sdk	Run TestBuild Tests	Log	#60836
1412610	dotnet/sdk	Run TestBuild Tests	Log	#54194
1411419	dotnet/sdk	Run FullFramework Tests	Log	#54031
1410975	dotnet/sdk	🟣 Run FullFramework Tests	Log	#54215
1410818	dotnet/sdk	Run TestBuild Tests	Log	#54214
1410775	dotnet/sdk	Run TestBuild Tests	Log	#54204
1410380	dotnet/sdk	Run FullFramework Tests	Log	#54204
1410237	dotnet/sdk	Run TestBuild Tests	Log	#53878
1410113	dotnet/sdk	Run TestBuild Tests	Log	#54007
1410136	dotnet/sdk	Run TestBuild Tests	Log	#54042
2968189	dotnet-sdk	Run TestBuild Tests	Log	#60805
1409690	dotnet/sdk	Run TestBuild Tests	Log	#54194
1409527	dotnet/sdk	Run TestBuild Tests	Log	#54191
1409525	dotnet/sdk	Run FullFramework Tests	Log	#54190
2964193	dotnet-sdk	Run TestBuild Tests	Log	#60657
1402970	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53847
2963887	dotnet-sdk	Run TestBuild Tests	Log	#60627
2963888	dotnet-sdk	Run FullFramework Tests	Log	#60628
1402794	dotnet/sdk	🟣 Run FullFramework Tests	Log	#54116
1402113	dotnet/sdk	🟣 Run FullFramework Tests	Log	#54115
2963221	dotnet-sdk	Run TestBuild Tests	Log	#60586
1401850	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53946
2963122	dotnet-sdk	Run FullFramework Tests	Log	#60361
2962905	dotnet-sdk	Run TestBuild Tests	Log	#60361
1386618	dotnet/sdk	🟣 Run TestBuild Tests	Log	#53938
1396988	dotnet/sdk	Run TestBuild Tests	Log	#54073
1395510	dotnet/sdk	🟣 Run TestBuild Tests	Log	#51088

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	23	70

[marcpopMSFT]

@tmat we still see this fairly frequently. Any luck? Here's a recent one from a codeflow PR:
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-sdk-refs-pull-41616-merge-58511284b77c40778a/dotnet-watch.Tests.dll.1/1/console.418de7ca.log?helixlogtype=result

[tmat]

No luck yet. This might help: https://github.com/dotnet/sdk/blob/d45e40d665e647804de03d2b4eec2aae9c488cb5/test/dotnet-watch.Tests/xunit.runner.json

It will print out test info every 20 seconds a test is running.

[marcpopMSFT]

@tmat another failure this morning shows a bunch of build processes for watch apps still running... https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-sdk-refs-pull-41958-merge-84357342ee154a5088/dotnet-watch.Tests.dll.2/1/console.3dddc463.log?helixlogtype=result

[tmat]

Looks like that's in 8.0 branch. Porting test diagnostics from 9: #41963

[Forgind]

This has been making one of my PRs fail today, so I looked into it a bit. Based on when this issue was created and which tests are failing, I suspect this PR:
#39618

I asked jasonmalinowski whether the MSBuildWorkspace is supposed to support F# because I'd found a code snippet that suggested that it shouldn't, and he said it doesn't and never did. Did that PR make us use MSBuildWorkspace in a different way than we had before?

[Forgind]

The changes to HotReloadDotNetWatcher look most suspicious to me. It looks like a substantial change to how we call MSBuild.