Skip to content

Improve ExecutionContext propagation#5156

Merged
nohwnd merged 23 commits into
microsoft:mainfrom
Youssef1313:execution-context
Mar 7, 2025
Merged

Improve ExecutionContext propagation#5156
nohwnd merged 23 commits into
microsoft:mainfrom
Youssef1313:execution-context

Conversation

@Youssef1313
Copy link
Copy Markdown
Member

@Youssef1313 Youssef1313 commented Feb 28, 2025

Fixes #5154
Fixes #4132

Original behavior on 3.6.x (before introduction of RetryAttribute and making RunSingleTest being async)

When not enabling parallelization

We execute tests in a loop, and at any step, async local changes are visible to all subsequent steps. (assuming no Timeout is involved, because that was previously a very different story).

By "step", I mean assembly initialize, class initialize, ctor, test initialize, test method, test cleanup, Dispose, and DisposeAsync.

When enabling parallelization

We create N worker threads, each is dequeuing test cases from a common concurrent queue, and then running them.

All tests in each thread are run on a similar fashion as the "non-parallelizing" case. But when parallelization is enabled, a change to an async local at any step, won't be visible from the other thread. That means, a change to an async local in assembly initialize won't be visible to all tests. It will be visible to tests that run on the same worker thread though.

New behavior from this PR

We have three levels. First is assembly, second is class, third is test.
We propagate from the highest level down, but not the other way around.

That means, a change of an async local in AssemblyInitialize is visible to all ClassInitialize methods regardless of which thread they are on. Similarly, a change in ClassInitialize is visible to all test methods (including initialize and cleanup). Also, a change from test initialize is visible to the test method itself, etc.

However, A change of an async local in a TestCleanup is NEVER seen in ClassCleanup. That's what I mean by propagating from highest level down, and not the other way around. This creates reasonable isolation.

This works as follows:

  1. After assembly initialize, we capture the execution context and save it as the "assembly-level" context.
  2. We execute class initialize on the assembly-level execution context.
  3. After running class init, we capture the execution context and save it as the "class-level" context.
  4. etc etc

ℹ️ The test methods are duplicated, because the first method will already have the correct context. So it would not fail. The second method will restore the context, and if we don't restore it correctly we will fail.

@Youssef1313 Youssef1313 marked this pull request as ready for review March 3, 2025 13:24
@Youssef1313
Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Copy Markdown
Member

@nohwnd nohwnd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seeing minimal changes in tests, is the change of how tests are isolated not happening in this PR, or we don't test it?

@Youssef1313
Copy link
Copy Markdown
Member Author

seeing minimal changes in tests, is the change of how tests are isolated not happening in this PR, or we don't test it?

There were some existing tests which I extended a bit.

@nohwnd nohwnd merged commit 858905f into microsoft:main Mar 7, 2025
@Youssef1313 Youssef1313 deleted the execution-context branch March 16, 2025 11:24
@Youssef1313
Copy link
Copy Markdown
Member Author

/backport to rel/3.8

@github-actions
Copy link
Copy Markdown
Contributor

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patch

Applying: Improve ExecutionContext propagation
Using index info to reconstruct a base tree...
M	src/Adapter/MSTest.TestAdapter/Execution/TestClassInfo.cs
M	src/Adapter/MSTest.TestAdapter/Execution/TestMethodInfo.cs
M	src/Adapter/MSTest.TestAdapter/Execution/UnitTestRunner.cs
Falling back to patching base and 3-way merge...
Removing src/Adapter/MSTestAdapter.PlatformServices/Services/InstanceExecutionContextScope.cs
Removing src/Adapter/MSTestAdapter.PlatformServices/Services/IExecutionContextScope.cs
Removing src/Adapter/MSTestAdapter.PlatformServices/Services/ExecutionContextService.cs
Removing src/Adapter/MSTestAdapter.PlatformServices/Services/ClassExecutionContextScope.cs
Removing src/Adapter/MSTestAdapter.PlatformServices/Services/AssemblyExecutionContextScope.cs
Auto-merging src/Adapter/MSTest.TestAdapter/Execution/UnitTestRunner.cs
Auto-merging src/Adapter/MSTest.TestAdapter/Execution/TestMethodInfo.cs
CONFLICT (content): Merge conflict in src/Adapter/MSTest.TestAdapter/Execution/TestMethodInfo.cs
Auto-merging src/Adapter/MSTest.TestAdapter/Execution/TestClassInfo.cs
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0001 Improve ExecutionContext propagation
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

Applying: Improve ExecutionContext propagation
Applying: Fix tests
Applying: Fix for class cleanups
Applying: Adjust
Applying: Adjust for netfx
Applying: Adjust
Applying: Delete wrong comment
Applying: Remove misleading comment
Applying: Enhance tests
Applying: Fix test
Applying: More test fixes
Applying: Fix tests
Applying: Delete unintentionally committed code
Applying: Create LogMessageListener on the correct execution context
error: sha1 information is lacking or useless (src/Adapter/MSTest.TestAdapter/Execution/TestClassInfo.cs).
error: could not build fake ancestor
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0014 Create LogMessageListener on the correct execution context
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

16 similar comments
@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

@Youssef1313 backporting to rel/3.8 failed, the patch most likely resulted in conflicts:

$ git am --continue

hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Applying: Create LogMessageListener on the correct execution context
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.
Error: The process '/usr/bin/git' failed with exit code 128

NOTE: A PR will be created, but needs to be revised manually!

@github-actions
Copy link
Copy Markdown
Contributor

Potential infinite loop guard hit. Stopping

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CallContext.LogicalSetData not preserved Huge performance degradation between 3.2 and 3.6.3

2 participants