Skip to content

Sync with upstream main branch #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 188 commits into from
Dec 16, 2021

Conversation

joshpeterson
Copy link

This is an automatically generated pull request to merge changes from the upstream main branch.

radical and others added 30 commits December 4, 2021 01:48
Includes:
- Better error logging, and handling
- Add @(NativeFileReference) to up-to-date check items for VS
- Add `WasmBuild.sln`
- Better fix up of symbol names for pinvokes, and callbacks, based on @lambdageek's suggestion in dotnet#60814 (comment)

Fixes dotnet#60862
During my work on runtime test refactoring I noticed that when
CG2-compiling the merged test, the component assemblies end up
with an ever-increasing reference list. I tracked it down to this
poorly placed environment variable update.

Thanks

Tomas
These three projects apparently already use the [Fact] model;
this causes trouble in the switch-over to support new-style merged
test wrappers as the system believes these are standalone tests
due to the explicit Exe output type specification and and they fail
to provide a Main method. Thanks to Jeremy's source generation
change we can now tag them as "new-style tests".

Thanks

Tomas
In .NET 6, we added "real" support for Initialize on hash objects.
However, the Reset call would create a new hash object without the CNG
"resuable" flag. This led to the HashProvider's "_reusable" field and
the actual reusability of the hash instance to disagree.
* Add regex test for parsing runtime-assets expressions

* Address PR feedback
I recently noticed that the copynative step often takes an
excessive amount of time (up to 20 minutes). I tracked this down
to a deficiency caused by my summer cleanup of the test build
scripts - in the copynative step we should also use out-of-proc
msbuild executions for the individual test groups otherwise msbuild
chokes on the huge number of properties and slows down dramatically.

Thanks

Tomas
… done by fgMorphArgs() (dotnet#62379)

* Add regression test for dotnet#62249

* On Arm32 FEATURE_SIMD is not supported, hence varTypeIsSIMD() always returns false.
Change the assert, so it checks that the argument type is TYP_DOUBLE in Lowering::NewPutArg() in src/coreclr/jit/lower.cpp

* Fix GCC error
* Mark ArmBase.Yiels with HW_Flag_NoFloatingPointUsed in src/coreclr/jit/hwintrinsiclistarm64.h

* Move handling of ArmBase.Yield to an earlier point in impSpecialIntrinsic and simplify the remaining logic in src/coreclr/jit/hwintrinsicarm64.cpp

* Address feedback
* Move FabricBot rules to Configuration-As-Code

* Update docs/infra/automation.md

Co-authored-by: Dan Moseley <[email protected]>

* update FabricBot dump to most recent version

Co-authored-by: Dan Moseley <[email protected]>
* fix for dotnet#61899

* proper fix

* Fix for dotnet#62238

* misc change

* Revert "fix for dotnet#61899"

This reverts commit 1fc26a5.

* fix formatting

* fix formatting once again

* add validJumpKind check

* review comments
…he (dotnet#62411)

.. desktop. Earlier we did this for one test, but others are hitting
this issue randomly too. New one:

```
[00:32:41] fail: [FAIL] System.Diagnostics.Tests.StopwatchTests.StartNewAndReset
[00:32:41] info: Assert.True() Failure
[00:32:41] info: Expected: True
[00:32:41] info: Actual:   False
[00:32:41] info:    at System.Diagnostics.Tests.StopwatchTests.StartNewAndReset()
[00:32:41] info:    at System.Reflection.RuntimeMethodInfo.InvokeWorker(Object obj, BindingFlags invokeAttr, Span`1 parameters)
```

Issue: dotnet#62021
…2400)

* [mono][wasm] Fix an issue with the passing of small vtypes.

Fixes an issue introduced by 915ee6d.

* Reenable tests.

* Enable the tests for windows/aot also

Co-authored-by: Ankit Jain <[email protected]>
- silence errors before aborting the iframe
- separate iframe code
Co-authored-by: Radek Doulik <[email protected]>
…otnet#62382)

* Ensure that FMA codegen operand swapping matches the lsra selections

* Ensure operands end up in the right slots
In the case where replay of a test collection contains MISSING items,
or other SuperPMI replay failures, we strip the failing method contexts
from the result. This is very rare in the test scenario, but does happen
mostly (presumably) due to SuperPMI tool bugs. However, the strip command
wasn't actually being passed the set of things to strip, so it was just
doing a straight copy. Thus, we ended up failing the "clean replay" because
the failing MCs were still in the collection.

The test has had this problem for a long time, and it looks like it has
periodically hit in testing, but we haven't had bugs opened on these failures,
so we haven't noticed.
* Improve comments in generated Regex code

To help make the generated code easier to understand and debug.

* Format RegexOptions textually when possible
…otnet#62426)

* Reduce (?!) to nothing

The expression "(?!)" (an empty negative lookahead) is sometimes used in expressions at a point where we want to match nothing and thus fail a match, often in combination with a conditional operation.  This negative lookahead wrapped around an empty can be reduced to simply a Nothing, which is cheaper to process.

* Special-case an empty pattern in the source generator

* Output more idiomatic C# for some backreference conditionals

We currently output goto-based code, avoiding an actual if/else construct in case the child branches need to backtrack (and with the forced scoping for the if/else blocks, backtracking would result in compilation failures due to trying to jump to labels defined inside those scopes).  But if we know in advance that no such backtracking will happen, we can output the nicer if/else code.

* Improve code generated for backreferences

We were iterating through each individual character in our own open-coded loop, but for case-sensitive, we can just delegate to SequenceEqual.

* Avoid mutating RegexNode tree for expression conditionals

Rather than wrapping a node in a new positive lookahead assertion node, just call the logic to emit the node as if it were a positive lookahead.

* Change a few -1 comparisons to be 0 comparisons

Ever so slightly cheaper

* Factor stack pushing/popping into helper methods
* load dotnet.js with import in tests
* fix proxy fail and handle errors of this type
* feedback
…the 3 overloads could convert all the argument types` on VS2019 with warning level 4. (dotnet#60967)
When analyzing PR failures I noticed that the log / wrn / err / binlog
combo only gets produced on Unix runs. I think this is also a bug
I made in the build script refactoring, the logging parameters
were used for the cmake build instead of the managed build.
[I can add a second set of logging parameters for the cmake build
phase if needed but I believe it wasn't there before my change.]

Thanks

Tomas
* Fix Some Date and Time parsing cases

* Avoid running the new test on Mac

* Exclude the browser for the new test

* Update src/libraries/System.Private.CoreLib/src/System/Globalization/DateTimeParse.cs

Co-authored-by: Santiago Fernandez Madero <[email protected]>
dotnet#62460)

* Renaming some local variables to make the generated code more readable

* Renaming crawlpos as well.
* Create symlink at build time

* Pick up latest Installers package from Arcade
joperezr and others added 24 commits December 14, 2021 15:25
The issue seems to be caused by a bug in the test. The problematic call
to `TestCreateMutex` is passed `nullptr` as the `name` parameter. It
then calls `convert` helper on it to convert it to wide char. However,
the `convert` helper doesn't check whether it is `nullptr` or not and
ends up returning a pointer to a memory with possibly random data,
that is returned by `malloc(0)`. The returned pointer is then passed to
the CreateMutex PAL api that probably ends up attempting to get the
length of the name or something. And depending on the random data, it
sometimes fails.

The fix is to change the `convert` function to handle `nullptr` so that
it returns `nullptr` too.
This is similar in spirit to the static library for the single file host.

The main different from the single file host one is that we don't want `/GL` because the static library is going to be linked on end user machine and `/GL` is super version fragile.

We build two flavors - with control flow guard and without control flow guard. CFG is a compile time option for end users in NativeAOT and we need static libs that support both.
* [mono] Fix types for arguments to printf calls

   These came up while I was using a code scanning tool.

* Fix build
This partially reverts this change: dotnet#62652

The problem is that although we no longer need to patch, we do need corerun from the clr.hosts subset. Corerun still ends up as part of the pubished clr product artifact, and not in the mono product artifact. And thus we still end up downloading clr, and still need this dependency for now.
We'll probably want to spread these out to the appropriate categories (CoreMangLib, etc.) and enable them for non-NativeAOT scenarios eventually. For now, moving these as they were in runtimelab because being able to build them as a subtree (`build.sh tree nativeaot`) in one go makes things easy.
* Improved ARM64 disassembly by emitting the right register

* Assume INS_OPTS_NONE and INS_OPTS_LSL are 64bit when displaying an extended register
* Fix 62551

* Addressing @radical comments.

* Adding comments to not forget what we were trying to test.
Indenting.
* Update libunwind to v1.6.2

* Apply libunwind changes from 1b5719c

Co-authored-by: Jan Vorlicek <[email protected]>

* Suppress Wincompatible-pointer-types on arm64

* Fix remote unwinding on win-arm64

* Mark base as both input and output

Co-authored-by: Jan Vorlicek <[email protected]>

Co-authored-by: Jan Vorlicek <[email protected]>
Co-authored-by: Jan Vorlicek <[email protected]>
…2725)

Fixes: dotnet#56163

PR dotnet#58523 fixed something on Windows, but it didn't actually address
our issues on Android.

A directory name like `foo Ümläüts` fails:

* `mkdir 'foo Ümläüts' ; cd 'foo Ümläüts'`
* `dotnet new android`
* `dotnet build -c Release -p:RunAOTCompilation=true` (adding
  `-p:EnableLLVM=true` complicates further)

The error:

    Precompiling failed for C:\src\foo Ümläüts\obj\Release\android-arm64\linked\System.Private.CoreLib.dll: Error: Loaded assembly 'C:\src\foo ├£ml├ñ├╝ts\obj\Release\android-arm64\linked\System.Private.CoreLib.dll' doesn't match original file name 'C:\foo ▄mlΣⁿts\obj\Release\android-arm64\linked\System.Private.CoreLib.dll'. Set MONO_PATH to the assembly's location.

Reviewing the existing AOT implementation in Xamarin.Android, I found
out *why* Xamarin.Android works:

    [AOT] response file obj\Release\120\aot\arm64-v8a\App36.dll\response.txt: --llvm "--aot=temp-path=obj\Release\120\aot\arm64-v8a\App36.dll,llvm-path=C:\Program Files\Microsoft Visual Studio\2022\Preview\MSBuild\Xamarin\Android,outfile=obj\Release\120\aot\arm64-v8a\libaot-App36.dll.so,msym-dir=obj\Release\120\aot\arm64-v8a,asmwriter,mtriple=aarch64-linux-android,tool-prefix=C:\Program Files (x86)\Android\android-sdk\ndk-bundle\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android-,ld-name=ld.EXE,ld-flags=\"-LC:\Program Files (x86)\Android\android-sdk\ndk-bundle\toolchains\llvm\prebuilt\windows-x86_64\lib\gcc\aarch64-linux-android\4.9.x\";\"-LC:\Program Files (x86)\Android\android-sdk\ndk-bundle\platforms\android-21\arch-arm64\usr\lib\";\"C:\Program Files (x86)\Android\android-sdk\ndk-bundle\toolchains\llvm\prebuilt\windows-x86_64\lib\gcc\aarch64-linux-android\4.9.x\libgcc.a\";\"C:\Program Files (x86)\Android\android-sdk\ndk-bundle\platforms\android-21\arch-arm64\usr\lib\libc.so\";\"C:\Program Files (x86)\Android\android-sdk\ndk-bundle\platforms\android-21\arch-arm64\usr\lib\libm.so\"" C:\Users\jopepper\source\repos\App36\App36\obj\Release\120\android\assets\shrunk\App36.dll

1. Xamarin.Android passes *relative* paths. The `foo Ümläüts`
   directory name doesn't even come into play for some arguments.

2. With LLVM, `ld-flags` contains a `;`.

The existing code splits on `;` and joins on `,`:

https://github.com/dotnet/runtime/blob/25c207351c4f57cf2daa98caaf327a8b8d83edb8/src/tasks/AotCompilerTask/MonoAOTCompiler.cs#L505-L509

So we lose any `;` delimiters for the `ld-flags` value, they get
replaced by `,`.

I think the solution here is:

1. Add several missing properties to `<MonoAOTCompiler/>` so we don't
   have to rely on the `%(AotArguments)` item metadata. No splitting
   on `;` would be required, `ld-flags` can be passed in and used as-is.

2. Add a new `WorkingDirectory` property. When this is set, assume
   paths passed in might be relative -- and don't transform paths by
   calling `Path.GetFullPath()`.

Lastly, I fixed a place where the UTF8 encoding wasn't passed when
MSBuild logging the response file.

These changes I tried to make in a way where this shouldn't break
other .NET workloads like wasm. If existing MSBuild targets call this
task (not using the new properties), the behavior should remain
unchanged.

I tested these changes by commiting a modified `MonoAOTCompiler.dll`:

dotnet/android#6562

I'm able to enable several AOT tests related to dotnet#56163.
It is not used on arm32/arm64.
* Have more concise check for "HFAs should not be morphed to FieldList when they are not passed on SIMD registers" in src/coreclr/jit/morph.cpp

* Pass HFA/HVA arguments in registers in src/coreclr/jit/morph.cpp
@joshpeterson joshpeterson merged commit 9a08aad into unity-main Dec 16, 2021
@joshpeterson joshpeterson deleted the bot-upstream-main-merge-2021-12-15 branch December 16, 2021 13:27
alexey-zakharov pushed a commit that referenced this pull request Apr 28, 2025
…rious correctness fixes, and stability improvements (dotnet#114927)

* Refactor Tensor to be more reusable and validate appropriate state

* Handle Equals, GreaterThan, GreaterThanOrEqual, LessThan, LessThanOrEqual, and the *All/*Any variants

* Many implementations correctly swapped to new form. (#25)

* Refactor Tensor to be more reusable and validate appropriate state

* finishing tensor primitives work

---------

Co-authored-by: Tanner Gooding <[email protected]>

* more tensors updates (#26)

* Resolve a few build failures

* Ensure SetSlice and ToString are working as expected

* Tensors lastfew (#27)

* only couple left

* pausing for food

* fixed rented buffer

* squeeze/unsqueeze

* set slice/ split

* only 2 left

* Minor cleanup of the Tensor files

* Ensure that tensor tests are building

* Resolving various build failures due to API compatibility

* Ensure flattendLength is adjusted after the stride is set for that dimension

* Ensure that we set linearLength if -1 is passed in when strides is empty

* Ensure that the first index is correct

* Cleanup to ensure iteration and construction initializes correctly

* Ensure that broadcasting is allowed to be in any stride position

* Have AreCompatible handle empty shapes

* Ensure IndexOutOfRangeException is thrown for invalid indexes

* Ensure that the stride is set to 0 when the length of a dimension is 1, so embedded broadcasting works

* Fixing Broadcasting Loop (#29)

* Fixing Broadcasting Loop

* fixes from pr coments

* squeeze fixed

* unsqueeze

* set slice

* more tensor fies

* Ensure that minimumLinearLength is actually the minimum

* Ensure the rented buffer is cleared

* Fix the AreCompatible checks

* Tensor finishing (#30)

* stack working

* more tensor tests working

* fix factory create tests

* only2 tests left

* Update src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorShape.cs

* Update compatibility suppressions

* transpose working

* reverse working

* Revert the unnecessary sln changes

* Remove an unnecessary using

---------

Co-authored-by: Michael Sharp <[email protected]>
Co-authored-by: Michael Sharp <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.