Web app user experience is broken

[dseefeld]

When doing a smoke test on the built source-build cli, the user experience for building a web app is broken.

Steps to repro:

dotnet new web
dotnet restore
dotnet build
dotnet run

The results of dotnet run are:

Using launch settings from /src/tarball-output/testing-smoke/C#_web/Properties/launchSettings.json...
It was not possible to find any compatible framework version
The specified framework 'Microsoft.AspNetCore.App', version '2.1.0-preview2-final' was not found.
  - Check application dependencies and target a framework version installed at:
      /src/tarball-output/testing-smoke/builtCli/
  - Installing .NET Core prerequisites might help resolve this problem:
      http://go.microsoft.com/fwlink/?LinkID=798306&clcid=0x409
  - The .NET Core framework and SDK can be installed from:
      https://aka.ms/dotnet-download

This message seems to indicate that I need to download Microsoft.AspNetCore.App, but I don't see that it's available separately. It is only available if you download dotnet core.

There should be some way to acquire just the aspnet bits.

[dseefeld]

cc @natemcmaster

[natemcmaster]

ASP.NET Core is not source-buildable yet, so this is expected #375. https://aka.ms/dotnet-download doesn't show Microsoft.AspNetCore.App yet because we haven't RTM-ed the first version. Presumably the instructions look similar to https://www.microsoft.com/net/download/linux-package-manager/fedora26/runtime-2.1.0-preview2, but install the aspnetcore-runtime-2.1.0-preview2-final package.

cc @leecow @rowanmiller - it looks like ASP.NET Core runtime installers are not yet listed on https://www.microsoft.com/net/download/dotnet-core/runtime-2.1.0-preview2. How can we add a section for this?

[rowanmiller]

Downloads are driven by https://github.com/dotnet/core/blob/master/release-notes/releases.json. Work with @leecow to get the missing entries added.

[omajid]

@natemcmaster We are planning on using source-build on RHEL. There are are a few major issues with the suggestions that I can see.

1. We can't ask users to use a non-RHEL repository to have a working ASP.NET Core setup. If we are asking users to do that, why not just ask them to use all the packages from the Microsoft repository? Same issues applies for anyone else trying to use source-build on any Linux distribution.

2. What about dependencies? Most packages from the Microsoft RPM repo have dependencies on other Microsoft packages. Is installing just ASP.NET Core runtime package (without installing the Microsoft-built RPMs) even an option? For any distribution not supported by Microsoft, it looks like using ASP.NET Core may not even work.

3. The packages at the microsoft repository will install the ASP.NTE Core shared runtime at /usr/share/dotnet/.., while our installation of .NET Core will be somewhere under /opt/... Our installation will not be able to make use of the ASP.NET Core packages installed that way.

cc @aslicerh @tmds @Bob-Davis It looks like ASP.NET Core will not work out of the box from source-build. And will require users to install extra bits by hand, to possibly work at all.

[rowanmiller]

@natemcmaster was just looking at the file and I see the entries there and the corresponding links on the site. Are there some that are missing?

[tmds]

We can't ask users to use a non-RHEL repository to have a working ASP.NET Core setup.

That would completely break .NET Core support in OpenShift.

AFAIK this is supposed to work similar as 2.0 for source-build binaries, that is: the package comes from nuget.org when the user restores his app (https://www.nuget.org/packages/Microsoft.AspNetCore.App/2.1.0-preview2-final)

[natemcmaster]

@rowanmiller yes, we're missing a few download links on the list. We also have Alpine Linux .tar.gz file downloads. Is https://github.com/dotnet/core the best place to file an issue for this? Also, sorry if I wasn't clear, but I was trying to ask how we can get this page to show that users need to run apt-get install aspnetcore-runtime-$version (and likewise for yum, dnf, etc.) https://www.microsoft.com/net/download/linux-package-manager/fedora26/runtime-2.1.0-preview2. Right now this page only shows instructions for apt-get install dotnet-runtime-$version.

@omajid may we should start a mail thread on this subject. ASP.NET Core has never been source-buildable, and as described in #375, it is considerable effort to enable it. We don't have that work scheduled for 2.1.0 RTM. If I remember right, you worked around this in 2.0.x by disabling the ASP.NET Core runtime store. We'll have to investigate if a similar workaround is available now that we are shipping aspnet as a shared framework.

cc @muratg @DamianEdwards

[Bob-Davis]

I would echo what @tmds stated - the expectation here is that it would work how 2.0 worked, and pull the package(s) from nuget.

This sounds like this point release is breaking the expected functionality.
Am I misunderstanding this?

[natemcmaster]

To be clear, what I meant by

If I remember right, you worked around this in 2.0.x by disabling the ASP.NET Core runtime store. We'll have to investigate if a similar workaround is available now that we are shipping aspnet as a shared framework.

is that in 2.0, this was the story.

	Source built installation	Default installations
NuGet Packages	NuGet.org	NuGet.org
ASP.NET Core runtime binaries	In app's publish folder (via opt-ing out of the runtime store trimming)	ASP.NET Core Runtime Package Store

In 2.1, the issue at hand is this

	Source built installation	Default installations
NuGet Packages	NuGet.org (no change)	NuGet.org
ASP.NET Core runtime binaries	???? (need to determine how to make this app local)	ASP.NET Core Shared Runtime Framework

[tmds]

???? (need to determine how to make this app local)

Yes, we are missing the equivalent of 2.0 'PublishWithAspNetCoreTargetManifest=false'.

[DamianEdwards]

The shared framework can be opted out of by having the templates explicitly set Microsoft.NETCore.App as the platform package.

[tmds]

The shared framework can be opted out of by having the templates explicitly set Microsoft.NETCore.App as the platform package.

@DamianEdwards how do you csproj that?

[natemcmaster]

Presumably you can do that by setting this:

  <PropertyGroup>
    <MicrosoftNETPlatformLibrary>Microsoft.NETCore.App</MicrosoftNETPlatformLibrary>
  </PropertyGroup>

Disclaimer: haven't tried it myself.

[tmds]

And setting MicrosoftNETPlatformLibrary makes those assemblies part of the published application?

Would it be possible to make this a single property we can set (like we were setting PublishWithAspNetCoreTargetManifest), since in our scripts we don't know if the user is targeting Microsoft.NETCore.App or Microsoft.NETCore.All.

[natemcmaster]

And setting MicrosoftNETPlatformLibrary makes those assemblies part of the published application?

That's the theory. It instructs the SDK which NuGet package represents the shared framework.

Would it be possible to make this a single property we can set

I believe <MicrosoftNETPlatformLibrary>Microsoft.NETCore.App</MicrosoftNETPlatformLibrary> is the default value for all projects using Microsoft.NET.Sdk. But if you find setting this unconditionally causes issues, let me know.

[tmds]

But if you find setting this unconditionally causes issues, let me know.

We are unconditionally setting PublishWithAspNetCoreTargetManifest. And our users can be using anything with this setting. For 2.1, they may be using Microsoft.NETCore.App or Microsoft.NETCore.All or their application may even not be ASP.NET Core based webapp. All these cases should work. I will do some experimentation. If you know some of these things may not work/have unexpected side-effects, we should look into something that works more similar like PublishWithAspNetCoreTargetManifest.

My use-case is focusing on the publish command. The issue report is on the run command, which we like to work out-of-the-box too on source-built .NET Core.

[tmds]

I believe Microsoft.NETCore.App is the default value for all projects using Microsoft.NET.Sdk. But if you find setting this unconditionally causes issues, let me know.

@natemcmaster Passing /p:MicrosoftNETPlatformLibrary=Microsoft.NETCore.App to dotnet publish works! It is a suitable replacement for 2.0 /p:PublishWithAspNetCoreTargetManifest=false.

Any thoughts on how we can make dotnet run work out-of-the-box on source-build .NET Core? For 2.0 this worked without making any changes to the project file or passing additional parameters.

[leecow]

@livarcocc and @KathleenDollard for thoughts on dotnet run

[natemcmaster]

Any thoughts on how we can make dotnet run work out-of-the-box on source-build .NET Core?

There are a range of options, from ones requiring explicit user action to overriding the default. My preference would be to add a condition to detect when the aspnet sharedfx is unavailable.

• Explicit user choice: aspnet adds an option to templates such as dotnet new web --no-shared-fx so new projects contain the override for MicrosoftNETPlatformLibrary. Source-build users are expected to use this option
• Detect source-built SDK: ASP.NET could honor some conditions which a source-build SDK could set differently from a normal build of the SDK.

<!-- In the source-build SDK -->
  <_AspNetSharedFxIsAvailable>false</_AspNetSharedFxIsAvailable>

<!-- In Microsoft.AspNetCore.App.nupkg -->
  <MicrosoftNETPlatformLibrary Condition="'$(_AspNetSharedFxIsAvailable)' != 'false'">Microsoft.AspNetCore.App</MicrosoftNETPlatformLibrary>

• Override the SDK: Source build adds properties to Microsoft.NET.Sdk to override MicrosoftNETPlatformLibrary, even when set via PackageReference to Microsoft.AspNetCore.App

Thoughts @JunTaoLuo @DamianEdwards @muratg ?

[DamianEdwards]

How would that work if the customer then installs the MS-produced shared FX?

[natemcmaster]

If we went with the "Detect source-built SDK" option, we could make detection smart enough to know when Microsoft.AspNetCore.App is installed. But even if we didn't, user's apps will still run...they will just be sub-optimal and bulky.

[tmds]

@natemcmaster thank you for providing some options.

If we went with the "Detect source-built SDK" option, we could make detection smart enough to know when Microsoft.AspNetCore.App is installed.

That is the best option.

But even if we didn't, user's apps will still run...they will just be sub-optimal and bulky.

And imo this is also a good option.

[tmds]

@DamianEdwards, @natemcmaster, what option are we going for?

[natemcmaster]

We haven't made a decision yet. I think we have scheduled time to discuss this and other aspnet related things tomorrow.

[tmds]

Thanks for the update!

[tmds]

I just realized the lack of a shared framework also impacts global tools.
A tool like dotnet-serve has a dependency on the ASP.NET Core shared framework. So it won't run on source-build .NET Core.

I've checked the tools that ship out-of-the-box with Microsoft sdk (watch, dev-certs, user-secrets, sql-cache, ef). These all depend on 'Microsoft.NETCore.App', so they should work on source-build (to be verified).

I don't think it is feasible to solve this for 2.1 in general. We definitely like tools that come with the Microsoft installation to be able to work out-of-the-box on source-build (after manual dotnet tool install). This should be the case already.

Side-note: the dotnet-ef package wasn't published on nuget.org yet, the other packages are still at preview1.

[natemcmaster]

The tools that you listed (watch, dev-certs, user-secrets, sql-cache, ef) will actually ship in-box with the SDK. They only shipped to nuget.org for preview1, but were then moved into the SDK (see aspnet/Announcements#290). That means for preview2 and rc1, those tools are not available in source-build. This is an important part of the aspnet story on source-build we need to fix. I was going to bring up in our meeting in a few hours, but I'll share my thinking now anyways.

Two options:

• Getting all of aspnet supported in source-build is too big a task to accomplish in the 2.1.0 timeframe (see Support the Microsoft.AspNetCore.App shared runtime in source-build #375 ). However, pieces of aspnet could be made to build in source-build. We could start with the most used global tools first like watch and ef. I can't commit to the work required, but I think it's feasible.
• Alternative: ship the tools to nuget.org as packages. Source-build users acquire them via dotnet tool install. On our end, we need to change these tools to support shipping to nuget.org. This will take much less work than getting these into source build, however, IMO its a less desirable solution. Pros: unblock source-build users. Cons: requires manual install, the presence of these packages on nuget.org may confuse users about how they are expected to acquire and use them.

[tmds]

@natemcmaster afaik it was already decided we'd consume these from nuget.org. So my side-note was mostly to point out there are no preview2 packages for the tools and there is no dotnet-ef tool. If some of this can be in source-build, that would be nice off course.

[natemcmaster]

We had another meeting today and I think we can squeeze in a few last minute changes. They won't be in 2.1.0-rc1, but we believe we can land them before 2.1.0.

The planned work is described here aspnet/Universe#1125 and here aspnet/Universe#1126. In a nutshell, we'll disable the shared framework trimming in the SDK and ship global tools to nuget.org so they are available via dotnet tool install. Our hope is this will mitigate the biggest issues in the web experience when using a source-build SDK. Please take a look at the proposals and let me know if you have questions or concerns.

[tmds]

@natemcmaster Great!! This will give the best experience for source-build .NET Core users to deal with 'missing bits'!

[natemcmaster]

I found something I don't understand about source-build. Does dotnet new web actually work on a source-built SDK? If so, where is the web template coming from?

We deliver the web template to the CLI via the Microsoft.DotNet.Web.ProjectTemplates.2.1 package, which is built with the rest of the aspnetcore runtime and shouldn't be available during source-build.

cc @dseefeld

[dseefeld]

@natemcmaster Yes, dotnet new web does work. The Microsoft.DotNet.Web.ProjectTemplates.2.1 package is included as a prebuilt package, since we don't build any aspnet assets as part of source-build, yet.

[natemcmaster]

Interesting...I thought prebuilt packages weren't allowed. What are the rules for including prebuild packages? Any reason we couldn't include other packages too, like the ones that contain aspnet console tools?

[DamianEdwards]

My guess is because the templates package doesn't actually include any binaries, just text files, so it's essentially just source.

[tmds]

For 2.0 we are in fact unzipping the prebuild package and patching the templates (adding PublishWithAspNetCoreTargetManifest=false).
Ideally everything is built from source and those sources can just be patched.

[natemcmaster]

Ok, I've completed all the work we are planning to do for 2.1.0. This won't be in rc1, but should be in the rtm build. See aspnet/Universe#1131, aspnet/Universe#1130, and dotnet/cli#9178.

I believe there is at least one other item affecting global tools acquisition, but let's use https://github.com/dotnet/cli/issues/9114 to track that. That's not specific to aspnet tools.

This should solve the scenario as described in the original issue. If it doesn't, we can re-open for more investigation.