No-Op scenario hole with floating/non-exact versions

[nkolev92]

Floating/Open ended version requested - Scenario breaking change

Since the No-Op evaluation needs to be minimal, we avoiding checking the sources, so the case where a new version of the requested package is uploaded, or a version of the package is unlisted it might change the package resolution behavior.

Example:

<ItemGroup>
    	<!-- ... -->
    	<PackageReference Include="My.Awesome.Package" Version="3.6.*" />
	<!-- ... -->
</ItemGroup>

At the moment of the first restore, version="3.6.0" was the latest version on the package source.
Sometime between that restore and the new restore, version="3.6.1" was uploaded to the source.
Running a full restore would resolve 3.6.1 and downloaded that version of the package. However since the restore inputs have not changed, restore will think no action needs to be taken.

This might cause issues for scenarios like the following:
A project(ProjX) that depends on a local feed that contains the latest bits of the desired projects (ProjA, ProjB etc).
ProjA is built and the nupkgs from that dropped into the local feed for the ProjX.
ProjX is restored and packed.
The expectation is that the nupkg generated by projX has the latest ProjA nupkg that was built.
However since the restore parameters have not changed, the restore of ProjX might no-op.
The workaround is to simply use the force option for scenarios like this.

Potential improvements

• Make the cache expire, based on a timestamp. The timestamp can be either in the cache or just use the timestamp of when the file was written. This doesn't fix the above scenario though, it makes it less likely.
• Print a warning whenever we no-op on a project that has open ended/ floating dependencies. Caveat is this could become really noisy and users might hate it.

Workarounds

Use -force selectively when you need to update the floating version. (dotnet.exe/nuget.exe)
In Visual Studio, do a rebuild with restore on build enabled. That restore on build is equivalent to force.
//cc
@rohit21agrawal , @emgarten

[mvput]

Currently running into this issue. We are using pre-release packages during development. So project B has a dependency to project A with a floating version: 1.0.0-*. When there is a new version of A on the feed, Visual Studio doesn't always download the latest package. We are using dotnet restore --no-cache to resolve this issue.

Perhaps a solution is to create an option to ignore the cache like the --no-cache inside visual studio when building projects?

[emgarten]

Perhaps a solution is to create an option to ignore the cache like the --no-cache inside visual studio when building projects?

An option in VS would be useful to explicitly force a restore, this could match the --no-cache and --force behavior in dotnet.exe.

Floating versions are useful for keeping packages in sync when you are working across different solutions and repositories. If external projects are updated they get put on a feed, and automatically pulled in.

Unless you just went and made a change that you need and published the external projects you probably don't want to update. For example if you are developing in VS and are debugging a unit test, you don't want restore to quietly pull in new packages between builds/test running if someone else on your team updates something. It would be both slow and confusing.

Command line scenarios are similar in my opinion, and even on a CI machine it seems dangerous to have packages change between the start and end of a build, which could happen through implicit restore.

[mvput]

Yes that can be dangerous, what you want to achieve is while using floating versions you can update packages with the latest version but retain the floating version.

Another option might be to use the package manager to update packages, but to retain the floating version in the csproj file. An update now would change the csproj file to the exact version.

[nkolev92]

@mvput
I just updated the issue for future reference, but in VS there's already a way to "force" restore. If you do a rebuild with restore on build enabled, it will update the floating versions.
There's no "--no-cache" option however, so it might take a while if the source of the updated version is across http.

[gallivantor]

This issue causes some real workflow problems when iterating rapidly over a prerelease package.

Would it not be reasonable for the no-op scenario to always check the feed for a newer version for any floating version references? While this would take a bit longer, it would at least produce the correct result, which is better than a fast process that produces the wrong result regularly

[timotei]

@nkolev92 How are you testing that rebuild scenario? For example, if I do rebuild, NuGet still caches the results from our NuGet server:

  CACHE http://artifactory.lan/artifactory/api/nuget/nuget/FindPackagesById()?id='tools-sample'&semVerLevel=2.0.0
  CACHE http://artifactory.ullink.lan/artifactory/api/nuget/nuget/FindPackagesById()?$skip=80&semVerLevel=2.0.0&id='tools-sample'

Even if I delete the bin/obj it still does that, and I don't see any other way to stop it from caching between rebuilds

PS: This is a .NET framework 4.7 project, using the PackageReference format to specify a wildcard version(i.e., <PackageReference Include="tools-sample"> <Version>1.0.0.1-*</Version> </PackageReference>

Wouldn't it be possible to automatically skip the caches at all on rebuilds?

[nkolev92]

@timotei

This is about the no-op cache. The no-op cache is a collection of info that tells NuGet what were the dependencies this project last requested. It allows NuGet to do quick up-to-date checks to avoid hitting the network unnecessarily.

What you are seeing the http cache. That's a cache that expires every 30 minutes.

Wouldn't it be possible to automatically skip the caches at all on rebuilds?

Personally, I don't think that's a good idea. I don't even think that rebuild should always be a full restore evaluation.
That would lead to a much slower and non-repeatable build.
The fact that rebuild does a full restore(rather than quick up to date check) is very much hidden right now. As floating versions are not the majority of users, eliminating the http caches completely just during rebuild is likely to introduce more noise.

Related:
#3116
#3389
#7198
#6987