Dotnet Tool Integration

[afscrome]

Is there an existing issue for this?

• I have searched the existing issues

Is your feature request related to a problem? Please describe the problem.

Today aspire supports containers and executables as base resource types. Containers are great for redistributing as docker handles not only the running of containers, but also the acquisition.

Executables however do not handle acquisition, so are only really useful if you've already got the binaries installed. This isn't too bad for things that may be pre-installed on your machine, but for anything else, you need to work out a way to get the executables onto the machine in the first place. These solutions often involve manual installation / configuration instructions 🤢.

This is where dotnet tools come into play as it can acquire tools through nuget, and then execute them. With the .Net 10 SDK's Native AOT tool capabilities, you can even smuggle non .Net native binaries through tools to allow this to be a method to distribute any executable.

Describe the solution you'd like

Ultimately I'd like to see something as simple as .AddDotnetTool("migrations", "dotnet-ef"), along with some extension method for configuring tool properties .WithVersion(), .WithAllowPreRelease() etc. As well as the usual extension methods you'd expect for an executable (args, environment, endpoint etc.)

Additional context

I have been playing around with an implementation of this locally - for one .Net executable, and one native executable. Below are some of the learnings & challenges I've faced that should be considered as part of a formal design. I am including my initial implementation at the end of this post, not as a formal API design proposal (although I'm sure it gets some things right), but rather as a starter for 10 to play with, inform a formal design, and maybe to explain some of my experiences.

🚀 dotnet tool install vs dotnet tool exec

There are few different ways you can run dotnet tools
1. dotnet tool install / dotnet tool run
a. Local tool
b. Global tool
c. Custom path (--tool-path)
2. dotnet tool exec

dotnet tool exec would be the simpler option, however it does require .Net 10, and I've also hit a few issues that are easier to work around with dotnet tool install (see Concurrency & Offline Access). These may ultimately make dotnet tool install the better approach, or could indicate gaps better solved in dotnet tool exec rather than working around them.

For my implementation I chose dotnet tool install with --tool-path. My initial thinking was as follows, alhtough I'll admit it was mostly gut instinct so I could get started with something:

• I avoided dotnet tool exec because I started this prior to .Net 10.
• I avoided local tools as working with the tool manifest seemed more trouble than it was worth.
• I avoided global tools as I didn't want different app hosts using different tool versions to fight each other with up/downgrades.

Leaving me with the --tool-path, although that ended up with it's own set of challenges (See Long Paths).

👯 Concurrency

I have a couple of apps which I need to run multiple instances of, and if aspire starts them as tools at once, the dotnet tool installation process often ends up fighting, causing one to fail with "File in Use" type issues around the nuget packages path, or the tool installation directory.

When using dotnet tool install, I can work around this by including the resource name as a component in --tool-path, or by adding some WaitForCompletions between the tool installers. These options don't work with dotnet tool exec - the former due to a bug, and the latter because dotnet tool is a single process for installation and execution, so you can't tell when the installation is finished.

📵 Offline access

Another challenge I faced was making the tools run if you're offline or have no network connectivity. With docker, you can docker run a container you have in your cache even if you don't have network access. With dotnet tool install, you can somewhat fake this by --add-source $NUGET_PACKAGES --ignore-failed-sources, and then the tool execution will work fine. This same hack doesn't work due to dotnet/sdk#50579, although this does feel like a scenario dotnet tool exec should support natively, rather than having to cheat it in.

(I should also note the story here isn't perfect for containers - see #10719)

⚙️ Non .Net Apps

With the changes in the .Net 10 sdk to support native AOT apps, you can actually include non .Net Apps as tool packages. They're a bit of a pain to create as there's a bit of complexity around having a central package, and then rid specific packages and little documentation, but that is all solvable.

📏 Long Paths

With the dotnet tool install approach, I have hit a few MAX_PATH issues I installed tools to a path under IAspireStore to get a reliable path. However if you add in some "enterprisey" naming schemes for projects and packages, and the path structure used by dotnet tool install, your paths can easily get above 250 characters.

{PATH TO OBJ}\.aspire\tools\{RESOURCE-NAME}\.store\{PACKAGENAME}\0.0.1.12345-alpha\{PACKAGENAME}\0.0.1.12345-pr81923\tools\net8.0\any

The MAX_PATH length issues haven't been common, but I've hit them often enough to be a pain.

I don't believe dotnet tool exec would be affected by these issues. Similar for global tools.

🔀 Substitution

For a few use cases, I would really benefit from being able to swap tool resources with regular executable resourcesor project resources. This would be relatively straight forward if the dotnet tool exec approach was used as everything would be a single resource, but is a bit more complicated when a second installer resoruce is used as you have to remove that, and some wait fors. My current implementation also involves removing an OnBeforeResourceStart event handler, although I'm sure some refactoring can solve that problem.

👨🏻‍💻 An Implementation

Sample

/// <summary>
/// Represents a .NET tool resource that encapsulates metadata about a .NET CLI tool, including its name, package ID,
/// and command.
/// </summary>
/// <remarks>This class is used to define and manage resources for .NET CLI tools. It associates a tool's name and
/// command with its package ID, and ensures that the required metadata is properly annotated.</remarks>
public class DotnetToolResource : ExecutableResource
{
   /// <param name="name">The name of the resource.</param>
   /// <param name="packageId">The package id of the tool</param>
   /// <param name="command">The command to execute.</param>
   public DotnetToolResource(string name, string packageId, string command) 
      : base(name, command, string.Empty)
   {
      ArgumentException.ThrowIfNullOrWhiteSpace(packageId, nameof(packageId));
      Annotations.Add(new DotNetToolAnnotation { PackageId = packageId });
   }

   internal DotNetToolAnnotation ToolConfiguration
   {
      get
      {
         if (!this.TryGetLastAnnotation<DotNetToolAnnotation>(out var toolConfig))
         {
            throw new InvalidOperationException("DotNetToolAnnotation is missing");
         }
         return toolConfig;
      }
   }
}

internal class DotnetToolInstaller(string name, string command) :
   ExecutableResource(name, command, string.Empty)
{
   public required DotnetToolResource Parent { get; init; }
}

internal class DotNetToolAnnotation : IResourceAnnotation
{
   public required string PackageId { get; set; }
   public string? Version { get; set; }
   public bool Prerelease { get; set; } = false;
   public List<string> Sources { get; } = [];
   public bool IgnoreExistingFeeds { get; set; } = false;
   public bool IgnoreFailedSources { get; set; } = false;
   public bool AllowDowngrade { get; set; } = false;
}

public static class DotNetToolExtensions
{
   internal static IResourceBuilder<DotnetToolResource> AddDotnetTool(this IDistributedApplicationBuilder builder, string name, string command, string packageId, Action<IResourceBuilder<DotnetToolResource>> configure)
   {
      var tool = builder.AddDotnetTool(name, command, packageId);
      configure(tool);
      return tool;
   }

   internal static IResourceBuilder<DotnetToolResource> AddDotnetTool(this IDistributedApplicationBuilder builder, string name, string command, string packageId)
      => builder.AddDotnetTool(new DotnetToolResource(name, packageId, command));

   internal static IResourceBuilder<T> AddDotnetTool<T>(this IDistributedApplicationBuilder builder, T resource)
      where T: DotnetToolResource
   {
      var tool = builder.AddResource(resource)
         .WithIconName("Toolbox");

      var installer = BuildInstaller();
      RewriteToolCommand();

      return tool.WaitForCompletion(installer);

      void RewriteToolCommand()
      {
         // .Net 10's `dotnet tool exec` would handle a lot of that natively
         // Although https://github.com/dotnet/sdk/issues/50579 is a complication
         // In the meantime, download tool to a path based on IAspireStore
         //
         // Using BeforeStartEvent rather than BeforeResourceStart as the latter event
         // gets called multiple times, and prepending would break the path
         builder.Eventing.Subscribe<BeforeStartEvent>((evt, ct) =>
         {
            if (Path.IsPathFullyQualified(resource.Command))
            {
               throw new ArgumentException(nameof(builder), "Executable must not have an absolute path to run as a tool");
            }

            var toolDirectory = GetToolDirectory(evt.Services, tool);
            tool.WithCommand(Path.Combine(toolDirectory, resource.Command));

            return Task.CompletedTask;
         });
      }

      IResourceBuilder<DotnetToolInstaller> BuildInstaller()
      {
         var installerResource = new DotnetToolInstaller($"{tool.Resource.Name}-installer", "dotnet") { Parent = tool.Resource };

         return builder
            .AddResource(installerResource)
            .WithArgs(x =>
            {
               var toolDirectory = GetToolDirectory(x.ExecutionContext.ServiceProvider, tool);
               var toolConfig = tool.Resource.ToolConfiguration;

               x.Args.Add("tool");
               x.Args.Add("install");
               x.Args.Add(toolConfig.PackageId);
               x.Args.Add("--tool-path");
               x.Args.Add(toolDirectory);

               var sourceArg = toolConfig.IgnoreExistingFeeds ? "--source" : "--add-source";

               foreach (var source in toolConfig.Sources)
               {
                  x.Args.Add(sourceArg);
                  x.Args.Add(source);
               }

               if (toolConfig.IgnoreFailedSources)
               {
                  x.Args.Add("--ignore-failed-sources");
               }

               if (toolConfig.Version is not null)
               {
                  x.Args.Add("--version");
                  x.Args.Add(toolConfig.Version);
               }
               else if (toolConfig.Prerelease)
               {
                  x.Args.Add("--prerelease");
               }

               if (toolConfig.AllowDowngrade)
               {
                  x.Args.Add("--allow-downgrade");
               }

               x.Args.Add("--verbosity");
               x.Args.Add("detailed");
            })
            .WithIconName("ArrowDownload")
            .WithParentRelationship(tool)
            .WithOfflineFallback();
      }

      string GetToolDirectory(IServiceProvider serviceProvider, IResourceBuilder<DotnetToolResource> tool)
      {
         var builder = tool.ApplicationBuilder;
         var explicitPath = builder.Configuration["OVERRIDE_TOOLBASEPATH"];

         if (!string.IsNullOrEmpty(explicitPath))
         {
            return Path.Combine(explicitPath, builder.Environment.ApplicationName, tool.Resource.Name);
         }
         else
         {
            var store = serviceProvider.GetRequiredService<IAspireStore>();
            return Path.Combine(store.BasePath, "tools", tool.Resource.Name);
         }
      }
   }

   internal static IResourceBuilder<T> WithPackageId<T>(this IResourceBuilder<T> builder, string packageId)
      where T : DotnetToolResource
   {
      builder.Resource.ToolConfiguration.PackageId = packageId;
      return builder;
   }

   /// <summary>
   /// Set the package version for a tool to use
   /// </summary>
   /// <typeparam name="T">The Dotnet Tool resource type</typeparam>
   /// <param name="builder">The <see cref="IDistributedApplicationBuilder"/>.</param>
   /// <param name="version">The package version to use</param>
   /// <returns>The <see cref="IResourceBuilder{T}"/> for chaining.</returns>
   public static IResourceBuilder<T> WithPackageVersion<T>(this IResourceBuilder<T> builder, string version)
      where T : DotnetToolResource
   {
      builder.Resource.ToolConfiguration.Version = version;
      return builder;
   }

   internal static IResourceBuilder<T> WithPackagePrerelease<T>(this IResourceBuilder<T> builder)
      where T : DotnetToolResource
   {
      builder.Resource.ToolConfiguration.Prerelease = true;
      return builder;
   }

   internal static IResourceBuilder<T> WithPackageSource<T>(this IResourceBuilder<T> builder, string source)
      where T : DotnetToolResource
   {
      builder.Resource.ToolConfiguration.Sources.Add(source);
      return builder;
   }

   internal static IResourceBuilder<T> WithPackageIgnoreExistingFeeds<T>(this IResourceBuilder<T> builder)
      where T : DotnetToolResource
   {
      builder.Resource.ToolConfiguration.IgnoreExistingFeeds = true;
      return builder;
   }

   internal static IResourceBuilder<T> WithPackageIgnoreFailedSources<T>(this IResourceBuilder<T> builder)
      where T : DotnetToolResource
   {
      builder.Resource.ToolConfiguration.IgnoreFailedSources = true;
      return builder;
   }

   internal static IResourceBuilder<T> WithPackageAllowDowngrade<T>(this IResourceBuilder<T> builder)
      where T : DotnetToolResource
   {
      builder.Resource.ToolConfiguration.AllowDowngrade = true;
      return builder;
   }

   private static IResourceBuilder<DotnetToolInstaller> WithOfflineFallback(this IResourceBuilder<DotnetToolInstaller> builder)
   {
      return builder.WithArgs(x =>
      {
         var settings = NuGet.Configuration.Settings.LoadDefaultSettings(root: builder.Resource.WorkingDirectory);
         var packagesPath = NuGet.Configuration.SettingsUtility.GetGlobalPackagesFolder(settings);

         builder.ApplicationBuilder.CreateResourceBuilder(builder.Resource.Parent)
            .WithPackageSource(packagesPath)
            .WithPackageIgnoreFailedSources();
      });
   }
}

[baronfel]

For concurrency if simultaneous dnx/dotnet tool exec invocations of the same tool are hitting concurrency errors defintiely raise that on the SDK. We probably should have some kind of named mutex for serializing the parts of that process that are file-manager-y.

For offline access, if the version of the tool is known and specified, and that tool is already available locally, then the dotnet CLI should work offline perfectly well. If that isn't working for you, definitely raise an issue there - the version is how we intend for offline access to work in a one-shot-invocation mode.

Another approach might be for Aspire to create and manage a local tool manifest, and still use dnx/dotnet tool exec to execute the tools. It'll use the versions from the config file if one is present - it's a unified access layer over global and local tools in this way.

[afscrome]

For offline access, if the version of the tool is known and specified, and that tool is already available locally, then the dotnet CLI should work offline perfectly well. If that isn't working for you, definitely raise an issue there - the version is how we intend for offline access to work in a one-shot-invocation mode.

I am specifying a version, although I am specifying it as a wildcard dnx TOOL --version 0.0.1.*-main*. (I'm having to do this because of a slightly crazy versioning scheme we have whereby main builds get 0.0.1-mainXYZ version numbers, but the end of sprint release gets a 1.x.y version number, so my version number really means latest).

Let me play around with the local tool manifest.

[davidfowl]

@afscrome are you going to contribute this one? We can create a copilot PR from your PR and use it as the starting point once you have an implementation that works.

[normj]

This would be useful for the AWS Aspire integrations which relies on AWS .NET Tools we ship.