Rework UseChatOptions as ConfigureOptions (dotnet#5606)

* Update README to include a section on UseChatOptions

* Rework UseChat/EmbeddingGenerationOptions to always clone

The callbacks now configure the supplied instance.

Author: stephentoub

src/Libraries/Microsoft.Extensions.AI.Abstractions/README.md

@@ -212,6 +212,22 @@ IChatClient client = new ChatClientBuilder()
 Console.WriteLine((await client.CompleteAsync("What is AI?")).Message);
 ```
 
+#### Options
+
+Every call to `CompleteAsync` or `CompleteStreamingAsync` may optionally supply a `ChatOptions` instance containing additional parameters for the operation. The most common parameters that are common amongst AI models and services show up as strongly-typed properties on the type, such as `ChatOptions.Temperature`. Other parameters may be supplied by name in a weakly-typed manner via the `ChatOptions.AdditionalProperties` dictionary.
+
+Options may also be baked into an `IChatClient` via the `ConfigureOptions` extension method on `ChatClientBuilder`. This delegating client wraps another client and invokes the supplied delegate to populate a `ChatOptions` instance for every call. For example, to ensure that the `ChatOptions.ModelId` property defaults to a particular model name, code like the following may be used:
+```csharp
+using Microsoft.Extensions.AI;
+
+IChatClient client = new ChatClientBuilder()
+    .ConfigureOptions(options => options.ModelId ??= "phi3")
+    .Use(new OllamaChatClient(new Uri("http://localhost:11434")));
+
+Console.WriteLine(await client.CompleteAsync("What is AI?")); // will request "phi3"
+Console.WriteLine(await client.CompleteAsync("What is AI?", new() { ModelId = "llama3.1" })); // will request "llama3.1"
+```
+
 #### Pipelines of Functionality
 
 All of these `IChatClient`s may be layered, creating a pipeline of any number of components that all add additional functionality. Such components may come from `Microsoft.Extensions.AI`, may come from other NuGet packages, or may be your own custom implementations that augment the behavior in whatever ways you need.

src/Libraries/Microsoft.Extensions.AI/ChatCompletion/ConfigureOptionsChatClient.cs

@@ -8,67 +8,54 @@
 using System.Threading.Tasks;
 using Microsoft.Shared.Diagnostics;
 
-#pragma warning disable SA1629 // Documentation text should end with a period
-
 namespace Microsoft.Extensions.AI;
 
-/// <summary>A delegating chat client that updates or replaces the <see cref="ChatOptions"/> used by the remainder of the pipeline.</summary>
-/// <remarks>
-/// <para>
-/// The configuration callback is invoked with the caller-supplied <see cref="ChatOptions"/> instance. To override the caller-supplied options
-/// with a new instance, the callback may simply return that new instance, for example <c>_ => new ChatOptions() { MaxTokens = 1000 }</c>. To provide
-/// a new instance only if the caller-supplied instance is <see langword="null"/>, the callback may conditionally return a new instance, for example
-/// <c>options => options ?? new ChatOptions() { MaxTokens = 1000 }</c>. Any changes to the caller-provided options instance will persist on the
-/// original instance, so the callback must take care to only do so when such mutations are acceptable, such as by cloning the original instance
-/// and mutating the clone, for example:
-/// <c>
-/// options =>
-/// {
-///     var newOptions = options?.Clone() ?? new();
-///     newOptions.MaxTokens = 1000;
-///     return newOptions;
-/// }
-/// </c>
-/// </para>
-/// <para>
-/// The callback may return <see langword="null"/>, in which case a <see langword="null"/> options will be passed to the next client in the pipeline.
-/// </para>
-/// <para>
-/// The provided implementation of <see cref="IChatClient"/> is thread-safe for concurrent use so long as the employed configuration
-/// callback is also thread-safe for concurrent requests. If callers employ a shared options instance, care should be taken in the
-/// configuration callback, as multiple calls to it may end up running in parallel with the same options instance.
-/// </para>
-/// </remarks>
+/// <summary>A delegating chat client that configures a <see cref="ChatOptions"/> instance used by the remainder of the pipeline.</summary>
 public sealed class ConfigureOptionsChatClient : DelegatingChatClient
 {
     /// <summary>The callback delegate used to configure options.</summary>
-    private readonly Func<ChatOptions?, ChatOptions?> _configureOptions;
+    private readonly Action<ChatOptions> _configureOptions;
 
-    /// <summary>Initializes a new instance of the <see cref="ConfigureOptionsChatClient"/> class with the specified <paramref name="configureOptions"/> callback.</summary>
+    /// <summary>Initializes a new instance of the <see cref="ConfigureOptionsChatClient"/> class with the specified <paramref name="configure"/> callback.</summary>
     /// <param name="innerClient">The inner client.</param>
-    /// <param name="configureOptions">
-    /// The delegate to invoke to configure the <see cref="ChatOptions"/> instance. It is passed the caller-supplied <see cref="ChatOptions"/>
-    /// instance and should return the configured <see cref="ChatOptions"/> instance to use.
+    /// <param name="configure">
+    /// The delegate to invoke to configure the <see cref="ChatOptions"/> instance. It is passed a clone of the caller-supplied <see cref="ChatOptions"/> instance
+    /// (or a newly-constructed instance if the caller-supplied instance is <see langword="null"/>).
     /// </param>
-    public ConfigureOptionsChatClient(IChatClient innerClient, Func<ChatOptions?, ChatOptions?> configureOptions)
+    /// <remarks>
+    /// The <paramref name="configure"/> delegate is passed either a new instance of <see cref="ChatOptions"/> if
+    /// the caller didn't supply a <see cref="ChatOptions"/> instance, or a clone (via <see cref="ChatOptions.Clone"/> of the caller-supplied
+    /// instance if one was supplied.
+    /// </remarks>
+    public ConfigureOptionsChatClient(IChatClient innerClient, Action<ChatOptions> configure)
         : base(innerClient)
     {
-        _configureOptions = Throw.IfNull(configureOptions);
+        _configureOptions = Throw.IfNull(configure);
     }
 
     /// <inheritdoc/>
     public override async Task<ChatCompletion> CompleteAsync(IList<ChatMessage> chatMessages, ChatOptions? options = null, CancellationToken cancellationToken = default)
     {
-        return await base.CompleteAsync(chatMessages, _configureOptions(options), cancellationToken).ConfigureAwait(false);
+        return await base.CompleteAsync(chatMessages, Configure(options), cancellationToken).ConfigureAwait(false);
     }
 
     /// <inheritdoc/>
     public override async IAsyncEnumerable<StreamingChatCompletionUpdate> CompleteStreamingAsync(
         IList<ChatMessage> chatMessages, ChatOptions? options = null, [EnumeratorCancellation] CancellationToken cancellationToken = default)
     {
-        await foreach (var update in base.CompleteStreamingAsync(chatMessages, _configureOptions(options), cancellationToken).ConfigureAwait(false))
+        await foreach (var update in base.CompleteStreamingAsync(chatMessages, Configure(options), cancellationToken).ConfigureAwait(false))
         {
             yield return update;
         }
     }
+
+    /// <summary>Creates and configures the <see cref="ChatOptions"/> to pass along to the inner client.</summary>
+    private ChatOptions Configure(ChatOptions? options)
+    {
+        options = options?.Clone() ?? new();
+
+        _configureOptions(options);
+
+        return options;
+    }
 }

src/Libraries/Microsoft.Extensions.AI/ChatCompletion/ConfigureOptionsChatClientBuilderExtensions.cs

@@ -12,41 +12,25 @@ namespace Microsoft.Extensions.AI;
 public static class ConfigureOptionsChatClientBuilderExtensions
 {
     /// <summary>
-    /// Adds a callback that updates or replaces <see cref="ChatOptions"/>. This can be used to set default options.
+    /// Adds a callback that configures a <see cref="ChatOptions"/> to be passed to the next client in the pipeline.
     /// </summary>
     /// <param name="builder">The <see cref="ChatClientBuilder"/>.</param>
-    /// <param name="configureOptions">
-    /// The delegate to invoke to configure the <see cref="ChatOptions"/> instance. It is passed the caller-supplied <see cref="ChatOptions"/>
-    /// instance and should return the configured <see cref="ChatOptions"/> instance to use.
+    /// <param name="configure">
+    /// The delegate to invoke to configure the <see cref="ChatOptions"/> instance.
+    /// It is passed a clone of the caller-supplied <see cref="ChatOptions"/> instance (or a newly-constructed instance if the caller-supplied instance is <see langword="null"/>).
     /// </param>
-    /// <returns>The <paramref name="builder"/>.</returns>
     /// <remarks>
-    /// <para>
-    /// The configuration callback is invoked with the caller-supplied <see cref="ChatOptions"/> instance. To override the caller-supplied options
-    /// with a new instance, the callback may simply return that new instance, for example <c>_ => new ChatOptions() { MaxTokens = 1000 }</c>. To provide
-    /// a new instance only if the caller-supplied instance is <see langword="null"/>, the callback may conditionally return a new instance, for example
-    /// <c>options => options ?? new ChatOptions() { MaxTokens = 1000 }</c>. Any changes to the caller-provided options instance will persist on the
-    /// original instance, so the callback must take care to only do so when such mutations are acceptable, such as by cloning the original instance
-    /// and mutating the clone, for example:
-    /// <c>
-    /// options =>
-    /// {
-    ///     var newOptions = options?.Clone() ?? new();
-    ///     newOptions.MaxTokens = 1000;
-    ///     return newOptions;
-    /// }
-    /// </c>
-    /// </para>
-    /// <para>
-    /// The callback may return <see langword="null"/>, in which case a <see langword="null"/> options will be passed to the next client in the pipeline.
-    /// </para>
+    /// This can be used to set default options. The <paramref name="configure"/> delegate is passed either a new instance of
+    /// <see cref="ChatOptions"/> if the caller didn't supply a <see cref="ChatOptions"/> instance, or a clone (via <see cref="ChatOptions.Clone"/>
+    /// of the caller-supplied instance if one was supplied.
     /// </remarks>
-    public static ChatClientBuilder UseChatOptions(
-        this ChatClientBuilder builder, Func<ChatOptions?, ChatOptions?> configureOptions)
+    /// <returns>The <paramref name="builder"/>.</returns>
+    public static ChatClientBuilder ConfigureOptions(
+        this ChatClientBuilder builder, Action<ChatOptions> configure)
     {
         _ = Throw.IfNull(builder);
-        _ = Throw.IfNull(configureOptions);
+        _ = Throw.IfNull(configure);
 
-        return builder.Use(innerClient => new ConfigureOptionsChatClient(innerClient, configureOptions));
+        return builder.Use(innerClient => new ConfigureOptionsChatClient(innerClient, configure));
     }
 }

src/Libraries/Microsoft.Extensions.AI/Embeddings/ConfigureOptionsEmbeddingGenerator.cs

@@ -3,65 +3,41 @@
 
 using System;
 using System.Collections.Generic;
-using System.Runtime.CompilerServices;
 using System.Threading;
 using System.Threading.Tasks;
 using Microsoft.Shared.Diagnostics;
 
-#pragma warning disable SA1629 // Documentation text should end with a period
-
 namespace Microsoft.Extensions.AI;
 
-/// <summary>A delegating embedding generator that updates or replaces the <see cref="EmbeddingGenerationOptions"/> used by the remainder of the pipeline.</summary>
+/// <summary>A delegating embedding generator that configures a <see cref="EmbeddingGenerationOptions"/> instance used by the remainder of the pipeline.</summary>
 /// <typeparam name="TInput">Specifies the type of the input passed to the generator.</typeparam>
 /// <typeparam name="TEmbedding">Specifies the type of the embedding instance produced by the generator.</typeparam>
-/// <remarks>
-/// <para>
-/// The configuration callback is invoked with the caller-supplied <see cref="EmbeddingGenerationOptions"/> instance. To override the caller-supplied options
-/// with a new instance, the callback may simply return that new instance, for example <c>_ => new EmbeddingGenerationOptions() { Dimensions = 100 }</c>. To provide
-/// a new instance only if the caller-supplied instance is <see langword="null"/>, the callback may conditionally return a new instance, for example
-/// <c>options => options ?? new EmbeddingGenerationOptions() { Dimensions = 100 }</c>. Any changes to the caller-provided options instance will persist on the
-/// original instance, so the callback must take care to only do so when such mutations are acceptable, such as by cloning the original instance
-/// and mutating the clone, for example:
-/// <c>
-/// options =>
-/// {
-///     var newOptions = options?.Clone() ?? new();
-///     newOptions.Dimensions = 100;
-///     return newOptions;
-/// }
-/// </c>
-/// </para>
-/// <para>
-/// The callback may return <see langword="null"/>, in which case a <see langword="null"/> options will be passed to the next generator in the pipeline.
-/// </para>
-/// <para>
-/// The provided implementation of <see cref="IEmbeddingGenerator{TInput, TEmbedding}"/> is thread-safe for concurrent use so long as the employed configuration
-/// callback is also thread-safe for concurrent requests. If callers employ a shared options instance, care should be taken in the
-/// configuration callback, as multiple calls to it may end up running in parallel with the same options instance.
-/// </para>
-/// </remarks>
 public sealed class ConfigureOptionsEmbeddingGenerator<TInput, TEmbedding> : DelegatingEmbeddingGenerator<TInput, TEmbedding>
     where TEmbedding : Embedding
 {
     /// <summary>The callback delegate used to configure options.</summary>
-    private readonly Func<EmbeddingGenerationOptions?, EmbeddingGenerationOptions?> _configureOptions;
+    private readonly Action<EmbeddingGenerationOptions> _configureOptions;
 
     /// <summary>
     /// Initializes a new instance of the <see cref="ConfigureOptionsEmbeddingGenerator{TInput, TEmbedding}"/> class with the
-    /// specified <paramref name="configureOptions"/> callback.
+    /// specified <paramref name="configure"/> callback.
     /// </summary>
     /// <param name="innerGenerator">The inner generator.</param>
-    /// <param name="configureOptions">
-    /// The delegate to invoke to configure the <see cref="EmbeddingGenerationOptions"/> instance. It is passed the caller-supplied
-    /// <see cref="EmbeddingGenerationOptions"/> instance and should return the configured <see cref="EmbeddingGenerationOptions"/> instance to use.
+    /// <param name="configure">
+    /// The delegate to invoke to configure the <see cref="EmbeddingGenerationOptions"/> instance. It is passed a clone of the caller-supplied
+    /// <see cref="EmbeddingGenerationOptions"/> instance (or a newly-constructed instance if the caller-supplied instance is <see langword="null"/>).
     /// </param>
+    /// <remarks>
+    /// The <paramref name="configure"/> delegate is passed either a new instance of <see cref="EmbeddingGenerationOptions"/> if
+    /// the caller didn't supply a <see cref="EmbeddingGenerationOptions"/> instance, or a clone (via <see cref="EmbeddingGenerationOptions.Clone"/> of the caller-supplied
+    /// instance if one was supplied.
+    /// </remarks>
     public ConfigureOptionsEmbeddingGenerator(
         IEmbeddingGenerator<TInput, TEmbedding> innerGenerator,
-        Func<EmbeddingGenerationOptions?, EmbeddingGenerationOptions?> configureOptions)
+        Action<EmbeddingGenerationOptions> configure)
         : base(innerGenerator)
     {
-        _configureOptions = Throw.IfNull(configureOptions);
+        _configureOptions = Throw.IfNull(configure);
     }
 
     /// <inheritdoc/>
@@ -70,6 +46,16 @@ public override async Task<GeneratedEmbeddings<TEmbedding>> GenerateAsync(
         EmbeddingGenerationOptions? options = null,
         CancellationToken cancellationToken = default)
     {
-        return await base.GenerateAsync(values, _configureOptions(options), cancellationToken).ConfigureAwait(false);
+        return await base.GenerateAsync(values, Configure(options), cancellationToken).ConfigureAwait(false);
+    }
+
+    /// <summary>Creates and configures the <see cref="EmbeddingGenerationOptions"/> to pass along to the inner client.</summary>
+    private EmbeddingGenerationOptions Configure(EmbeddingGenerationOptions? options)
+    {
+        options = options?.Clone() ?? new();
+
+        _configureOptions(options);
+
+        return options;
     }
 }

src/Libraries/Microsoft.Extensions.AI/Embeddings/ConfigureOptionsEmbeddingGeneratorBuilderExtensions.cs

@@ -12,45 +12,30 @@ namespace Microsoft.Extensions.AI;
 public static class ConfigureOptionsEmbeddingGeneratorBuilderExtensions
 {
     /// <summary>
-    /// Adds a callback that updates or replaces <see cref="EmbeddingGenerationOptions"/>. This can be used to set default options.
+    /// Adds a callback that configures a <see cref="EmbeddingGenerationOptions"/> to be passed to the next client in the pipeline.
     /// </summary>
     /// <typeparam name="TInput">Specifies the type of the input passed to the generator.</typeparam>
     /// <typeparam name="TEmbedding">Specifies the type of the embedding instance produced by the generator.</typeparam>
     /// <param name="builder">The <see cref="EmbeddingGeneratorBuilder{TInput, TEmbedding}"/>.</param>
-    /// <param name="configureOptions">
-    /// The delegate to invoke to configure the <see cref="EmbeddingGenerationOptions"/> instance. It is passed the caller-supplied
-    /// <see cref="EmbeddingGenerationOptions"/> instance and should return the configured <see cref="EmbeddingGenerationOptions"/> instance to use.
+    /// <param name="configure">
+    /// The delegate to invoke to configure the <see cref="EmbeddingGenerationOptions"/> instance. It is passed a clone of the caller-supplied
+    /// <see cref="EmbeddingGenerationOptions"/> instance (or a newly-constructed instance if the caller-supplied instance is <see langword="null"/>).
     /// </param>
-    /// <returns>The <paramref name="builder"/>.</returns>
     /// <remarks>
-    /// <para>
-    /// The configuration callback is invoked with the caller-supplied <see cref="EmbeddingGenerationOptions"/> instance. To override the caller-supplied options
-    /// with a new instance, the callback may simply return that new instance, for example <c>_ => new EmbeddingGenerationOptions() { Dimensions = 100 }</c>. To provide
-    /// a new instance only if the caller-supplied instance is <see langword="null"/>, the callback may conditionally return a new instance, for example
-    /// <c>options => options ?? new EmbeddingGenerationOptions() { Dimensions = 100 }</c>. Any changes to the caller-provided options instance will persist on the
-    /// original instance, so the callback must take care to only do so when such mutations are acceptable, such as by cloning the original instance
-    /// and mutating the clone, for example:
-    /// <c>
-    /// options =>
-    /// {
-    ///     var newOptions = options?.Clone() ?? new();
-    ///     newOptions.Dimensions = 100;
-    ///     return newOptions;
-    /// }
-    /// </c>
-    /// </para>
-    /// <para>
-    /// The callback may return <see langword="null"/>, in which case a <see langword="null"/> options will be passed to the next generator in the pipeline.
-    /// </para>
+    /// This can be used to set default options. The <paramref name="configure"/> delegate is passed either a new instance of
+    /// <see cref="EmbeddingGenerationOptions"/> if the caller didn't supply a <see cref="EmbeddingGenerationOptions"/> instance, or
+    /// a clone (via <see cref="EmbeddingGenerationOptions.Clone"/>
+    /// of the caller-supplied instance if one was supplied.
     /// </remarks>
-    public static EmbeddingGeneratorBuilder<TInput, TEmbedding> UseEmbeddingGenerationOptions<TInput, TEmbedding>(
+    /// <returns>The <paramref name="builder"/>.</returns>
+    public static EmbeddingGeneratorBuilder<TInput, TEmbedding> ConfigureOptions<TInput, TEmbedding>(
         this EmbeddingGeneratorBuilder<TInput, TEmbedding> builder,
-        Func<EmbeddingGenerationOptions?, EmbeddingGenerationOptions?> configureOptions)
+        Action<EmbeddingGenerationOptions> configure)
         where TEmbedding : Embedding
     {
         _ = Throw.IfNull(builder);
-        _ = Throw.IfNull(configureOptions);
+        _ = Throw.IfNull(configure);
 
-        return builder.Use(innerGenerator => new ConfigureOptionsEmbeddingGenerator<TInput, TEmbedding>(innerGenerator, configureOptions));
+        return builder.Use(innerGenerator => new ConfigureOptionsEmbeddingGenerator<TInput, TEmbedding>(innerGenerator, configure));
     }
 }