Add pre-built expression tree caching for IQueryable operations (#234)

* Initial plan

* Add pre-built expression tree configuration and implementation

- Added LinqraftUsePrebuildExpression configuration option (default: false)
- Created ExpressionTreeBuilder helper class for generating cached expression fields
- Updated SelectExprInfoAnonymous to use pre-built expressions when enabled
- Updated SelectExprInfoNamed to use pre-built expressions when enabled
- Updated SelectExprInfoExplicitDto to use pre-built expressions when enabled
- Pre-built expressions only apply to IQueryable (not IEnumerable)
- Pre-built expressions are disabled when captures are used

Co-authored-by: arika0093 <[email protected]>

* Refactor static field generation to class level

- Moved static field generation from method body to class level
- Added GenerateStaticFields() virtual method to SelectExprInfo
- Updated SelectExprGroups to collect and pass static fields
- Updated GenerateSourceCodeSnippets to output static fields before methods
- Fixed null reference warning in SelectExprGroups

Co-authored-by: arika0093 <[email protected]>

* Complete pre-built expression tree implementation

- Fixed CompilerVisibleProperty list to include LinqraftUsePrebuildExpression
- Anonymous types cannot use pre-built expressions (result type is compiler-generated)
- Pre-built expressions only work for named/predefined/explicit DTO types
- Added comprehensive tests for IQueryable with different DTO patterns
- All tests passing with pre-built expressions enabled

Co-authored-by: arika0093 <[email protected]>

* Add LinqraftUsePrebuildExpression to all CompilerVisibleProperty lists

- Updated examples/Directory.Build.props with new configuration property
- Fixed playground/Services/CodeGenerationService.cs to use updated API signature
- All projects now support the new LinqraftUsePrebuildExpression configuration

Co-authored-by: arika0093 <[email protected]>

* Add LinqraftUsePrebuildExpression property to project files

* Fix code generation issues per review feedback

- Use 'var' instead of explicit type in expression initialization (line 93)
- Remove unnecessary '!' null-forgiving operator when fieldName is not null
- Fix indentation of generated multi-line lambda expressions
- Use explicit Expression<> type instead of 'as' operator (lambdas cannot use 'as')

Co-authored-by: arika0093 <[email protected]>

* Refactor to use direct expression initialization per review feedback

- Updated documentation in global-properties.md for LinqraftUsePrebuildExpression
- Added note about captures not supporting pre-built expressions
- Refactored ExpressionTreeBuilder to directly initialize expression fields (no lazy init)
- Removed GenerateNamedExpressionTreeInitialization and GenerateAnonymousExpressionTreeInitialization
- Expression trees now initialized at field declaration, eliminating null checks
- Updated SelectExprInfoNamed and SelectExprInfoExplicitDto to use new approach
- Added test for multiple DTO types to verify pre-built expressions work correctly
- All 6 tests passing

Co-authored-by: arika0093 <[email protected]>

* Address final review feedback: readonly, indentation, remove test, add playground toggle

- Added 'readonly' keyword to static expression fields for immutability
- Fixed indentation: ensured proper line ending with AppendLine for multi-line expressions
- Deleted Linqraft.Tests.Configuration project as requested
- Removed project reference from Linqraft.slnx
- Added toggle for LinqraftUsePrebuildExpression in playground Settings section
- All tests passing (130/130)

Co-authored-by: arika0093 <[email protected]>

* revert: reset Linqraft.Tests.Configuration

* Fix indentation in BuildExprCodeSnippets for static fields

* Add static fields generation to expression code snippets

* formatt

---------

Co-authored-by: copilot-swe-agent[bot] <[email protected]>
Co-authored-by: arika0093 <[email protected]>
Co-authored-by: Arika Ishinami <[email protected]>
(cherry picked from commit 1761753)

Author: Copilot

docs/library/global-properties.md

@@ -30,6 +30,11 @@ Linqraft supports several MSBuild properties to customize code generation global
     <!-- Use hash-named namespace for nested DTOs (e.g., LinqraftGenerated_HASH.ItemsDto) -->
     <!-- When false, uses hash-suffixed class names (e.g., ItemsDto_HASH) -->
     <LinqraftNestedDtoUseHashNamespace>true</LinqraftNestedDtoUseHashNamespace>
+
+    <!-- Pre-build and cache expression trees for IQueryable operations -->
+    <!-- Improves performance by avoiding repeated lambda-to-expression-tree conversion -->
+    <!-- Only applies to IQueryable with named/predefined/explicit DTO types (not anonymous types) -->
+    <LinqraftUsePrebuildExpression>false</LinqraftUsePrebuildExpression>
   </PropertyGroup>
 </Project>
 ```
@@ -115,6 +120,40 @@ See [Array Nullability Removal](./array-nullability.md) for details.
 
 See [Nested DTO Naming](./nested-dto-naming.md) for details.
 
+### LinqraftUsePrebuildExpression
+
+Pre-build and cache expression trees for IQueryable operations to improve performance:
+
+```xml
+<LinqraftUsePrebuildExpression>true</LinqraftUsePrebuildExpression>
+```
+
+When enabled, expression trees are constructed at compile-time and cached as static fields, avoiding the overhead of repeated lambda-to-expression-tree conversion at runtime.
+
+**Important Notes:**
+- Only applies to IQueryable operations (not IEnumerable)
+- Only works with named, predefined, or explicit DTO types (not anonymous types)
+- Not applied when there are capture variables in the lambda expression
+
+**Example:**
+
+```csharp
+// Without pre-built expressions (default):
+// Expression tree is created on every call
+var results = dbContext.Orders
+    .AsQueryable()
+    .SelectExpr(o => new OrderDto { Id = o.Id, Name = o.Name })
+    .ToList();
+
+// With LinqraftUsePrebuildExpression=true:
+// Expression tree is created once and cached
+// Subsequent calls reuse the same expression tree
+```
+
+**Performance Benefit:**
+
+Eliminates the runtime overhead of converting lambda expressions to expression trees for IQueryable operations. The expression tree is built once at the field declaration and reused for all invocations.
+
 ## Viewing Generated Code
 
 To inspect the generated code:

examples/Directory.Build.props

@@ -27,5 +27,6 @@
     <CompilerVisibleProperty Include="LinqraftCommentOutput" />
     <CompilerVisibleProperty Include="LinqraftArrayNullabilityRemoval" />
     <CompilerVisibleProperty Include="LinqraftNestedDtoUseHashNamespace" />
+    <CompilerVisibleProperty Include="LinqraftUsePrebuildExpression" />
   </ItemGroup>
 </Project>

examples/Linqraft.Benchmark/Linqraft.Benchmark.csproj

@@ -4,6 +4,9 @@
     <Nullable>enable</Nullable>
     <ImplicitUsings>enable</ImplicitUsings>
     <NoWarn>$(NoWarn);NU1608</NoWarn>
+    <LinqraftUsePrebuildExpression>true</LinqraftUsePrebuildExpression>
+    <EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
+    <CompilerGeneratedFilesOutputPath>.generated</CompilerGeneratedFilesOutputPath>
   </PropertyGroup>
   <ItemGroup>
     <PackageReference Include="BenchmarkDotNet" Version="*" />

playground/Components/Playground/Sidebar.razor

@@ -154,6 +154,16 @@
                                @onchange="@(e => UpdateConfig(c => c with { NestedDtoUseHashNamespace = (bool)(e.Value ?? false) }))" />
                         <label for="nestedDtoUseHashNamespace" class="cursor-pointer text-xs text-gray-300">Nested DTO Use Hash Namespace</label>
                     </div>
+                    
+                    <!-- Use Prebuild Expression -->
+                    <div class="flex cursor-pointer items-center gap-2">
+                        <input type="checkbox" 
+                               id="usePrebuildExpression"
+                               class="h-3 w-3"
+                               checked="@Configuration.UsePrebuildExpression"
+                               @onchange="@(e => UpdateConfig(c => c with { UsePrebuildExpression = (bool)(e.Value ?? false) }))" />
+                        <label for="usePrebuildExpression" class="cursor-pointer text-xs text-gray-300">Pre-build Expression Trees</label>
+                    </div>
                 </div>
             }
         </div>

playground/Services/CodeGenerationService.cs

@@ -78,14 +78,15 @@ public GeneratedOutput GenerateOutput(
                         info.Invocation.SyntaxTree
                     );
                     var location = semanticModel.GetInterceptableLocation(info.Invocation);
+                    var fields = info.GenerateStaticFields();
                     var selectExprCodes = info.GenerateSelectExprCodes(location!);
                     var dtoClasses = info.GenerateDtoClasses()
                         .GroupBy(c => c.FullName)
                         .Select(g => g.First())
                         .ToList();
 
                     queryExpressionBuilder.AppendLine(
-                        GenerateSourceCodeSnippets.BuildExprCodeSnippets(selectExprCodes)
+                        GenerateSourceCodeSnippets.BuildExprCodeSnippets(selectExprCodes, [fields])
                     );
                     dtoClassBuilder.AppendLine(
                         GenerateSourceCodeSnippets.BuildDtoCodeSnippetsGroupedByNamespace(

src/Linqraft.Core/ExpressionTreeBuilder.cs

@@ -0,0 +1,63 @@
+using System;
+using System.Text;
+
+namespace Linqraft.Core;
+
+/// <summary>
+/// Helper class for generating pre-built Expression Tree code
+/// This generates static cached expression fields with direct initialization,
+/// avoiding the overhead of building expression trees at runtime for IQueryable operations.
+/// </summary>
+public static class ExpressionTreeBuilder
+{
+    /// <summary>
+    /// Generates a static cached expression tree field with direct initialization
+    /// </summary>
+    /// <param name="sourceTypeFullName">The fully qualified source type name</param>
+    /// <param name="resultTypeFullName">The fully qualified result type name (DTO type)</param>
+    /// <param name="lambdaParameterName">The lambda parameter name</param>
+    /// <param name="lambdaBody">The lambda body code</param>
+    /// <param name="uniqueId">A unique identifier for this expression to avoid collisions</param>
+    /// <returns>A tuple of (fieldDeclaration, fieldName)</returns>
+    public static (string FieldDeclaration, string FieldName) GenerateExpressionTreeField(
+        string sourceTypeFullName,
+        string resultTypeFullName,
+        string lambdaParameterName,
+        string lambdaBody,
+        string uniqueId
+    )
+    {
+        // Generate a unique field name based on the hash
+        var hash = HashUtility.GenerateSha256Hash(uniqueId).Substring(0, 8);
+        var fieldName = $"_cachedExpression_{hash}";
+
+        // Build the expression tree field with direct initialization
+        var sb = new StringBuilder();
+
+        // The lambda body may contain newlines, so we need to properly handle multi-line expressions
+        var lines = lambdaBody.Split(new[] { "\r\n", "\r", "\n" }, StringSplitOptions.None);
+        var firstLine =
+            $"private static readonly Expression<Func<{sourceTypeFullName}, {resultTypeFullName}>> {fieldName} = {lambdaParameterName} =>";
+        if (lines.Length == 1)
+        {
+            // Single line lambda body
+            sb.AppendLine($"{firstLine} {lambdaBody};");
+        }
+        else
+        {
+            // Multi-line lambda body - need to format it properly
+            sb.AppendLine(firstLine);
+            // Indent the lambda body properly - it should be at the same level as the lambda parameter
+            // The field declaration starts at column 4 (after "    ")
+            // We want the body to align after "= " which is at column 4 + "private static readonly ... = ".length
+            // For simplicity and consistency, indent by 2 additional levels from the field level
+            var indentedBody = Formatting.CodeFormatter.IndentCode(
+                lambdaBody.TrimEnd(),
+                Formatting.CodeFormatter.IndentSize
+            );
+            sb.AppendLine(indentedBody + ";");
+        }
+
+        return (sb.ToString(), fieldName);
+    }
+}

src/Linqraft.Core/GenerateSourceCodeSnippets.cs

@@ -20,11 +20,12 @@ public static void ExportAll(IncrementalGeneratorPostInitializationContext conte
     // Generate total code with DTOs that may have different namespaces.
     public static string BuildCodeSnippetAll(
         List<string> expressions,
+        List<string> staticFields,
         List<GenerateDtoClassInfo> dtoClassInfos,
         LinqraftConfiguration configuration
     )
     {
-        var exprPart = BuildExprCodeSnippets(expressions);
+        var exprPart = BuildExprCodeSnippets(expressions, staticFields);
         var dtoPart = BuildDtoCodeSnippetsGroupedByNamespace(dtoClassInfos, configuration);
         return $$"""
             {{GenerateCommentHeaderPart()}}
@@ -35,8 +36,16 @@ LinqraftConfiguration configuration
     }
 
     // Generate expression part
-    public static string BuildExprCodeSnippets(List<string> expressions)
+    public static string BuildExprCodeSnippets(List<string> expressions, List<string> staticFields)
     {
+        var fieldsPart =
+            staticFields.Count > 0
+                ? CodeFormatter.IndentCode(
+                    string.Join(CodeFormatter.DefaultNewLine, staticFields),
+                    CodeFormatter.IndentSize * 2
+                ) + CodeFormatter.DefaultNewLine
+                : "";
+
         var indentedExpr = CodeFormatter.IndentCode(
             string.Join(CodeFormatter.DefaultNewLine, expressions),
             CodeFormatter.IndentSize * 2
@@ -47,7 +56,7 @@ namespace Linqraft
             {
                 file static partial class GeneratedExpression
                 {
-            {{indentedExpr}}
+            {{fieldsPart}}{{indentedExpr}}
                 }
             }
             """;
@@ -275,6 +284,7 @@ private static string GenerateCommentHeaderPart()
     private const string GenerateHeaderUsingPart = """
         using System;
         using System.Linq;
+        using System.Linq.Expressions;
         using System.Collections.Generic;
         """;
 

src/Linqraft.Core/LinqraftConfiguration.cs

@@ -16,6 +16,8 @@ public record LinqraftConfiguration
         "build_property.LinqraftArrayNullabilityRemoval";
     const string LinqraftNestedDtoUseHashNamespaceOptionKey =
         "build_property.LinqraftNestedDtoUseHashNamespace";
+    const string LinqraftUsePrebuildExpressionOptionKey =
+        "build_property.LinqraftUsePrebuildExpression";
 
     /// <summary>
     /// The namespace where global namespace DTOs should exist.
@@ -62,6 +64,15 @@ public record LinqraftConfiguration
     /// </summary>
     public bool NestedDtoUseHashNamespace { get; init; } = true;
 
+    /// <summary>
+    /// Whether to pre-build and cache Expression Trees for IQueryable operations to improve performance.
+    /// When enabled, expression trees are built at compile-time and cached as static fields, avoiding runtime construction.
+    /// Only applies to IQueryable patterns (not IEnumerable) and only for named/predefined/explicit DTO types (not anonymous types).
+    /// Note: Pre-building is not applied when there are capture variables in the lambda expression.
+    /// Default is false (disabled)
+    /// </summary>
+    public bool UsePrebuildExpression { get; init; } = false;
+
     /// <summary>
     /// Gets the actual property accessor to use based on configuration
     /// </summary>
@@ -107,6 +118,10 @@ out var arrayNullabilityRemovalStr
             LinqraftNestedDtoUseHashNamespaceOptionKey,
             out var NestedDtoUseHashNamespaceStr
         );
+        globalOptions.GlobalOptions.TryGetValue(
+            LinqraftUsePrebuildExpressionOptionKey,
+            out var usePrebuildExpressionStr
+        );
 
         var linqraftOptions = new LinqraftConfiguration();
         if (!string.IsNullOrWhiteSpace(globalNamespace))
@@ -155,6 +170,13 @@ out var commentOutputEnum
                 NestedDtoUseHashNamespace = NestedDtoUseHashNamespace,
             };
         }
+        if (bool.TryParse(usePrebuildExpressionStr, out var usePrebuildExpression))
+        {
+            linqraftOptions = linqraftOptions with
+            {
+                UsePrebuildExpression = usePrebuildExpression,
+            };
+        }
         return linqraftOptions;
     }
 }

src/Linqraft.Core/SelectExprInfo.cs

@@ -140,6 +140,16 @@ protected abstract string GenerateSelectExprMethod(
         InterceptableLocation location
     );
 
+    /// <summary>
+    /// Generates static field declarations for pre-built expressions (if enabled)
+    /// </summary>
+    public virtual string? GenerateStaticFields()
+    {
+        // Default implementation returns null (no fields)
+        // Derived classes can override this if they need static fields
+        return null;
+    }
+
     /// <summary>
     /// Generates SelectExpr code for a given interceptable location
     /// </summary>

src/Linqraft.Core/SelectExprInfoAnonymous.cs

@@ -55,6 +55,16 @@ protected override DtoStructure GenerateDtoStructure()
     // Get expression type string (for documentation)
     protected override string GetExprTypeString() => "anonymous";
 
+    /// <summary>
+    /// Generates static field declarations for pre-built expressions (if enabled)
+    /// </summary>
+    public override string? GenerateStaticFields()
+    {
+        // Anonymous types cannot use pre-built expressions because we don't know the result type at compile time
+        // The result type is inferred from the lambda and is an compiler-generated anonymous type
+        return null;
+    }
+
     // Generate SelectExpr method
     protected override string GenerateSelectExprMethod(
         string dtoName,
@@ -67,6 +77,10 @@ InterceptableLocation location
         var sb = new StringBuilder();
 
         var id = GetUniqueId();
+        
+        // Anonymous types cannot use pre-built expressions (result type is unknown)
+        // so we always use inline lambda
+        
         sb.AppendLine(GenerateMethodHeaderPart("anonymous type", location));
 
         // Determine if we have capture parameters
@@ -113,6 +127,7 @@ InterceptableLocation location
                 sb.AppendLine($"    var capture = ({captureTypeName})captureParam;");
             }
 
+            // Anonymous types always use inline lambda (cannot use pre-built expressions)
             sb.AppendLine($"    var converted = matchedQuery.Select({LambdaParameterName} => new");
         }
         else
@@ -126,6 +141,8 @@ InterceptableLocation location
             sb.AppendLine(
                 $"    var matchedQuery = query as object as {returnTypePrefix}<{sourceTypeFullName}>;"
             );
+            
+            // Anonymous types always use inline lambda (cannot use pre-built expressions)
             sb.AppendLine($"    var converted = matchedQuery.Select({LambdaParameterName} => new");
         }
 
@@ -141,6 +158,7 @@ InterceptableLocation location
             .ToList();
         sb.AppendLine(string.Join($",{CodeFormatter.DefaultNewLine}", propertyAssignments));
         sb.AppendLine("    });");
+        
         sb.AppendLine($"    return converted as object as {returnTypePrefix}<TResult>;");
         sb.AppendLine("}");
         sb.AppendLine();