[JsonIgnore] not working with EF Core types when requesting single resource when Lazy Loading is enabled (.NET 5)

[qwertie]

Describe the bug

When an object is serialized by ASP.NET controller methods, the [System.Text.Json.Serialization.JsonIgnore] attribute is (ironically) ignored under the following circumstances:

1. The object is returned directly, not wrapped in IEnumerable or IQueryable.

    [HttpGet("{id}")]
    public EntityDBO Get(int id) // BROKEN
    {
        return _EntityManager.GetById(id);
    }
   
    [HttpGet("{id}")]
    public IEnumerable<EntityDBO> Get(int id) // WORKS
    {
        return new[] { _EntityManager.GetById(id) };
    }
   

2. The object is an EF Core object with Lazy Loading enabled. Since the derived type is generated in a dynamic assembly in memory, I can't decompile it to investigate why this is relevant, but the mere fact that a derived class is being serialized (rather than EntityDBO itself) is insufficient to trigger the bug.

    protected override void OnConfiguring(DbContextOptionsBuilder builder)
    {
        ...
        builder.UseLazyLoadingProxies();
    }
   

3. Default JSON serialization options are used, or IMvcBuilder.AddJsonOptions is used to configure JSON (i.e. Newtonsoft is disabled)

This bug was described on StackOverflow over a year ago, but I can't find it in the issue tracker.

To Reproduce

I prepared a repo that demonstrates the problem with EF Core 5.0.4:

https://github.com/qwertie/aspnet-jsonignore-bug

To reproduce, simply run it and browse to https://localhost:44369/api/controller/1

Further technical details

ASP.NET Core version: the project file does not refer to any ASP.NET packages, but I am having this bug in a .NET 5.0 project as well as the .NET Core 3.1 test project

PM> dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.200
 Commit:    70b3e65d53

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19042
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.200\

Host (useful for support):
  Version: 5.0.3
  Commit:  c636bbdc8a

.NET SDKs installed:
  5.0.100 [C:\Program Files\dotnet\sdk]
  5.0.101 [C:\Program Files\dotnet\sdk]
  5.0.200 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

IDE: Visual Studio 2019, 16.9.0

[qwertie]

The workaround: use manual serialization and Content().

    [HttpGet("{id}")]
    public ActionResult Get(int id)
    {
        var result = _context.Examples.Find(id);
        return Content(JsonSerializer.Serialize(result), "application/json");
    }

Don't forget to provide the same serializer options you configured via AddJsonOptions in Startup (if any)

[pranavkm]

The output from the app you provided seems fine:

{"child":{"id":1,"name":"Child!"},"lazyLoader":{},"id":1,"good":"Good!","childId":1}

What am I missing?

[qwertie]

I am surprised to see that the first "Ignored" field is not showing up. I didn't notice that before.

However the other property Child is JsonIgnored and still appearing.

    [System.Text.Json.Serialization.JsonIgnore]
    public virtual ExampleChild Child { get; set; }

The workaround produces the intended output:

{"Id":1,"Good":"Good!","ChildId":1}

This leads me to a new hypothesis, which I tested as follows:

[HttpGet("{id}")]
public ActionResult Get(int id)
{
    var result = _context.Examples.Find(id);
    return Content(JsonSerializer.Serialize<object>(result), "application/json");
}

This produces the incorrect output

{"child":{"id":1,"name":"Child!"},"lazyLoader":{},"id":1,"good":"Good!","childId":1}

My interpretation is that

1. ASP.NET is using Serialize<object> rather than Serialize<controller's return type>
2. EF Core generates a proxy override ExampleChild Child property which lacks the JsonIgnore attribute (or so I assume, as it is not possible for developers to view or decompile the derived class!)
3. System.Text.Json sees the derived class's lack of JsonIgnore and treats the property as not ignored

None of these behaviors, on its own, looks like a bug, but in combination, the unwanted behavior occurs.

I'm pretty sure that manual serialization in the controller will prevent the OData service from working. Is there another workaround that would not have this drawback?

[ghost]

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

[lus]

Are there any updates on this? I have the exact same issue (lazyLoader is not ignored under these circumstances, using STJ) and nearly 2 years passed since this report.

In my concrete case, the issue was that Ok(object) seems to use JsonSerializer.Serialize(object), rather than JsonSerializer.Serialize<T>(T).
I could reproduce the issue by comparing the results of these two functions.

I created a temporary FixedOk<T>(T) method like this:

private ContentResult FixedOk<T>(T obj)
{
    return Content(JsonSerializer.Serialize(obj, _jsonSerializerOptions), "application/json");
}

where _jsonSerializerOptions gets passed via DI.
Please note that the generic type argument is mandatory here in order to implicitly use JsonSerializer.Serialize<T>(T) and not JsonSerializer.Serialize(object).

[noontz]

Can confirm this bug is now 2 years old, and alive and thriving as ever in .NET 6 / Azure Functions

@msftbot That's a nice 1 year old dusty link to triage there. Could you maybe provide a link to wherever you stoved this bug away?

[mitchdenny]

@ajcvickers interested in your thoughts here?

[noontz]

Now there's life in this thread, I'd like to add to the scope, that the workaround, in my case, is adding BOTH System.Text.Json.Serialization.JsonIgnoreAttribute and System.Runtime.Serialization.IgnoreDataMemberAttribute. The reason being that System.Text.Json.JsonSerializer is immune to System.Runtime.Serialization.IgnoreDataMemberAttribute, but I use JsonSerializer in tests. Another issue is Refit that does not see System.Runtime.Serialization.IgnoreDataMemberAttribute either, so the choice of implementation is between inconsistent code (using a different attribute according to context where possible) or use both attributes. Both solutions are far from optimal.

[mitchdenny]

To be clear, this isn't an ASP.NET specific issue. The interaction of EF generated proxies and the JSON serializer exist outside the context of ASP.NET. The only flavor that ASP.NET adds to this discussion is some indirection that can change what serializer options come into play. How Refit intersects with this is even further removed.

But let's continue here for a little while and avoid conways law for a little while longer :) If @ajcvickers chimes in we might be able to make a case to move this over to the efcore repo for further conversation.

The fundamental reason we see this behaviour is that when EF core has lazy loading enabled it makes use of a proxy. That proxy overrides the implementation of the navigation properties (Child in this case) and also exposes an additional field/property (lazyloader). These show up in the JSON serialized representation of the proxy because the serializer cannot see a [JsonIgnore] attribute on these properties (you can use the Immediate window in VS to use reflection to verify this for yourself).

I'm not sure if the EF team has some clever ideas on how to work around this, but I think that we are unlikely to see a solution on the other side where STJ somehow becomes aware of Castle-based proxies and looks for the base type to infer how to ignore properties. You can make the case by filing an issue in the dotnet/runtime repo.

So the reality is that lazy loading and JSON serialization don't mix well together today - not a bug, just merely the absence of a feature.

[noontz]

@mitchdenny Thanks for the info! Assuming you mean Entity Framework when you mention EF, I can inform that I don't use EF. I see this issue when returning async System.Threading.Tasks.Task< Microsoft.AspNetCore.Mvc.ActionResult<MyType>> from an Azure function, and a property on MyType has been decorated with System.Text.Json.Serialization.JsonIgnoreAttribute.

[ajcvickers]

There is a comment above which sums up this issue nicely:

1. ASP.NET is using Serialize<object> rather than Serialize<controller's return type>
2. EF Core generates a proxy override ExampleChild Child property which lacks the JsonIgnore attribute (or so I assume, as it is not possible for developers to view or decompile the derived class!)
3. System.Text.Json sees the derived class's lack of JsonIgnore and treats the property as not ignored

None of these behaviors, on its own, looks like a bug, but in combination, the unwanted behavior occurs.

I don't think there is much can be done about this. If the code uses a proxy, but the serializer should treat it as a non-proxy for serialization purposes, then that is done by using the type with the generic method.

@noontz I don't think you are describing the same issue as is being discussed here.

[noontz]

@ajcvickers "I don't think you are describing the same issue as is being discussed here".. You are absolutely correct. Sorry for the stir.. Google sent me here as System.Text.Json.Serialization.JsonIgnoreAttribute was being ignored, and I pulled the trigger way to fast in the wrong context.. This seems to be a rather conceptual issue across technologies?

[mitchdenny]

I'm going to resolve this as by design for the moment then.

[ghost]

This issue has been resolved and has not had any activity for 1 day. It will be closed for housekeeping purposes.

See our Issue Management Policies for more information.