Skip to content

Recover root-level List.of()/Stream.toList() values on deserialization - #3446

Open
seonwooj0810 wants to merge 2 commits into
spring-projects:mainfrom
seonwooj0810:fix/issue-2697-root-list-type-hint
Open

seonwooj0810 wants to merge 2 commits into
spring-projects:mainfrom
seonwooj0810:fix/issue-2697-root-list-type-hint

Conversation

@seonwooj0810

Copy link
Copy Markdown

Fixes #2697

What's wrong

GenericJacksonJsonRedisSerializer's default typing intentionally omits a type hint for a root value whose concrete type is final and lives in the java.* namespace (TypeResolverBuilder#useForType), since such types are typically private and cannot be constructed via reflection. Lists returned by List.of(...)/Stream.toList() fall into that category (ImmutableCollections$ListN/List12 etc.).

On deserialization, requesting Object.class with default typing enabled always routes JSON arrays through Jackson's AsArrayTypeDeserializer, which expects the array's first element to be a type id. For a root array that was serialized without a hint, this misreads the first data element as an unresolvable type id and fails with a MismatchedInputException/InvalidTypeIdException, even though the very same value round-trips fine once wrapped in a plain ArrayList:

RedisSerializer<Object> serializer = RedisSerializer.json(); // or GenericJacksonJsonRedisSerializer

List<Integer> value = Stream.of(2953).toList();
serializer.deserialize(serializer.serialize(value)); // throws InvalidTypeIdException
serializer.deserialize(serializer.serialize(new ArrayList<>(value))); // works fine

This is a real-world footgun with @Cacheable: a method returning list.stream().toList() (or any List.of(...)) silently breaks caching the moment the value round-trips through Redis, without any exception at write time - only at the next read.

The fix

GenericJacksonJsonRedisSerializer#deserialize now catches the MismatchedInputException that surfaces from this scenario and, only when default typing is active and the caller asked for Object.class (the common @Cacheable/untyped-cache case), attempts a narrow recovery: decompose the root JSON array and re-read each element independently through the same ObjectMapper. Nested type hints are unaffected, since only the outermost value is ever exempt from typing, so e.g. List.of(List.of(1, 2)) still recovers the inner List correctly.

To avoid masking a genuinely broken/renamed type id elsewhere, the fallback only kicks in when the source cannot possibly be a valid [typeId, value] pair - i.e. it doesn't have exactly two elements led by a JSON string. A two-element array led by a string remains ambiguous from the wire format alone (it could be a real type-wrapped value whose class was removed) and is intentionally left throwing, matching the concern a maintainer raised on the issue about not having "sufficient hooks to either always or never enable typing" for top-level arrays.

Testing

Added regression tests in GenericJacksonJsonRedisSerializerUnitTests covering:

  • The exact issue repro (Stream.toList()), plus List.of(...) and empty lists.
  • Multiple serialize/deserialize round-trips of the recovered value.
  • Nested lists, to confirm inner type hints still resolve correctly.
  • That a genuinely unresolvable type id (a two-element array led by a string) still throws SerializationException, so the fallback doesn't mask real errors.

Verification done: ran the full GenericJacksonJsonRedisSerializerUnitTests (44 tests) and RedisJsonSerializerSplitterUnitTests (85 tests, since it also exercises this serializer) via ./mvnw test -Dtest=... - all passing, no regressions. Also confirmed no in-flight PR references this issue and no maintainer/CLA/DCO blocker (DCO sign-off included below).

GenericJacksonJsonRedisSerializer's default typing intentionally omits a type hint for a root value whose concrete type is final and lives in the java.* namespace (see TypeResolverBuilder#useForType), since such types are typically private and cannot be constructed via reflection. Lists returned by List.of(...)/Stream.toList() fall into that category.

On deserialization, however, requesting Object.class with default typing enabled always routes JSON arrays through Jackson's AsArrayTypeDeserializer, which expects the array's first element to be a type id. For a root array that was serialized without a hint, this misreads the first data element as an unresolvable type id and fails with a MismatchedInputException, even though the same value round-trips fine when wrapped in a plain ArrayList.

Recover from that failure by decomposing the root JSON array and reading each element independently through the same ObjectMapper, which preserves any type hints nested one level down (only the outermost value is exempt from typing). To avoid masking a genuinely broken/renamed type id, the fallback only engages when the source cannot possibly be a valid [typeId, value] pair, i.e. it doesn't have exactly two elements led by a string.

Closes spring-projects#2697

Signed-off-by: seonwoo_jung <79202163+seonwooj0810@users.noreply.github.com>
@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Sep 17, 2026
@renechoi

Copy link
Copy Markdown
Contributor

Measured on bb552e2 vs base 0fea9fc. Three of the four added tests fail on base, so the repro holds.

root.get(0) runs before the !root.isArray() guard and ObjectNode.get(int) is null, so a cached Map.of("a",1,"b",2) now exits deserialize as a raw NPE at line 303 where base threw SerializationException, the @throws on that method. One- and three-entry maps are fine. convertValue failures escape unwrapped.

Separately, readTree costs the fidelity #3366 was declined for: List.of(map) with the deserializeRetainsNumberFidelity literal returns bigDecimal=1.0.

@seonwooj0810

Copy link
Copy Markdown
Author

Thanks for the careful re-check @renechoi — both hold up.

  1. The NPE: root.get(0) (called by the looksLikeTypeWrappedValue check) runs before the !root.isArray() guard, so for a 2-entry ObjectNode root (e.g. Map.of("a",1,"b",2)) get(int) returns null and .isString() NPEs instead of surfacing SerializationException. Fix is a straightforward guard reorder (check isArray() first).
  2. The fidelity regression: I tried the obvious fix (enabling USE_BIG_DECIMAL_FOR_FLOATS while building the recovery tree so nested BigDecimal-hinted literals keep full precision), but that turns out to convert every untyped floating literal in the recovered array into BigDecimal — including plain double fields with no type hint — which is a regression of its own and is caught by a fidelity test with mixed numeric fields. A correct fix needs the per-element re-read to distinguish "has an explicit numeric type hint" from "plain default-typed float" without reintroducing the original missing-type-id failure for nested untyped values (e.g. List.of(List.of(1,2))), and I don't yet have a version of that which passes the full existing GenericJackson2JsonRedisSerializer can't deserialize previously serialized Stream.toList() #2697 test suite. I'm not going to push a guessed patch for this — will follow up separately once I have something that verifiably passes GenericJacksonJsonRedisSerializerUnitTests end to end (including the multi-round-trip and nested-type-hint cases already covered there).

No push in this pass; flagging so it doesn't look ignored.

@mp911de mp911de added type: bug A general bug and removed status: waiting-for-triage An issue we've not yet triaged labels Sep 21, 2026
Signed-off-by: seonwooj0810 <seonwooj0810@gmail.com>
@seonwooj0810

Copy link
Copy Markdown
Author

@renechoi fixed in c07cf54. Recovery now reads each element from the original JSON tokens instead of converting tree nodes, preserving both hinted BigDecimal precision and ordinary Double values. The array guard runs before indexed access, and recovery failures are wrapped in SerializationException.

Added regressions for the two-entry map, invalid nested type IDs, and mixed numeric values in nested lists across two round trips, covering both deserialize overloads. The map and precision tests both fail on bb552e2 with the reported NPE and 1.0 result.

./mvnw -q -Dtest=GenericJacksonJsonRedisSerializerUnitTests test passes locally: 46 tests, no failures or errors.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: bug A general bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GenericJackson2JsonRedisSerializer can't deserialize previously serialized Stream.toList()

5 participants