Proposal: introduce Schema.NullableMode enum to mirror RequiredMode / AccessMode pattern

[thejeff77]

Background

swagger-core has twice introduced enum-based "Mode" replacements for boolean fields on @Schema whose default false made it impossible to distinguish "user explicitly set false" from "not specified":

• 2018 — AccessMode { AUTO, READ_ONLY, WRITE_ONLY, READ_WRITE } introduced (refs #2379 - deprecates ApiModelProperty.readOnly and introduces accessMode field #2675, refs #2379 (2.0) - deprecates Schema.readOnly and introduces accessMode field #2677), deprecating readOnly and writeOnly.
• 2022 — RequiredMode { AUTO, REQUIRED, NOT_REQUIRED } introduced (feat: replace required attribute on @Schema by requiredMode #4221, feat: replace required attribute on @Schema by requiredMode #4286, commit b1729fc), deprecating required. Commit message rationale: "so we can have a property annotated @NotNull but still have required = false in the openapi spec."

The nullable field has the same shape — boolean nullable() default false — and is now hitting the same problem class as required did pre-2022.

Why now

Auto-detection of nullable has expanded substantially in the last year:

• 5001: Add support for @Nullable annotations in OpenAPI 3.1 schemas #5018 (Nov 2025) added native auto-detection of @Nullable annotations (Jakarta / Spring / JetBrains / JSR-305) with OAS 3.1 type-array translation.
• Downstream, springdoc-openapi added auto-detection of Kotlin T? types in feat: auto-set nullable: true for Kotlin nullable types in schema properties springdoc/springdoc-openapi#3256 (Apr 2026), which has now been reverted in revert: auto-set nullable for Kotlin nullable types (#3256) springdoc/springdoc-openapi#3276 due to the same missing-override problem this proposal addresses.
• Recent bug iterations on the auto-detection edges: [Bug]: Issues using @Nullable annotations in OpenAPI 3.1 schemata #5077, [Bug]: Wrong schema for nullable object #5158, upcoming fix: ensure that @Nullable does not incorrectly affect object schemas #5124.

Both auto-detection paths feed the same Schema.nullable boolean and have no way to express "user explicitly opts this property out of being marked nullable, regardless of heuristics." Users who try @Schema(nullable = false) find it has no effect (it's the default value and reflection can't distinguish it from absence). See springdoc/springdoc-openapi#3138 and the discussion under springdoc/springdoc-openapi#3256 for concrete user reports.

Proposal

Add to Schema:

/**
 * Allows specifying the nullable mode (NullableMode.AUTO, NULLABLE, NOT_NULLABLE).
 *
 * NullableMode.AUTO: lets the library decide based on heuristics (e.g. @Nullable
 *   annotations, downstream Kotlin T? detection in springdoc).
 * NullableMode.NULLABLE: forces the item to be considered nullable regardless
 *   of heuristics.
 * NullableMode.NOT_NULLABLE: forces the item to be considered not nullable
 *   regardless of heuristics.
 */
NullableMode nullableMode() default NullableMode.AUTO;

enum NullableMode {
    AUTO,
    NULLABLE,
    NOT_NULLABLE;
}

Deprecate boolean nullable() in favor of nullableMode(), mirroring the deprecation pattern used for required and readOnly/writeOnly.

Precedence

Following the precedent set in #4533 ("give precedence to requiredMode annotation"), nullableMode should take precedence over the legacy nullable boolean and over auto-detection heuristics. Suggested precedence:

1. nullableMode = NULLABLE or NOT_NULLABLE → use that
2. Legacy nullable = true (deprecated path) → treat as NULLABLE
3. nullableMode = AUTO (default) → run heuristics (@Nullable annotations, downstream Kotlin reflection, etc.)

OAS 3.0 vs 3.1 mapping

Same as the existing handling for nullable: true:

• OAS 3.0: sets nullable: true on simple types; for $ref, wraps in allOf with nullable: true sibling.
• OAS 3.1: adds "null" to the type array; for $ref, wraps in oneOf with { "type": "null" }.

NOT_NULLABLE simply suppresses any of the above, regardless of what heuristics would otherwise have applied.

Impact on downstream auto-detection

The @Nullable auto-detection added in #5018 should defer to nullableMode != AUTO. Likewise, downstream libraries (springdoc's planned re-introduction of KotlinNullablePropertyCustomizer) can defer to the same field. This gives users a single, uniform escape hatch across all auto-detection sources.

Happy to follow up with a PR

If this direction is acceptable, I'm willing to put up a PR following the pattern of #4221 (RequiredMode introduction) and #4533 (precedence handling). Confirming intent here first since this is an annotation-surface change.

[thejeff77]

PR opened: #5161 — implementation, tests, and full passing existing test suite. Happy to iterate based on review.