|
| 1 | +# Migrate to Jackson 3 (`tools.jackson`) — JSONata4Java 3.0.0 |
| 2 | + |
| 3 | +**Issue:** [#430](https://github.com/IBM/JSONata4Java/issues/430) — Migrate code from |
| 4 | +`com.fasterxml.jackson.core` to `tools.jackson.core`. |
| 5 | + |
| 6 | +## Summary |
| 7 | + |
| 8 | +Jackson 3.0 renamed both its Maven group-ids and Java packages from `com.fasterxml.jackson` |
| 9 | +to `tools.jackson`, made the core exception hierarchy unchecked, and moved mapper |
| 10 | +configuration to an immutable builder pattern. JSONata4Java exposes |
| 11 | +`com.fasterxml.jackson.databind.JsonNode` throughout its public API, so migrating to Jackson 3 |
| 12 | +is a breaking change and warrants a new major version. |
| 13 | + |
| 14 | +The project's `pom.xml` already stages the transition by listing both the Jackson 2.x and |
| 15 | +Jackson 3.x dependencies. This work completes the cutover: remove the 2.x stack, migrate all |
| 16 | +source and test code to `tools.jackson`, and release as **3.0.0**. |
| 17 | + |
| 18 | +## Decisions |
| 19 | + |
| 20 | +- **Strategy: hard cutover.** Replace all `com.fasterxml.jackson.*` usage with `tools.jackson.*` |
| 21 | + and drop the 2.x dependencies entirely. No dual-support/abstraction layer (JsonNode is woven |
| 22 | + too deeply into the public API to make that worthwhile — YAGNI). |
| 23 | +- **Version: 3.0.0.** Bump from `2.6.4`. The public-API break (JsonNode package change) requires |
| 24 | + a major version. |
| 25 | +- **XML in scope.** Port the `XmlMapper`/`ToXmlGenerator` usage to Jackson 3's |
| 26 | + `tools.jackson.dataformat.xml`. |
| 27 | +- **CLI/UI helpers in scope.** `Tester`, `TesterTimeBox`, `TesterUI`, and `Test` are migrated |
| 28 | + along with the core library, even though they carry most of the API churn. |
| 29 | +- **Target dependency version: 3.2.0** (current Jackson 3 line; confirmed published on Maven |
| 30 | + Central for both `jackson-databind` and `jackson-dataformat-xml`). |
| 31 | + |
| 32 | +## Scope |
| 33 | + |
| 34 | +103 Java files reference Jackson today: 87 under `src/main/java`, 16 under `src/test/java`. |
| 35 | +The overwhelming majority only use JSON node types whose class names are unchanged in Jackson 3 |
| 36 | +— only the package prefix moves. |
| 37 | + |
| 38 | +## Dependency changes (`pom.xml`) |
| 39 | + |
| 40 | +| Action | Coordinates | |
| 41 | +| --- | --- | |
| 42 | +| Remove | `com.fasterxml.jackson.core:jackson-databind:2.22.0` | |
| 43 | +| Remove | `com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.22.0` | |
| 44 | +| Keep | `tools.jackson.core:jackson-databind:3.2.0` (already present) | |
| 45 | +| Add | `tools.jackson.dataformat:jackson-dataformat-xml:3.2.0` | |
| 46 | +| Keep | `com.fasterxml.woodstox:woodstox-core:7.2.1` (Jackson 3 XML still uses Stax/woodstox) | |
| 47 | +| Keep | `re2j` (test), `gson`, `commons-text`, `antlr4-runtime`, `spring-context` (test), `junit` (test) | |
| 48 | +| Change | `<version>` `2.6.4` → `3.0.0` | |
| 49 | + |
| 50 | +Notes: |
| 51 | +- **Version string lives in more than `pom.xml`.** Per `ReleaseProcedure.txt`, the release |
| 52 | + version appears in six places across five files, all of which move `2.6.4` → `3.0.0`: |
| 53 | + `pom.xml` (1), `README.md` (3 — lines 17, 54, 55), `tester.sh` (1), `testerui.sh` (1), |
| 54 | + `testerui.cmd` (1). The script/README occurrences are the `JSONata4Java-<version>...jar` names. |
| 55 | +- `jackson-core` is pulled in transitively by `jackson-databind`; no direct entry needed. |
| 56 | +- **`jackson-annotations` was NOT renamed** — it stays under `com.fasterxml.jackson.annotation`. |
| 57 | + The codebase has no annotation imports, so this is a non-issue here, but do not blindly |
| 58 | + rewrite `com.fasterxml.jackson.annotation` should it appear. |
| 59 | + |
| 60 | +## Migration approach |
| 61 | + |
| 62 | +Execute in three passes so the risky hand-edits stay isolated from the bulk mechanical change. |
| 63 | + |
| 64 | +### Pass 1 — Mechanical package rename (~103 files) |
| 65 | + |
| 66 | +Scripted find-replace across `src/main/java` and `src/test/java`, longest-prefix first to avoid |
| 67 | +double rewriting: |
| 68 | + |
| 69 | +1. `com.fasterxml.jackson.dataformat.xml` → `tools.jackson.dataformat.xml` |
| 70 | +2. `com.fasterxml.jackson.databind` → `tools.jackson.databind` |
| 71 | +3. `com.fasterxml.jackson.core` → `tools.jackson.core` |
| 72 | + |
| 73 | +This covers all the node types (`JsonNode`, `JsonNodeFactory`, `ArrayNode`, `ObjectNode`, |
| 74 | +`TextNode`, `LongNode`, `IntNode`, `DoubleNode`, `FloatNode`, `BooleanNode`, `NullNode`, |
| 75 | +`POJONode`, `ValueNode`, `NumericNode`, `JsonNodeType`) and the streaming/databind entry points, |
| 76 | +all of which keep their simple class names in Jackson 3. |
| 77 | + |
| 78 | +### Pass 2 — Hand-fix API-change hotspots (< 10 files) |
| 79 | + |
| 80 | +These sites use APIs that changed shape in Jackson 3 and will not compile after a pure rename. |
| 81 | + |
| 82 | +**Mapper configuration → immutable builder.** In Jackson 3, mappers are immutable; the mutating |
| 83 | +`enable`/`configure`/`getFactory().configure(...)` methods are gone. All affected sites are in |
| 84 | +CLI/UI helpers, not the core library: |
| 85 | + |
| 86 | +- `Tester.java:124` — `mapper.getFactory().configure(JsonWriteFeature.ESCAPE_NON_ASCII.mappedFeature(), true)` |
| 87 | + → construct via `JsonMapper.builder().enable(JsonWriteFeature.ESCAPE_NON_ASCII).build()`. |
| 88 | + `JsonWriteFeature` moves to `tools.jackson.core`. |
| 89 | +- `TesterUI.java:108` — `xmlMapper.enable(SerializationFeature.INDENT_OUTPUT)` |
| 90 | + → set in the `XmlMapper.builder()...build()` chain. |
| 91 | +- `TesterUI.java:488–491` — `xmlMapper.configure(SerializationFeature.INDENT_OUTPUT, true)`, |
| 92 | + `xmlMapper.configure(ToXmlGenerator.Feature.WRITE_XML_DECLARATION, true)`, |
| 93 | + `xmlMapper.configure(ToXmlGenerator.Feature.WRITE_XML_1_1, true)` |
| 94 | + → fold into the `XmlMapper.builder()` chain (verify the `ToXmlGenerator.Feature` enum location |
| 95 | + under `tools.jackson.dataformat.xml`). |
| 96 | + |
| 97 | +**Plain mapper construction.** Sites that only call `readTree` / `writeValueAsString` on a |
| 98 | +`new ObjectMapper()` (in `Binding`, `ExpressionsVisitor`, `ContainsFunction`, `Expression`, |
| 99 | +`JSONataUtils`, `Issue180`, `Test`) need only the package rename; keep `new ObjectMapper()` (or |
| 100 | +switch to `JsonMapper.shared()` / `new JsonMapper()` if `new ObjectMapper()` proves unavailable). |
| 101 | +The exact constructor availability is confirmed at compile time in Pass 3. |
| 102 | + |
| 103 | +**Unchecked exceptions.** `JsonProcessingException`'s parent became `RuntimeException` and core |
| 104 | +introduced unchecked `JacksonException`. Catch blocks referencing `JsonProcessingException` |
| 105 | +generally still compile because it is now unchecked. Where a `catch (JsonProcessingException e)` |
| 106 | +becomes unreachable, or the type is no longer resolvable, replace with |
| 107 | +`tools.jackson.core.JacksonException`. Review each multi-catch |
| 108 | +(`EvaluateException | JsonProcessingException`) individually. Affected files: `TesterTimeBox`, |
| 109 | +`Test`, `Tester`, `TesterUI`, `ExpressionsVisitor`, `Issue180`. |
| 110 | + |
| 111 | +### Pass 3 — Compile, test, verify |
| 112 | + |
| 113 | +- `mvn clean compile` — resolve any remaining API breaks surfaced by the compiler; iterate until |
| 114 | + clean. |
| 115 | +- `mvn test` — the existing unit tests plus `AgnosticTestSuite` are the correctness safety net; |
| 116 | + the 16 migrated test files run here too. |
| 117 | +- Manually exercise the `Tester` CLI (JSON read/pretty-print path) and, where feasible, the Swing |
| 118 | + `TesterUI` XML↔JSON conversion, since those hold the most API churn. |
| 119 | + |
| 120 | +## Build configuration |
| 121 | + |
| 122 | +- **OSGi (`bnd-maven-plugin`).** The `Export-Package` list references only `com.api.jsonata4java.*` |
| 123 | + and needs no change. Confirm the generated `MANIFEST.MF` `Import-Package` header now imports |
| 124 | + `tools.jackson.*` rather than `com.fasterxml.jackson.*`. |
| 125 | +- **JPMS.** No `module-info.java` exists; no module changes required. |
| 126 | +- No other plugin configuration references Jackson. |
| 127 | + |
| 128 | +### Maven build-warning cleanup (unrelated to Jackson, done on this branch) |
| 129 | + |
| 130 | +While migrating, three pre-existing Maven warnings surfaced on every build and were fixed in |
| 131 | +`pom.xml`: |
| 132 | + |
| 133 | +- **bnd private-reference warnings (×2).** The exported `com.api.jsonata4java` and |
| 134 | + `com.api.jsonata4java.expressions` packages expose types from |
| 135 | + `com.api.jsonata4java.expressions.regex` (`RegexEngine`, `RegexPattern`, …) in their public API, |
| 136 | + but that sub-package was not in the `Export-Package` list, so bnd flagged it as a leaked private |
| 137 | + reference. Added `com.api.jsonata4java.expressions.regex` to `Export-Package`. |
| 138 | +- **`additionalClasspathElements` unknown (×2).** Removed the empty |
| 139 | + `<additionalClasspathElements/>` from `maven-compiler-plugin` — not a valid parameter for that |
| 140 | + plugin (it did nothing). |
| 141 | +- **`excludes` unknown on assembly plugin.** Removed `<excludes>` from the `maven-assembly-plugin` |
| 142 | + `single` goal — not a valid parameter there (exclusions belong in an assembly descriptor); it was |
| 143 | + silently ignored and does not affect the produced `jar-with-dependencies`. |
| 144 | + |
| 145 | +A fourth warning — `gpg.passphrase` deprecated — originates from the developer's |
| 146 | +`~/.m2/settings.xml` (an `activeByDefault` profile setting the deprecated property), **not** from |
| 147 | +`pom.xml`, and is resolved locally by removing that property and relying on `gpg-agent`. |
| 148 | + |
| 149 | +## Out of scope |
| 150 | + |
| 151 | +- Any behavioral changes beyond what the package/API migration requires. |
| 152 | +- Adopting new Jackson 3 features (e.g., new default flips such as `FAIL_ON_UNKNOWN_PROPERTIES`) |
| 153 | + beyond preserving existing behavior. If a Jackson 3 default change alters observed test |
| 154 | + behavior, restore the prior behavior via builder configuration rather than accepting the new |
| 155 | + default silently. |
| 156 | +- Unrelated refactoring of the touched files. |
| 157 | + |
| 158 | +## Risks / watch-items |
| 159 | + |
| 160 | +- **Default-value flips in Jackson 3** (e.g., `WRITE_DATES_AS_TIMESTAMPS`, `FAIL_ON_UNKNOWN_PROPERTIES`) |
| 161 | + could shift test output. The test suite is the detector; restore prior behavior via builder |
| 162 | + config where a test regresses. |
| 163 | +- **`ToXmlGenerator.Feature` and `JsonWriteFeature` relocations** — confirm exact package/enum |
| 164 | + homes at compile time; the builder `enable(...)` overloads may differ from the 2.x |
| 165 | + `configure(feature, boolean)` shape. |
| 166 | +- **`new ObjectMapper()` availability** — if the no-arg constructor is unavailable/discouraged in |
| 167 | + 3.x, fall back to `JsonMapper.shared()` or `new JsonMapper()`. |
| 168 | + |
| 169 | +## Retained Jackson 2 behaviors (as-built) |
| 170 | + |
| 171 | +Jackson 3 tightened several coercion/parse defaults that would otherwise change observable |
| 172 | +behavior. Each site below was fixed to **preserve the Jackson 2 behavior** — no test assertion or |
| 173 | +expected value was changed. These are the compatibility decisions a future maintainer should know |
| 174 | +about (and can revisit if the project decides to adopt the stricter Jackson 3 semantics): |
| 175 | + |
| 176 | +- **Container coercion — `ArrayUtils.compare` (main).** Jackson 3's no-arg `asText()`/`asString()` |
| 177 | + throws for container nodes (`ObjectNode`/`ArrayNode`); Jackson 2 returned `""`. Fixed by passing |
| 178 | + the default: `asText("")`. `$sort` on non-scalar operands therefore coerces to `""` exactly as |
| 179 | + before instead of throwing. |
| 180 | +- **Regex `POJONode` emptiness checks — `MatchFunction`, `ReplaceFunction` (main).** A compiled |
| 181 | + `RegularExpression` is stored in a `POJONode`. Jackson 2's `asText()` returned the POJO's |
| 182 | + `toString()` (non-empty); Jackson 3 throws for a `POJONode`. The empty-pattern checks are now |
| 183 | + guarded by `isTextual()` so the throwing call is never reached for a `POJONode`, while the |
| 184 | + boolean outcome is identical to Jackson 2. |
| 185 | +- **Out-of-range `asLong()` — `AgnosticTestSuite` (test harness).** Jackson 3's `DoubleNode.asLong()` |
| 186 | + throws when the value is outside `long` range (e.g. `1.0E46`); Jackson 2 performed a silent |
| 187 | + narrowing cast. Expected-value normalization now uses `(long) d`, replicating Jackson 2 exactly. |
| 188 | +- **`FAIL_ON_TRAILING_TOKENS` — `Utils` test mapper (test harness).** Jackson 3 flips this default |
| 189 | + to `true`; Jackson 2 silently ignored content after the first parsed value. The shared test |
| 190 | + mapper disables the feature so existing test inputs parse identically. Note: this tolerates a |
| 191 | + long-standing typo in a test input (`BasicExpressionsTests`, `"[\"h11\", \"h21\"]]"` — a stray |
| 192 | + trailing `]`). The typo was intentionally **not** corrected, to keep the test data byte-for-byte |
| 193 | + unchanged. |
| 194 | +- **Unchecked exceptions — several files.** Where Jackson 3 methods no longer throw checked |
| 195 | + `IOException`, `catch (IOException)` blocks over Jackson-only bodies became unreachable and were |
| 196 | + changed to `catch (JacksonException)`. |
| 197 | + |
| 198 | +### Other known items (not changed by this migration) |
| 199 | + |
| 200 | +- **Pre-existing `Tester` REPL NPE.** `Tester.main` (`Tester.java:156`) calls |
| 201 | + `expression.length()` without a null check; `JSONataUtils.prompt()` returns `null` at stdin EOF, |
| 202 | + causing a `NullPointerException` when input ends without a `q` line. This predates the migration |
| 203 | + (authored 2022) and was left as-is. |
| 204 | +- **Transitive `jackson-annotations` 2.x.** `tools.jackson.core:jackson-databind:3.2.0` still pulls |
| 205 | + in `com.fasterxml.jackson.core:jackson-annotations` — Jackson 3 continues to publish annotations |
| 206 | + under the `com.fasterxml.jackson.annotation` namespace. This is expected and acceptable; it is not |
| 207 | + a leftover Jackson 2 core/databind/dataformat dependency. |
0 commit comments