feat: support new CDC keys structure#46
feat: support new CDC keys structure#46venikkin merged 5 commits intoneo4j:mainfrom venikkin:change-cdc-key-structure
Conversation
| SchemaBuilder.struct() | ||
| .apply { | ||
| keys.forEach { key -> | ||
| key.forEach { field(it.key, DynamicTypes.schemaFor(it.value, true)) } | ||
| } | ||
| } | ||
| .optional() | ||
| .build()) |
There was a problem hiding this comment.
I got a bit confused about having keys as an array of structs, probably worth commenting on this.
There was a problem hiding this comment.
Sure. Before the change, each key, e.g. Map<String, Object> was mapped to a struct (struct of struct if we talk about about key per label). Now we have multiple keys, so I just wrapped the struct representing a key with an array. However, to reflect that each element if the array can have different structure, we have to define all possible key name in struct. For example: [ { "id": 1 }, { "name": "Joe" } ] corresponds to
array(struct {
optional int id,
optional string name
})
Optional because id is null in the second element and name is null in the first element. For Avro value will like like (preudo-Avro here, just to show the structure): [{id: 1, name: null}, {id: null, name: "Joe"}].
Alternative option would be something like
array(struct {
string key,
optional int intValue,
optional string stringValue
})
with corresponding value [{"key": "id", "intValue": 1, "stringValue": null}, {"key": "name", "intValue": null, "stringValue": "Joe"}]
Maybe there could be a better option. Let's discuss.
| SchemaBuilder.map( | ||
| Schema.STRING_SCHEMA, Schema.STRING_SCHEMA) | ||
| .build()) |
There was a problem hiding this comment.
This should be a struct, right?
There was a problem hiding this comment.
I can be a struct, but I left it as map, just wrapped it in array. Let's discuss it we should change it to struct.
| SchemaBuilder.array( | ||
| SchemaBuilder.map( | ||
| Schema.STRING_SCHEMA, Schema.STRING_SCHEMA) | ||
| .build()) |
2 participants
Uh oh!
There was an error while loading. Please reload this page.