Skip to content

Frankkim/update tap to store nested data as jsonb field #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

francisace
Copy link

@francisace francisace commented Nov 1, 2023

PR Description for target-postgres

Title

Disable Table De-Nesting and Store All Nested JSON Objects in JSONB Columns

Description

Summary

This Pull Request addresses the handling of nested JSON objects in target-postgres. Instead of de-nesting the tables, all nested structures—including arrays—will now be stored directly in JSONB columns.

Motivation

Handling nested objects by creating separate tables (de-nesting) can result in a complex schema that's hard to manage and query. By storing the nested objects directly as JSONB, we simplify the schema and make it easier to query nested data.

Changes

  • Disabled the table de-nesting logic.
  • All nested JSON objects and arrays are now saved in JSONB columns.
  • Updated the test cases to reflect these changes.

How it works

Here is an example to illustrate the new behavior:

Previous Behavior

For a JSON payload like the following:

{
  "name": "John",
  "info": {
    "email": "[email protected]",
    "address": {
      "city": "NY",
      "state": "NY"
    }
  }
}

The data would be saved in multiple tables, such as main_table, info_table, and address_table.

New Behavior
Now, the data will be saved in a single table with JSONB columns:

name info
John {"email": "[email protected]", "address": {"city": "NY", "state": "NY"}}

@clarkeRS clarkeRS self-requested a review November 1, 2023 19:58
@francisace francisace merged commit ee56077 into master Nov 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants