Skip to content

Consolidate source + remove tests#178

Merged
fivetran-avinash merged 16 commits intomainfrom
MagicBot/consolidate-source-package
Oct 2, 2025
Merged

Consolidate source + remove tests#178
fivetran-avinash merged 16 commits intomainfrom
MagicBot/consolidate-source-package

Conversation

@fivetran-data-model-bot
Copy link
Copy Markdown
Contributor

@fivetran-data-model-bot fivetran-data-model-bot commented Sep 23, 2025

Summary

This PR is to combine the source and transform packages and remove tests not compatible with dbt Fusion 1.10.6+.

Please check the below automated updates were successful:

MANUAL UPDATES

  • If applicable, merge in the Q2 FY26 Automatic Package Updates PR.
  • If there are 2 docs.md files, consolidate the definitions the file in the main folder and remove the 2nd file.
  • Scan the README for non-standard references to the source package and remove or update.
  • Copy the passthrough vars or other non-standard vars from the source dbt_project.yml to the transform dbt_project.yml.
  • Check BK run logs for at least 2 warehouses
    • No further deprecation warnings
    • No union data "table not found" type warnings
  • Make sure below automatic updates were applied correctly.

Source + Transform Consolidation

  • Moved models/macros to models/staging/ and macros/staging/.
  • Updated all var('') references within models to use ref('stg_quickbooks__').
  • Updated vars in dbt_project.yml from refs to sources.
  • Removed the source package from packages.yml.
  • Major version bump to v1.0.0 (or higher if applicable)

YAML & Config Updates

  • Added +schema and +materialization for the staging models in dbt_project.yml.

README

  • Removed references to the source package in:
    • Overview
    • Installation instructions
    • Dependencies list
    • Schema override examples

@fivetran-data-model-bot fivetran-data-model-bot added the next-release Include in the next release. label Sep 23, 2025
@fivetran-avinash fivetran-avinash self-assigned this Sep 24, 2025
@fivetran-avinash fivetran-avinash added the docs:ready Triggers the docs generator workflow. label Sep 30, 2025
@fivetran-avinash fivetran-avinash added docs:ready Triggers the docs generator workflow. and removed docs:ready Triggers the docs generator workflow. labels Sep 30, 2025
Copy link
Copy Markdown
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-avinash a few comments change requests before approval. Thanks for working through this update!

Comment thread DECISIONLOG.md Outdated
Comment thread CHANGELOG.md Outdated
Comment thread CHANGELOG.md
Comment thread CHANGELOG.md
Comment thread CHANGELOG.md Outdated
Comment thread CHANGELOG.md Outdated
Comment thread README.md Outdated
Comment thread integration_tests/tests/consistency/consistency_general_ledger.sql Outdated
Comment thread integration_tests/tests/consistency/consistency_general_ledger.sql Outdated
@fivetran-avinash fivetran-avinash linked an issue Oct 1, 2025 that may be closed by this pull request
Copy link
Copy Markdown
Contributor

@fivetran-avinash fivetran-avinash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-joemarkiewicz Ready for re-review! One question around your limit 1 issue.

Copy link
Copy Markdown
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with small requests to bring the schema name back to the original and also regenerate the docs. Once those updates are applied, this will be ready to go!

Comment thread integration_tests/dbt_project.yml Outdated
quickbooks_source:
quickbooks_schema: quickbooks_integration_tests_21
quickbooks:
quickbooks_schema: quickbooks_integration_tests_22
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's bring this back to original schema name.

Suggested change
quickbooks_schema: quickbooks_integration_tests_22
quickbooks_schema: quickbooks_integration_tests

dbname: "{{ env_var('CI_REDSHIFT_DBT_DBNAME') }}"
port: 5439
schema: quickbooks_integration_tests_21
schema: quickbooks_integration_tests_22
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
schema: quickbooks_integration_tests_22
schema: quickbooks_integration_tests

method: service-account-json
project: 'dbt-package-testing'
schema: quickbooks_integration_tests_21
schema: quickbooks_integration_tests_22
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
schema: quickbooks_integration_tests_22
schema: quickbooks_integration_tests

database: "{{ env_var('CI_SNOWFLAKE_DBT_DATABASE') }}"
warehouse: "{{ env_var('CI_SNOWFLAKE_DBT_WAREHOUSE') }}"
schema: quickbooks_integration_tests_21
schema: quickbooks_integration_tests_22
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
schema: quickbooks_integration_tests_22
schema: quickbooks_integration_tests

dbname: "{{ env_var('CI_POSTGRES_DBT_DBNAME') }}"
port: 5432
schema: quickbooks_integration_tests_21
schema: quickbooks_integration_tests_22
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
schema: quickbooks_integration_tests_22
schema: quickbooks_integration_tests

host: "{{ env_var('CI_DATABRICKS_DBT_HOST') }}"
http_path: "{{ env_var('CI_DATABRICKS_DBT_HTTP_PATH') }}"
schema: quickbooks_integration_tests_21
schema: quickbooks_integration_tests_22
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
schema: quickbooks_integration_tests_22
schema: quickbooks_integration_tests

Copy link
Copy Markdown
Contributor

@fivetran-catfritz fivetran-catfritz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@fivetran-avinash fivetran-avinash merged commit f9c08cb into main Oct 2, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs:ready Triggers the docs generator workflow. next-release Include in the next release.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consolidate quickbooks_source package into here

4 participants